This is a blog for everyday techies building real systems people use, and their still-with-it-and-technical CTOs. It's a blog for thinking people who want to build better systems. That often requires questioning current practices.
I build things. Mostly, I build software, typically backed by some principled reason for why it should work as advertised. I'm currently a professor at Cornell University.
These days, I work mostly on data stores of different types. My team built the HyperDex fast data store. It's a next-generation NoSQL database that is faster than first-gen entrants like Mongo, supports ACID transactions, and scales. We recently released Weaver, a graph store that can hold very large graphs that are dynamically changing.
I have worked to improve Bitcoin in various ways. Ittay Eyal and I co-authored the Selfish Mining paper, which showed that some commonly held folk theorems about Bitcoin were false, and that the network needed at least 2/3rds of the participants to be honest. We also provided a fix against this attack. I served as an expert witness in the U.S. lawsuit on behalf of the people who lost money at Mt. Gox. I was one of the main people to push back on GHash at the first coin summet to get them to back down from 51% of the hash power. Ittay Eyal and I have proposed 2P-PoW, a practical non-outsourcable puzzle scheme that can limit the size of open mining pools, while allowing individual miners to reduce their variance. I'm a huge Bitcoin fan and I work hard to clarify misconceptions related to Bitcoin, though I also call out various mistakes I see the community making, such as attacking people who are working to make the system better, refusing to be scientific, and doing things that, in their zeal, actually interfere with adoption by regular people.
In the past, I worked on peer-to-peer systems of various kinds. I was behind Karma, the first peer-to-peer currency with a distributed mint, to my knowledge. My group also built Credence, a peer-to-peer reputation scheme that counteracted spam and mislabeled files. I have built many other academic peer-to-peer systems, and I have worked to defend some of the most popular ones in court.
My original passion was in operating systems. I built the SPIN kernel when I was a graduate student, where we explored how to extend the interface and implementation of a running kernel, safely, at run time. More recently, I built the Nexus, an operating system for trustworthy computing, which can prove to others what behaviors a computer will and will not engage in, without revealing what code it is running.
Overall, I hope you find the discussion of distributed systems, operating systems and systems engineering to be thought provoking. If you are an undergraduate and find these topics interesting, you should consider further study -- almost every top notch school will have a masters program with courses on distributed systems, and if you're particularly daring, a PhD program where you get to advance the state of the art. If you are not thinking of pursuing graduate studies, I'll try to provide appropriate links to external sources so you can follow up on the tangents that you find interesting. If you have other questions, feel free to contact me via email.
I wrote up my goals in starting a blog. There is a lot of noise and misinformation being propagated online. Distributed systems is at an inflection point and taking off rapidly. I worry that we might have to face broken, flaky and untrustworthy systems for decades to come.
There are companies which spend millions of dollars on PR, bankrolling conferences and hiring salesmen to go from door to door in the valley to spread misinformation and peddle broken software. I have nothing that can compare -- certainly no PR engine or paid stooges whose paycheck depends on perpetuating myths and foggy thinking. What I do have is a grasp of distributed systems science (albeit partial, as no one can claim complete mastery over such a large area), and a willingness to write for a general audience.
Some provide enlightenment, others provide intrigue, and some are there to get my pulse up in the mornings.
- HyperDex A next-generation NoSQL store that provides ACID transactions, strong consistency and fault tolerance guarantees, along with a rich API. And it outperforms first-generation data stores.
- Weaver A fast and scalable data store for graph-structured data.
- OpenReplica An open object replication service. An easier to use alternative to ZooKeeper and Chubby.
- Nexus An operating system that can provide strong guarantees about the properties of its applications.