This is a blog for everyday techies building real systems people use, and their still-with-it-and-technical CTOs.
It should be clear to everyone that I do not regurgitate the standard blogospheric dreck. If anyone is here to hear what they already "know," they're in the wrong place. There are so many other outlets that already do that kind of regurgitation that it would be a waste of my readers' and my own time to join the fray. So I hope that my readers will find some content here that challenges widely-held beliefs. The more widely-held, the more pervasive a falsehood, the greater the need for someone to say something about it. So, I will pick on cargo-cult engineering and muddled thinking in all their forms. The people who cannot handle being challenged can always go to a safe corner of the Internet (sadly, I learned that Joel on Software is no longer active, but there are many others) and discuss convenient pablum such as agile strategy, why Obama should amend the Constitution to make Google bring Reader back, how the CAP Theorem implies that HyperDex, Calvin and Google's Spanner cannot possibly exist despite the existence proofs in code repositories, or whatever other confused bandwagon these non-coding hackers and broengineers might be riding that day. This is a blog for thinking people who want to build better systems, and that often requires questioning current practices.
I tried to clearly enunciate my goals in starting a blog. There is a lot of noise and misinformation being propagated online. Distributed systems is at an inflection point and taking off rapidly. I worry that unless we establish sound frameworks, we might have to face broken, flaky and untrustworthy systems for decades to come.
There are companies which spend millions of dollars on PR, bankrolling conferences and hiring salesmen to go from startup to startup in order to spread misinformation and peddle broken software. I have nothing that can compare -- certainly no PR engine or paid stooges whose paycheck depends on perpetuating myths and foggy thinking. What I do have is a grasp of distributed systems science (albeit partial, as no one can claim complete mastery over such a large area), and a willingness to write for a general audience.
I've found that this unnerves the very people spreading misinformation: they want their professors obscure, unclear and limited to an academic audience. This blog is not going to be a dry, academic exercise. I will not prove any theorems. I will do my best to avoid dropping either industry or academic jargon. I certainly will not get into boring meta-topics, such as academic publishing or how one should run massive online courses. I am passionate about distributed systems, and I will use accessible metaphors to make a point. So, I fully expect that some people will be reading this blog, seething with anger at something I wrote about their system or broken world view, and demand that I write something "more academic" or whatever their favorite euphemism is for making oneself less accessible and relevant. Or they'll get outraged that I do not toe the carefully-established lines established by PR engines, even if, or especially because, they are incorrect? These people are being challenged and just don't know how to respond with technical content.
In turn, academics might note my skepticism for much of the academic literature. All too often, academics chase phantom problems that don't exist; go after the easy, the incremental or the esoteric; and try to compete with industry on things that are best solved within industry. So if I offend someone's academic sensibilities by insinuating that, for instance, anonymous ring theory is a complete waste of time, well, tough.
Overall, I hope you find the discussion of distributed systems, operating systems and systems engineering to be thought provoking. If you are an undergraduate and find these topics interesting, you should consider further study -- almost every top notch school will have a masters program with courses on distributed systems, and if you're particularly daring, a PhD program where you get to advance the state of the art. If you are not thinking of pursuing graduate studies, I'll try to provide appropriate links to external sources so you can follow up on the tangents that you find interesting. If you have other questions, feel free to contact me via email.
Some provide enlightenment, others provide intrigue, and some are there to get my pulse up in the mornings.
HyperDex A second-generation NoSQL store that provides ACID transactions, strong consistency and fault tolerance guarantees, along with a rich API. And it outperforms first-generation data stores.
OpenReplica An open object replication service. An easier to use alternative to ZooKeeper and Chubby.
Nexus An operating system that can provide strong guarantees about the properties of its applications.