Miniature is an art form that tries to capture the reality from different viewpoints in one painting. (Artist: Matrakçı Nasuh, Date: 16th century)
Blockchains provide trustless auditability and many innovative use cases, which has led to a flurry of different blockchain proposals. This brings up the question of how to evaluate different blockchain platforms. How do we determine, scientifically, whether a blockchain protocol lives up to its promises? How do we perform comparative evaluations of blockchain protocols?
Clearly, we need an evaluation platform. An ideal platform would have the following properties:
The miniature version of the Bitcoin network resides in a server room in Ithaca, NY.
To achieve these goals, we are working on an evaluation platform called Miniature World. The vision for this project is to build the tools to create a virtual copy of an actual blockchain deployment in a controlled datacenter setting. For instance, we want to create a one-to-one replica of the Bitcoin network, where for every Bitcoin node out there in the wild, we have a corresponding node in our datacenter, in a controlled network that reflects real world conditions. We can then evaluate proposals within this Miniature World, and even run what-if scenarios under conditions that do not yet exist in the actual world.
Achieving high accuracy in an emulation testbed is an immense challenge in itself. To do this, we are characterizing the Bitcoin network by collecting data from the actual world with the aid of a measurement tool. In this post, we will focus on the design, objectives, and the architectural overview of Miniature World. Characterization of today's Bitcoin network will be the topic of a forthcoming blog post.
In previous work with Ittay Eyal and Robbert van Renesse, we developed five critical, specialized metrics for evaluating blockchain protocols. These metrics include mining power utilization, fairness, consensus delay, time to win, and time to prune.
Mining power utilization is a measure of robustness against rollback. This metric represents the fraction of mined blocks that remain as part of the main chain. In general, a blockchian protocol with high utilization is desirable. In such a protocol, the relative power of an attacker that attempts to build its own hidden chain and publish it at once to gain unfair advantage is low.
Does the super-rich get super-richer in your protocol?
Fairness represents the robustness against centralization. To understand what fairness means lets assume that the table in the figure contains all peers in the network. The first column shows the ratio of blocks mined by a peer. The second column represents the corresponding ratio of blocks a peer has in the main chain. Note that these two ratios could be different due to forks. The table is sorted by the first column. The sum of ratios in first circle represents the fair presence of peers excluding the most powerful peer. The second circle represents the corresponding sum for the actual presence. The ratio of actual and fair presence represents the fairness. In a fair blockchain protocol, fairness should be close to 1.
Consensus delay specifies how long peers should look back in the history of a blockchain to find a point that they agree on. This metric has two parameters. The first one defines the percent of times that the nodes agree. The second one indicates the ratio of nodes that reach an agreement. A protocol with low consensus delay enables high frequency placement of transactions to the underlying blockchain.
Time to win and time to prune are metrics that are related to the time it takes for the unwanted items to be pruned from the blockchain. Their precise descriptions appear in our Bitcoin-NG paper.
Miniature world enables repetitive evaluations of blockchain protocols under different scenarios and constraints. It is efficient because it doesn't do any actual mining.
The Miniature World system consists of a central, cluster-wide controller and an in situ controller per node.
The cluster-wide controller has two parts: a bootstrapper and an event-driven emulator. The bootstrapper handles the process before the evaluations. It initializes the topology and emulates the wide area network based on the data that the measurement tool has collected. For the emulation process, it relies on kernel rate limiting and virtual network interfaces. Finally, starts protocol clients across the cluster and then initiates the blockchain.
The cluster-wide event-driven emulator triggers block and transaction generation based on realistic scenarios and constraints. It coordinates in situ controllers and keeps track of the cluster health.
Local loggers keep track of the progress of evaluations and events. Upon the end of the evaluation, they upload the logs back into the controller for aggregation. They handle the clock drift using a local NTP server.
Overall, the ultimate goal of Miniature World is to provide the exact emulation of given blockchain protocols in a lab environment. This gives us the opportunity to make scientific comparisons of different architectural choices. Slides and video of Miniature World talk at Stanford BPASE '17 are now available.
Dr. Gencer is a researcher at LinkedIn. His thesis research focused on improving the scalability of blockchain technologies. more...
Hacker and professor at Cornell, with interests that span distributed systems, OSes and networking. Current projects include HyperDex, OpenReplica and the Nexus OS. more...
bitcoin / security / ethereum / hyperdex / release / nosql / selfish-mining / blocksize / dao / surveillance / privacy / mongo / broken / weaver / nsa / meta / leveldb / blockchain / 51% / voting / smart contracts / graph stores / decentralization / bitcoin-ng / vaults / snowden / satoshi / philosophy / mt. gox / mining pools