HyperDex Meets Docker

Our Environment

In our environment, we have three hosts, which we'll label A, B, and C for sake of this discussion. We'll assume that these hosts can freely talk to each other, and are bound to addresses in the 10.1.1.0/24 block. If you're following along at home, substitute the IPs of A, B, and C for your own nodes' IP addresses, and run the following on each of the three machines.

$ IP_A=10.1.1.101
$ IP_B=10.1.1.102
$ IP_C=10.1.1.103

The HyperDex cluster we're deploying in this environment will have a total of 6 components running on it. First, we'll launch a single coordinator node and a single daemon on server A. Then, we'll expand the cluster to a total of three coordinators and three daemons, one per machine.

Setting Up Docker

Install Docker per its own installation guide.

Once you have Docker running, pull the HyperDex images on each machine with:

$ docker pull hyperdex/coordinator
$ docker pull hyperdex/daemon

If you've never worked with Docker before, this step might take some time.

Initial Deployment

First, we will deploy the first coordinator in our cluster. This coordinator has the distinct position of initializing the cluster and bootstrapping the other, redundant coordinators for fault-tolerance. The ensemble of coordinators work together to maintain the daemons in the cluster.

Launch the first coordinator with:

$ # Run this on host A
$ docker run --net=host -t -i hyperdex/coordinator \
     hyperdex coordinator --foreground --listen=${IP_A} -D /hyperdex/coord

This runs the HyperDex coordinator in the foreground of a new container, and binds it to the host's address of ${IP_A}. Other hosts can use this address to connect to the cluster. For instance, to launch a daemon, run a new container:

$ # Run this on host A
$ docker run --net=host -t -i hyperdex/daemon \
     hyperdex daemon --foreground --listen=${IP_A} \
         --coordinator ${IP_A} -D /hyperdex/daemon

We now have a working, 1-node HyperDex cluster. Go ahead and walk through the HyperDex quickstart tutorial with your working cluster. Use ${IP_A} throughout that tutorial instead of 127.0.0.1, because you're now talking to the Docker instances.

Scaling the Cluster

Power users will want to scale up HyperDex to a larger cluster with fault tolerance. A scaled up cluster will also achieve higher performance than that of a single node.

Scaling up in this fashion is straightforward. We'll deploy additional coordinator and daemon nodes on servers B and C, and HyperDex's automatic configuration will take care of scaling the cluster to make use of these new resources.

To do this, first launch the additional coordinators:

$ # Run this on host B
$ docker run --net=host -t -i hyperdex/coordinator \
     hyperdex coordinator --foreground --listen=${IP_B} \
         --connect ${IP_A} --connect-port=1982 \
         -D /hyperdex/coord

$ # Run this on host C
$ docker run --net=host -t -i hyperdex/coordinator \
     hyperdex coordinator --foreground --listen=${IP_C} \
         --connect-string=${IP_A}:1982,${IP_B}:1982,${IP_C}:1982 \
         -D /hyperdex/coord

We now have a three-node coordinator that can keep the HyperDex cluster alive and moving so long as at most one of the nodes fails. The cluster will keep making progress in spite of the failed node. In real production settings, we recommend running a 5-node coordinator, which can tolerate two simultaneous failures.

If you're closely paying attention, you'll notice that we've used a different set of arguments for host C. For this host, we took advantage of a new feature in the 1.4.2 release that allows coordinators and clients to specify multiple coordinators to use for bootstrapping. HyperDex will transparently try all listed coordinators to connect to the cluster, and will succeed even if some of the listed nodes are failed. This simplifies deployment and makes bootstrapping much more robust.

Finally, let's bring the additional daemons online:

$ # Run this on host B
$ docker run --net=host -t -i hyperdex/daemon \
     hyperdex daemon --foreground --listen=${IP_B} \
         --coordinator ${IP_A} -D /hyperdex/daemon

$ # Run this on host C
$ docker run --net=host -t -i hyperdex/daemon \
     hyperdex daemon --foreground --listen=${IP_C} \
         --coordinator ${IP_A} -D /hyperdex/daemon

For the daemons, you can specify any of the coordinators to use for bootstraping.

You now have scaled the cluster up to a 3-node HyperDex cluster. The steps outlined above are quite literally the only steps necessary to scale up. HyperDex takes care of the rest.

Final Thoughts

We've focused much of our attention over the past six months on making HyperDex easy to install, deploy, and manage. Docker simplifies these tasks significantly, enabling you to consistently deploy a cluster in just a few commands. HyperDex picks up where Docker leaves off, and makes scaling so easy that scaling a cluster is as simple as adding additional daemons into the cluster.