NSQ is a messaging queue that allows one program to send messages or events to another, or to many other programs running either locally on the same machine, or on different nodes connected by a network. NSQ guarantees the delivery of messages, which means it keeps undelivered messages cached until all interested parties have received them. This means that, even if we stop our counter program, we won't miss any votes. You can contrast this capability with fire-and-forget message queues where information is deemed out-of-date, and therefore is forgotten if it isn't delivered in time, and where the sender of the messages doesn't care if the consumer received them or not.

A message queue abstraction allows you to have different components of a system running in different places, provided they have network connectivity to the queue. Your programs are decoupled from others; instead, your designs start to care about the ins and outs of specialized micro-services, rather than the flow of data through a monolithic program.

NSQ transfers raw bytes, which means it is up to us how we encode data into those bytes. For example, we could encode the data as JSON or in a binary format depending on our needs. In our case, we are going to send the vote option as a string without any additional encoding, since we are only sharing a single data field.

Open http://nsq.io/deployment/installing.html in a browser (or search install nsq) and follow the instructions for your environment. You can either download pre-compiled binaries or build your own from the source. If you have homebrew installed, installing NSQ is as simple as typing:

brew install nsq

Once you have installed NSQ, you will need to add the bin folder to your PATH environment variable so that the tools are available in a terminal.

To validate that NSQ is properly installed, open a terminal and run nsqlookupd; if the program successfully starts, you should see some output similar to the following:

nsqlookupd v0.2.27 (built w/go1.3)
TCP: listening on [::]:4160
HTTP: listening on [::]:4161

We are going to use the default ports to interact with NSQ so take note of the TCP and HTTP ports listed in the output, as we will be referring to them in our code.

Press Ctrl + C to stop the process for now; we'll start them properly later.

The key tools from the NSQ install that we are going to use are nsqlookupd and nsqd. The nsqlookupd program is a daemon that manages topology information about the distributed NSQ environment; it keeps track of all the nsqd producers for specific topics and provides interfaces for clients to query such information. The nsqd program is a daemon that does the heavy lifting for NSQ such as receiving, queuing, and delivering messages from and to interested parties. For more information and background on NSQ, visit http://nsq.io/.

go get github.com/bitly/go-nsq

MongoDB is a document database, which basically allows you to store and query JSON documents and the data within them. Each document goes into a collection that can be used to group the documents together without enforcing any schema on the data inside them. Unlike rows in a traditional RDBMS such as Oracle, Microsoft SQL Server, or MySQL, it is perfectly acceptable for documents to have a different shape. For example, a people collection can contain the following three JSON documents at the same time:

{"name":"Mat","lang":"en","points":57}
{"name":"Laurie","position":"Scrum Master"}
{"position":"Traditional Manager","exists":false}

This flexibility allows data with varying structure to coexist without impacting performance or wasting space. It is also extremely useful if you expect your software to evolve over time, as we really always should.

MongoDB was designed to scale while also remaining very easy to work with on single-box install such as our development machine. When we host our application for production, we would likely install a more complex multi-sharded, replicated system, which is distributed across many nodes and locations, but for now, just running mongod will do.

Head over to http://www.mongodb.org/downloads to grab the latest version of MongoDB and install it, making sure to register the bin folder with your PATH environment variable as usual.

To validate that MongoDB is successfully installed, run the mongod command, then hit Ctrl + C to stop it for now.

go get gopkg.in/mgo.v2

Now that we have all the pieces we need installed, we need to start our environment. In this section, we are going to:

Each of these daemons should run in their own terminal window, which will make it easy for us to stop them by just hitting Ctrl + C.

In a terminal window, run:

nsqlookupd

Take note of the TCP port, which by default is 4160, and in another terminal window, run:

nsqd --lookupd-tcp-address=localhost:4160

Make sure the port number in the --lookupd-tcp-address flag matches the TCP port of the nsqlookupd instance. Once you start nsqd, you will notice some output is printed to the terminal from both nsqlookupd and nsqd; this indicates that the two processes are talking to each other.

In yet another window or tab, start MongoDB by running:

mongod --dbpath ./db

The dbpath flag tells MongoDB where to store the data files for our database. You can pick any location you like, but you'll have to make sure the folder exists before mongod will run.

Now that our environment is running, we are ready to start building our components.