Technologies
Many of the technologies discussed here do more than just store service discovery data, and are often used for coordination as well as a configuration store. In addition to the features and libraries that integrate with these services, you should consider the following: If you are going to deploy Docker Swarm and use Consul for cluster coordination, then maybe you want to consider using it for service discovery so you don’t have to deploy and maintain another system. You need to be careful and consider how this could affect the scalability and performance of the cluster, as well as the different requirements and features of the technology, like the consistency models.
When sharing deployments of a service, be careful that one workload does not adversely affect the other. Using the Zookeeper deployment that Apache Mesos uses for maintaining cluster state as a service discovery store can impact the performance of Apache Mesos. In a production environment it is generally recommended that Apache Mesos should use a dedicated Zookeeper deployment.
DNS
The simplest solution would be to use DNS and create a name for each service; then clients can discover service instances using standard DNS. DNS for service discovery does have its limitations. First of all, DNS does not support pushing changes, so we need to poll for changes. Caching and propagation delays can cause challenges and latency in updating state. Some applications will even cache DNS when they start up.
If planning to use DNS for service discovery, consider some of the more modern DNS services. Mesosphere has created a project called Mesos-DNS which provides DNS-based discovery in Apache Mesos (https://github.com/mesosphere/mesos-dns). SkyDNS (https://github.com/kynetservices/skydns) is another popular option built on top of etcd. Consul is also capable of exposing service lookup through DNS as well as its standard API. The Docker and Weave networking features that are discussed further in the following overlay section also provide DNS-based container discovery.
DNS-based service discovery can be adequate and readily available in the platform, as with the Azure Container Service. When using DNS for the discovery of endpoints that are very dynamic, we need to take care in setting appropriate TTL and to understand how our clients are caching DNS. The argument against DNS is that it was designed for a different purpose and the caching can cause a lot of challenges in production.
Consul
Consul is an open-source project from the folks at Hashicorp, the same people that brought us Terraform, Vagrant, and Vault. Consul provides an HTTP API that we can use for service announcement and discovery as well as a DNS interface. In addition to service discovery, Consul can perform service health checks, which can be used for monitoring or service discovery routing. Consul also has a very nice templating client, which can be used to monitor service announcement changes and generate proxy client configurations. Other features include a key/value store and multiple data center support.
etcd
etcd is a distributed key value store from the folks at CoreOS. It provides reliable storage of data across a cluster of machines. It is often used as a service discovery store when building service discovery systems. It originated in the CoreOS project, is included in the distribution, and can be installed on other distributions.
Zookeeper
Zookeeper is an open-source Apache project, and provides a distributed, eventually consistent hierarchical key/value store. It’s a popular option for a service discovery store when building service discovery systems. Mesos uses Zookeeper for coordination, and you might want to consider whether or not the technology is already used in your infrastructure when selecting your service discovery tool.
Eureka
Eureka is an open-source service discovery solution developed by Netflix. It’s built for availability and resiliency, and is used at Netflix for locating services for the purpose of load balancing and failover of middle-tier servers. The project includes a Java-based client that offers simple round-robin load balancing. Additional application-specific metadata about a service can also be stored in Eureka.