Discovery provides a more flexible way to create a cluster. It doesn't need to know other peer IPs beforehand. It uses an existing etcd cluster to bootstrap one. In this section, we'll demonstrate how to leverage that to launch a three-node etcd cluster:

1. Firstly, we'll need to have an existing cluster with three-node configuration. Luckily, the etcd official website provides a discovery service (https://discovery.etcd.io/new?size=n); n will be the number of nodes in your etcd cluster, which is ready to use:

// get a request URL
# curl -w "n" 'https://discovery.etcd.io/new?size=3'
https://discovery.etcd.io/f6a3fb54b3fd1bb02e26a89fd40df0e8

2. Then we are able to use the URL to bootstrap a cluster easily. The command line is pretty much the same as in the static mechanism. What we need to do is change –initial-cluster to –discovery, which is used to specify the discovery service URL:

// in node1, 127.0.0.1 is used for internal client listeneretcd -name ip-172-31-3-80 -initial-advertise-peer-urls http://172.31.3.80:2380  -listen-peer-urls http://172.31.3.80:2380  -listen-client-urls http://172.31.3.80:2379,http://127.0.0.1:2379  -advertise-client-urls http://172.31.3.80:2379  -discovery https://discovery.etcd.io/f6a3fb54b3fd1bb02e26a89fd40df0e8

// in node2, 127.0.0.1 is used for internal client listener
etcd -name ip-172-31-14-133 -initial-advertise-peer-urls http://172.31.14.133:2380  -listen-peer-urls http://172.31.14.133:2380  -listen-client-urls http://172.31.14.133:2379,http://127.0.0.1:2379  -advertise-client-urls http://172.31.14.133:2379  -discovery https://discovery.etcd.io/f6a3fb54b3fd1bb02e26a89fd40df0e8

// in node3, 127.0.0.1 is used for internal client listener
etcd -name ip-172-31-13-239 -initial-advertise-peer-urls http://172.31.13.239:2380  -listen-peer-urls http://172.31.13.239:2380  -listen-client-urls http://172.31.13.239:2379,http://127.0.0.1:2379  -advertise-client-urls http://172.31.13.239:2379  -discovery https://discovery.etcd.io/f6a3fb54b3fd1bb02e26a89fd40df0e8

3. Let's take a closer look at node1's log:

2018-02-10 04:58:03.819963 I | etcdmain: etcd Version: 3.3.0
...
2018-02-10 04:58:03.820400 I | embed: listening for peers on http://172.31.3.80:2380
2018-02-10 04:58:03.820427 I | embed: listening for client requests on
127.0.0.1:2379
2018-02-10 04:58:03.820444 I | embed: listening for client requests on 172.31.3.80:2379
2018-02-10 04:58:03.947753 N | discovery: found self f60c98e749d41d1b in the cluster
2018-02-10 04:58:03.947771 N | discovery: found 1 peer(s), waiting for 2 more
2018-02-10 04:58:22.289571 N | discovery: found peer 6645fe871c820573 in the cluster
2018-02-10 04:58:22.289628 N | discovery: found 2 peer(s), waiting for 1 more
2018-02-10 04:58:36.907165 N | discovery: found peer 1ce61c15bdbb20b2 in the cluster
2018-02-10 04:58:36.907192 N | discovery: found 3 needed peer(s)
...
2018-02-10 04:58:36.931319 I | etcdserver/membership: added member 1ce61c15bdbb20b2 [http://172.31.13.239:2380] to cluster 29c0e2579c2f9563
2018-02-10 04:58:36.931422 I | etcdserver/membership: added member 6645fe871c820573 [http://172.31.14.133:2380] to cluster 29c0e2579c2f9563
2018-02-10 04:58:36.931494 I | etcdserver/membership: added member f60c98e749d41d1b [http://172.31.3.80:2380] to cluster 29c0e2579c2f9563
2018-02-10 04:58:37.116189 I | raft: f60c98e749d41d1b became leader at term 2

We can see that the first node waited for the other two members to join, and added member to cluster, became the leader in the election at term 2:

4. If you check the other server's log, you might find a clue to the effect that some members voted for the current leader:

// in node 2
2018-02-10 04:58:37.118601 I | raft: raft.node: 6645fe871c820573 elected leader f60c98e749d41d1b at term 2

5. We can also use member lists to check the current leader:

# etcdctl member list
1ce61c15bdbb20b2: name=ip-172-31-13-239 peerURLs=http://172.31.13.239:2380 clientURLs=http://172.31.13.239:2379 isLeader=false
6645fe871c820573: name=ip-172-31-14-133 peerURLs=http://172.31.14.133:2380 clientURLs=http://172.31.14.133:2379 isLeader=false
f60c98e749d41d1b: name=ip-172-31-3-80 peerURLs=http://172.31.3.80:2380 clientURLs=http://172.31.3.80:2379 isLeader=true

6. Then we can confirm the current leader is 172.31.3.80. We can also use etcdctl to check cluster health:

# etcdctl cluster-health
member 1ce61c15bdbb20b2 is healthy: got healthy result from http://172.31.13.239:2379
member 6645fe871c820573 is healthy: got healthy result from http://172.31.14.133:2379
member f60c98e749d41d1b is healthy: got healthy result from http://172.31.3.80:2379
cluster is healthy

7. If we remove the current leader by etcdctl command:

# etcdctl member remove f60c98e749d41d1b

8. We may find that the current leader has been changed:

# etcdctl member list
1ce61c15bdbb20b2: name=ip-172-31-13-239 peerURLs=http://172.31.13.239:2380 clientURLs=http://172.31.13.239:2379 isLeader=false
6645fe871c820573: name=ip-172-31-14-133 peerURLs=http://172.31.14.133:2380 clientURLs=http://172.31.14.133:2379 isLeader=true

By using etcd discovery, we can set up a cluster painlessly etcd also provides lots of APIs for us to use. We can leverage it to check cluster statistics:

9. For example, use /stats/leader to check the current cluster view:

# curl http://127.0.0.1:2379/v2/stats/leader
{"leader":"6645fe871c820573","followers":{"1ce61c15bdbb20b2":{"latency":{"current":0.002463,"average":0.0038775,"standardDeviation":0.0014144999999999997,"minimum":0.002463,"maximum":0.005292},"counts":{"fail":0,"success":2}}}}

For more information about APIs, check out the official API document: https://coreos.com/etcd/docs/latest/v2/api.html.