Monitoring and getting information out of Docker

It's often important to get some quick and useful information out of our Docker system when weird problems arise or strange issues start to cripple our performance. What's going on in the system? Is there a container taking up all of the memory? Maybe one minor container just crashed and is eating up all of the CPU. All of this information shouldn't be hard to get, but they are precious for building quality containers. We'll see two tools quite fit for the job: the first one is simply the one shipped with Docker itself, and the second one is a totally different tool by Google named cAdvisor—a web user interface with a lot of useful and easy-to-get information.

Getting ready

To step through this recipe, you will need:

  • A working Docker installation

How to do it...

There's a few ways to get information out of Docker. We'll explore the first one through the main Docker program.

Using docker stats

To get live metrics about the running containers (CPU, memory, and network), we can use the simple docker stats command:

$ docker stats
CONTAINER           CPU %               MEM USAGE / LIMIT     MEM %               NET I/O               BLOCK I/O             PIDS
c2904d5b5c89        0.01%               892.9 MB / 8.326 GB   10.72%              258.2 GB / 10.27 GB   374 MB / 0 B          16
0641790f1b30        3.36%               894.4 MB / 8.326 GB   10.74%              258.2 GB / 11.12 GB   419.1 MB / 0 B        16
bc8d85e05be8        112.65%             891.4 MB / 8.326 GB   10.71%              179.6 GB / 536.5 GB   326.6 MB / 0 B        10
a7be664792b3        0.02%               45.37 MB / 8.326 GB   0.54%               17.85 GB / 17.72 GB   18.78 MB / 110.6 kB   18
ab2d4e922949        2.37%               70.34 MB / 8.326 GB   0.84%               83.15 MB / 550 MB     459.7 MB / 143.4 kB   17
08e685124dfd        0.01%               192 MB / 8.326 GB     2.31%               8.76 MB / 42.11 MB    1.499 MB / 14.05 MB   3
5893c5d6f43f        0.74%               546.1 MB / 8.326 GB   6.56%               46.74 MB / 40.22 MB   160.7 MB / 317.9 MB   74
7f21e405bdee        5.23%               8.184 MB / 8.326 GB   0.10%               30.14 GB / 30.28 GB   8.192 kB / 0 B        7

It's, however, not overwhelmingly helpful as it's using containers' IDs and not names, and when running many containers, it can start becoming useless because it would be unreadable. So we can use a trick: ask the stats (docker stats) of all the running containers (docker ps) whose names we extracted using a Go template formatter (--format):

$ docker stats $(docker ps --format '{{.Names}}')
CONTAINER                   CPU %               MEM USAGE / LIMIT     MEM %               NET I/O               BLOCK I/O             PIDS
sm_streammachine-slave_2    18.34%              889.4 MB / 8.326 GB   10.68%              258.2 GB / 10.27 GB   374 MB / 0 B          16
sm_streammachine-slave_1    28.39%              900.1 MB / 8.326 GB   10.81%              258.2 GB / 11.12 GB   419.1 MB / 0 B        16
sm_streammachine-master_1   1.89%               890.4 MB / 8.326 GB   10.69%              179.6 GB / 536.5 GB   326.6 MB / 0 B        10
sm_proxy_1                  0.02%               45.37 MB / 8.326 GB   0.54%               17.85 GB / 17.72 GB   18.78 MB / 110.6 kB   18
sm_cadvisor_1               1.62%               70.34 MB / 8.326 GB   0.84%               83.16 MB / 550 MB     459.7 MB / 143.4 kB   17
sm_analytics_1              0.01%               192 MB / 8.326 GB     2.31%               8.76 MB / 42.11 MB    1.499 MB / 14.05 MB   3
sm_elasticsearch_1          0.72%               546.1 MB / 8.326 GB   6.56%               46.74 MB / 40.22 MB   160.7 MB / 317.9 MB   74
sm_streamer_1               8.17%               8.184 MB / 8.326 GB   0.10%               30.15 GB / 30.29 GB   8.192 kB / 0 B        7

Using Google's cAdvisor tool

Google created a nice web tool to see what's going on in machines that run containers: cAdvisor. It collects, organizes, and displays metrics about resource usage, container by container, on a given host. Though not interactive, it's still powerful enough, given how easy it is to install and use. To install and use it, simply run the cAdvisor Docker image with volume access to all of the required system information, such as the following:

$ sudo docker run 
  --volume=/:/rootfs:ro 
  --volume=/var/run:/var/run:rw 
  --volume=/sys:/sys:ro 
  --volume=/var/lib/docker/:/var/lib/docker:ro 
  --publish=8080:8080 
  --detach=true 
  --name=cadvisor 
  google/cadvisor:latest

Or, if using docker-compose:

  cadvisor:
    volumes:
      - /:/rootfs:ro
      - /var/run:/var/run:rw
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
    ports:
      - "8080:8080"
    image: google/cadvisor:latest
    restart: always

Navigating to the host's 8080 port (or whatever port you choose to publish) with a web browser will present a web interface where we can navigate and see graphical information about container usage on the host:

Using Google's cAdvisor tool

Or, we may have more general gauges giving live indication of resource usage:

Using Google's cAdvisor tool

A very useful process table with top-like data from the underlying host is also available with a container-aware context. All of these pieces of data are browsable and they help you gain more in-depth information about a specific container and its content and usage:

Using Google's cAdvisor tool

cAdvisor can also be plugged in to many backend storage systems, such as Prometheus, ElasticSearch, InfluxDB, Redis, statsD, and so on.

Note

If you plan to let cAdvisor run permanently, it is a good idea to restrict access using simple HTTP authentication. This is supported out of the box by cAdvisor using --http_auth_file /cadvisor.htpasswd --http_auth_realm my_message.

See also

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.141.47.25