In this chapter we will cover the following recipes:
Installing and using Cerebro
Installing Kibana and X-Pack
Managing Kibana dashboards
Monitoring with Kibana
Using Kibana console
Visualizing data with Kibana
Installing Kibana plugins
Generating Graph with Kibana
Introduction
In Elasticsearch ecosystem, it can be immensely useful to monitor nodes and cluster to manage and improve their performance and state. There are several issues that can arise at cluster level, such as:
Node overheads, where some nodes can have too many shards allocated and can become a bottleneck for the entire cluster
Node shutdown can happen due to many reasons, for example, full disks, hardware failures, and power problems
Shard relocation problems or corruptions, in which some shards are unable to get an online status
Too large shards happens when a shard is too big; the index performance decreases due to Lucene massive segments merging
Empty indices and shards waste memory and resources, but because every shard has a lot of active threads if there is a huge number of unused indices and shards, the general cluster performance is degraded
Node problems such as high CPU usage or disk full
Detecting malfunction or bad performances can be done via API or via some frontends that are designed to be used in Elasticsearch.
Some of the frontends introduced in this chapter allow the readers to have a working web dashboard on their Elasticsearch data, monitoring the cluster health, backuping/restoring your data and allow testing queries before implementing them in the code. In this chapter, we will have an overview of these frontends due to their complexity and the large number of features that will require a book to themselves. For an in-depth description I suggest the reader to have a look at the official documentation of Kibana at https://www.elastic.co/guide/en/kibana/current/index.html or dedicated books.
Note
In this chapter, we will see an Elasticsearch commercial product called X-Pack. Generally, I prefer to use open source solutions and applications, but for functionalities that are present in X-Pack there are no opensource alternatives at the moment.