After reading this chapter you will be able to set up proper monitoring for your Nginx installation using the features freely available as part of base Nginx distribution and several of the more popular monitoring and alerting systems available on the market.
You will also get acquainted with an advanced monitoring solution that is included in the Nginx Plus subscription package, which the makers of Nginx sell as a commercial service.
Knowing what is happening to your web server is the first step towards fixing and preventing any problems. Setting up monitoring and alerting is one of the essential steps in any production deployment, and Nginx web servers are not the exception here. Experienced web administrators will never feel calm enough until they have real-time insight into the performance and load indicators.
There are several sources of data for your monitoring system provided by Nginx. First, there are logs, which you may configure up to your preferences. There is also a number of statistics interfaces available for Nginx that you may enable and connect to various industrial monitoring systems, both free and commercial. The logging subsystem was described in the Chapter 2, Searching for Problems in Log Files, which can be referred to for additional information. One particular tool that uses logs to enable online monitoring and debugging should be mentioned.
When the first version of the famous Unix utility top(1) was conceived in 1984, imitating an old VMS command that did a similar thing, the author was unlikely to imagine that he actually created a whole class of online system administration tools suited for both routine and emergency situations. Since then, top has become an essential program, and many other utilities have been born with the same principle in mind—produce a dynamic interactive top N list of items sorted by a particular criteria. There are htop, iotop, mytop, pg_top, ntop, iftop, and many others. The Nginx ecosystem has its own top, which is named ngxtop and is hosted on https://github.com/lebinh/ngxtop.
The recommended way to install ngxtop is using the pip package manager for Python packages. Your distribution may or may not have pip installed by default, so you might also need to install pip first. On Debian-based Linux distributions, you will usually be all set up with:
$ sudo apt-get install python-pip
On a FreeBSD box, you need to install the port devel/py-pip with these:
$ cd /usr/ports/devel/py-pip $ sudo make install
When pip is available, you can install the actual ngxtop with it:
$ sudo pip install ngxtop
You can run ngxtop right away, and due to some heuristics, it will correctly locate your main Nginx log all by itself.
This is how the ngxtop default output looks like on a simple and rather idle website access log:
There are two tables in the output. The first one shows the summary line. The columns are pretty obvious. The ones that must catch your attention immediately are the rightmost two (titled 4xx and 5xx), which contain the counters for the HTTP requests that led to errors.
The second table is more interesting because it has all the incoming requests classified by the request path. You may immediately see which paths generate errors, for example. In the earlier example, you will note that the website lacks the robots.txt file and also the special high resolution icons, which were first used by the iOS. The counts in the lower table do not add up exactly to the respective counts in the top table because the least happening request paths are not included here.
ngxtop has several powerful command-line options, which make it one of the best tools to analyze a log from a misbehaving website. For example, in Chapter 2, Searching for Problems in Log Files, we described how to create custom log formats extending the information beyond what a common access logs offer. ngxtop is able to accept a log format definition and even parse it from the nginx.conf file itself.
The command-line switch for the custom log format is -f. The whole command will look just like this:
$ ngxtop -f '$remote_addr - $remote_user [$time_local] "$request" '