Using Cloudera Manager to troubleshoot problems

Installing Impala with Cloudera Manager will not only help in installing and upgrading Impala, but it will also be very helpful in Impala management and troubleshooting.

Impala log analysis using Cloudera Manager

In Cloudera Manager, you can navigate to the Diagnose | Logs option to select Impala-specific log configurations, as shown in the following screenshot:

Impala log analysis using Cloudera Manager

In the log configuration page, you can select Impala daemon, statestore, and log types such as INFO, DEBUG, WARN, ERROR, TRACE, or FATAL. This is shown in the following screenshot:

Impala log analysis using Cloudera Manager

After Impala-specific log configuration is completed, you can see the Impala log files on the same page along with log snippets. You can view the whole log files by just performing one-click operation, as shown in the following screenshot:

Impala log analysis using Cloudera Manager

Besides this, you can view the Impala log files at console windows by visiting the /var/log/impala or /var/log/impalad folders. These are the folders where Impala stores logs by default in each node where impalad is running. The entire log files are generated at the restart of the impalad process with a timestamp. In these directories, the Impala log files are as follows:

Log file name

The information stored in the log file

impalad.INFO

Impala daemon-specific configuration settings

impalad.WARNING

Information about all kinds of warnings generated by Impala

impalad.ERROR

All kinds of errors and potential problems encountered by Impala

impalad.FATAL

A FATAL error that could stop Impala execution or induce query failure

statestored.ERROR

Statestore-specific errors

Impala logs contain detailed information about the Impala daemon and statestore processes, along with each query processed by Impala. Some of the info, such as process and machine info, is logged once. However, when queries are processed, all query details are logged. This is described in the following table:

One time info in the logs at Impala startup

Each query-specific detail in the Impala log

Machine name

Impala version

Impala startup flags

CPU and disks information

Query composition

Degree of data locality

Data throughput statistics

Query response time

Using the Impala web interface for monitoring and troubleshooting

Impala provides the web interface for the Impala process at the 25000 port, as shown in the following screenshot. This is to check various information, such as configuration, logs, metrics, queries details, and memory:

Using the Impala web interface for monitoring and troubleshooting

Using the Impala statestore web interface

Same as the last section, the statestore web interface is available at the 25010 port with the same info as the last section. This is shown in the following screenshot:

Using the Impala statestore web interface

Using the Impala Maintenance Mode

Cloudera Manager provides a configuration named Maintenance Mode that can help you monitor the whole Impala cluster effectively. With Cloudera Manager, Maintenance Mode can be set for a service, a role, a host, or a whole cluster. Using Maintenance Mode as an administrator, you can suppress alerts for the host, role, service, or full cluster. While in Maintenance Mode, logging still works as it is. Therefore, even when there are no alerts, all the logs are still saved, so that anyone can check those logs to meet any specific requirement later. When making specific changes to the cluster or any specific component, Maintenance Mode helps to reduce unnecessary change notifications to all users and keeps the noise due to maintenance very low. The following screenshot shows how you can set your Cloudera Hadoop cluster into Maintenance Mode:

Using the Impala Maintenance Mode

Checking Impala events

Using Cloudera Manager, you can track various events that are generated by Impala, events such as activity, error code, file access, user access, exception, command execution, role, and several others listed in the Diagnose drop-down list, as shown in the following screenshot:

Checking Impala events
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.139.83.96