Logs provide detailed information to find out how a query/job runs. By checking the log details, we can identify runtime problems and issues that may cause bad performance. There are two types of log available, the system log and job log.
The system log contains the Hive running status and issues. It is configured in {HIVE_HOME}/conf/hive-log4j.properties. The following three lines of the log properties can be found in the file:
hive.root.logger=WARN,DRFA ## set logger level hive.log.dir=/tmp/${user.name} ## set log file path hive.log.file=hive.log ## set log file name
To modify the logger level, we can either modify the preceding property file that applies to all users, or set a Hive command-line config that only applies to the current user session, such as $hive --hiveconf hive.root.logger=DEBUG,console.
The job log contains job information and is usually managed by Yarn. To check a job log, use yarn logs -applicationId <application_id>.