Hadoop provides web interfaces for each of its services. The namenode UI or the namenode web interface is used to monitor the status of the namenode and can be accessed using the following URL:
http://<namenode-server>:50070/
The namenode UI has the following sections:
- Overview: The general information section provides basic information of the namenode with options to browse the filesystem and the namenode logs.
The following is the screenshot of the Overview section of the namenode UI:
The Cluster ID parameter displays the identification number of the cluster. This number is same across all the nodes within the cluster.
A block pool is a set of blocks that belong to a single namespace. The Block Pool Id parameter is used to segregate the block pools in case there are multiple namespaces configured when using HDFS federation. In HDFS federation, multiple namenodes are configured to scale the name service horizontally. These namenodes are configured to share datanodes amongst themselves. We will be exploring HDFS federation in detail a bit later.
- Summary: The following is the screenshot of the cluster's summary section from the namenode web interface:
Under the Summary section, the first parameter is related to the security configuration of the cluster. If Kerberos (the authorization and authentication system used in Hadoop) is configured, the parameter will show as Security is on. If Kerberos is not configured, the parameter will show as Security is off.
The next parameter displays information related to files and blocks in the cluster. Along with this information, the heap and non-heap memory utilization is also displayed. The other parameters displayed in the Summary section are as follows:
- Configured Capacity: This displays the total capacity (storage space) of HDFS.
- DFS Used: This displays the total space used in HDFS.
- Non DFS Used: This displays the amount of space used by other files that are not part of HDFS. This is the space used by the operating system and other files.
- DFS Remaining: This displays the total space remaining in HDFS.
- DFS Used%: This displays the total HDFS space utilization shown as percentage.
- DFS Remaining%: This displays the total HDFS space remaining shown as percentage.
- Block Pool Used: This displays the total space utilized by the current namespace.
- Block Pool Used%: This displays the total space utilized by the current namespace shown as percentage. As you can see in the preceding screenshot, in this case, the value matches that of the DFS Used% parameter. This is because there is only one namespace (one namenode) and HDFS is not federated.
- DataNodes usages% (Min, Median, Max, stdDev): This displays the usages across all datanodes in the cluster. This helps administrators identify unbalanced nodes, which may occur when data is not uniformly placed across the datanodes. Administrators have the option to rebalance the datanodes using a balancer.
- Live Nodes: This link displays all the datanodes in the cluster as shown in the following screenshot:
- Dead Nodes: This link displays all the datanodes that are currently in a dead state in the cluster. A dead state for a datanode daemon is when the datanode daemon is not running or has not sent a heartbeat message to the namenode daemon. Datanodes are unable to send heartbeats if there exists a network connection issue between the machines that host the datanode and namenode daemons. Excessive swapping on the datanode machine causes the machine to become unresponsive, which also prevents the datanode daemon from sending heartbeats.
- Decommissioning Nodes: This link lists all the datanodes that are being decommissioned.
- Number of Under-Replicated Blocks: This represents the number of blocks that have not replicated as per the replication factor configured in the
hdfs-site.xmlfile.
- Namenode Journal Status: The journal status provides location information of the
fsimagefile and the state of theeditslogfile. The following screenshot shows the NameNode Journal Status section: - NameNode Storage: The namenode storage table provides the location of the
fsimagefile along with the type of the location. In this case, it isIMAGE_AND_EDITS, which means the same location is used to store thefsimagefile as well as theeditslogfile. The other types of locations areIMAGE, which stores only thefsimagefile andEDITS, which stores only theeditslogfile. The following screenshot shows the NameNode Storage information: