A handy addition to Hadoop (as against Hadoop 1.x) is the resource negotiator YARN, enabling Hadoop to utilize resources in a dynamic fashion, allowing applications to do the job rather than figuring out their impact on the resources. It allows Hadoop to effectively and efficiently use each of the nodes in a cluster. YARN in Hadoop 2.x is also called MR V2 (MapReduce V2).
The following figure shows YARN’s architecture and its components in detail along with how it works:
Figure 10: YARN (MR V2) architecture showing off its components
These are the important components constituting the YARN architecture:
- ResourceManager (RM): This is the main agent that manages resources from within the cluster nodes and them allocates accordingly. It is entrusted with managing cluster resources efficiently and is the Master node of YARN and the real negotiator. ResourceManager has two sub-components, namely the following:
- Application Manager
- Scheduler caters to scheduler requirements for applications, monitors them, and then tracks them
- NodeManager (NM): Each node has one NodeManager and resources within a node are allocated and handled. It's considered as the Slave node of YARN. It accepts requests from Resource Manager and reports back on the health and resources of the nodes.
- ApplicationMaster (AM): ApplicationMaster exists for each application, and is entrusted with the application life cycle, and it's task allocation and execution.