In this section, we will do a deep dive into YARN application development. YARN offers flexibility to developers to write applications that can run on Hadoop clusters in different programming languages. In this section, we will focus on setting up a YARN project, we will write a sample client and application master, and we will see how it runs on a YARN cluster. The following block diagram shows typical interaction patterns between various components of Apache Hadoop when a YARN application is developed and deployed:

Primarily, there are three major components involved: Resource Manager, Application Master, and Node Manager. We will be creating a custom client application, a custom application master, and a YARN client app. As you can see, there are three different interactions that take place between different components:

• Client and Resource Manager through ClientRMProtocol
• ApplicationMaster and Resource Manager through AMRMProtocol
• ApplicationMaster and Node Manager through the ContainerManager mechanism

Let's look at each of them in detail.