YARN client mode

In YARN client mode, the Driver runs on a node outside the cluster (typically where the client is). Driver first contacts the resource manager requesting resources to run the Spark job. The resource manager allocates a container (container zero) and responds to the Driver. The Driver then launches the Spark application master in the container zero. The Spark application master then creates the executors on the containers allocated by the resource manager. The YARN containers can be on any node in the cluster controlled by node manager. So, all allocations are managed by resource manager.

Even the Spark application master needs to talk to resource manager to get subsequent containers to launch executors.

The following is the YARN-client mode deployment of Spark:

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
13.59.100.205