Yarn Architecture in Hadoop

 

🤔😎✍️Do you Know #Yarn Architecture in Hadoop 2.0 ?🤔🚀🥳
.

By Karthik Kondpak

..

.

.

.

Let's start

#Yarn stands for "Yet another resource Negotiator"

✍️Yarn  has three components
=========================

1) Resource Manager
2) Node Manager
3) Application Master

1) Resource Manager :-
    ================

✏️When Client Runs the application the application a request is created .

✏️Resource Manager accepts  the client request and looks after all the resources.

✏️Resource Manager creates a container in one the node manager.

🤔What is meant by Container ?
     ----------------------------------------------

✏️Container is nothing but a logical unit having Memory and CPU

🚀Example: Container :- 1gb + 2 Cpu cores

✏️It  also launches a Application Master in the container for that particular application.

✏️Resource Manager is a master it will be one per cluster .

2) Application Master :-
     ================

✏️Application Master is responsible for end to end monitoring.

✏️When client request comes to resource manager and resource manager creates a container and launches a Application master .

✏️The Application master looks after the resources and if resources are not available it asks resources to Resource Manager.

✏️Application Master ask on which location, and how many containers he wants .

✏️Resource Manager provides the resources in the form of containers.

✏️Resource Manager send the container I'd and hostname to the application master.

✏️Application Master taking that container I'd and hostname launches the tasks to execute.

✏️Each task is executed on each container

✏️Once Resource Managers provides  the resources ,it work ends there. It doesn't have any further work

✍️Application Master is also responsible for end-end monitoring of tasks:-
==============================

📌  Like is task is running properly or not

📌  If  task is failed reintiate them

3) Node Manager:-
     =============
     ✏️Node manger provides computational resources in the from of containers.

✏️Node manager manages the resources running in the container.

✏️Containers are nothing but Memory and CPU.

##Note :-
=======
✏️If there is a 20 node cluster 20 data nodes are present and each data node is managed by a node manager

✏️If  there is one application one application master is available

✏️If there are 2 applications then 2 application masters are available

✏️Resource manger is one per cluster.

✍️With the evolution of yarn in Hadoop 2.0 Limitations are handled Gracefully.