How to Set Up High Availability for Resource Manager – Part 6

YARN is the Processing Layer of Hadoop, which consists of the Master (Resource Manager) and Slave (Node Manager) services to process the data. Resource Manager (RM) is the critical component that is responsible for resource allocation and management among all the jobs running in Hadoop Cluster.

It is always recommended and best practice to have the Cluster High Availability (HA) enabled on Critical services like Namenode and Resource Manager.

Requirements

In this article, we will see the steps to enable High Availability on Resource Manager.

Enabling High Availability on Resource Manager

1. Go to Cloudera Manager at the following addresses and navigate to YARN –> Actions –> Enable High Availability.

http://13.233.129.39:7180/cmf/home
Cloudera Manager Dashboard
Cloudera Manager Dashboard
Enable High Availability
Enable High Availability

2. Select a server where you are going to have a second Resource Manager. Usually, we will be having a second master server to deploy High Availability. Here, we are selecting master2 for enabling HA.

Add Resource Manager Host
Add Resource Manager Host
Select Resource Manager Hosts
Select Resource Manager Hosts

3. Once selected the master2, click ‘Continue’ to proceed.

Resource Manager Hosts
Resource Manager Hosts

4. Enabling HA process will be started. You can view the background operations by clicking each steps.

Enable High Availability Process
Enable High Availability Process

5. Once all the processes completed, you will be getting ‘Finished’ status. Click ‘Finish’.

Finish High Availability Process
Finish High Availability Process

6. Verify the Resource Manager High availability by viewing the Yarn instances at Cloudera Manager –> YARN –> Instances.

Verify High Availability
Verify High Availability

You can see two Resource Manager, one will be in the state ‘Active’, another will be in ‘Standby’.

Check Resource Manager High Availability
Check Resource Manager High Availability
Conclusion

In this article, we have gone through the step by step procedure to enable High Availability on Resource Manager. Whenever the Active Resource Manager goes down, the Standby Resource manager will become Active so that production will not be having an outage.

Mohan Sivam
A Solution Architect and Bigdata infrastructure with over 10 years of experience in Information technology.

Each tutorial at TecMint is created by a team of experienced Linux system administrators so that it meets our high-quality standards.

Join the TecMint Weekly Newsletter (More Than 156,129 Linux Enthusiasts Have Subscribed)
Was this article helpful? Please add a comment or buy me a coffee to show your appreciation.

Got something to say? Join the discussion.

Thank you for taking the time to share your thoughts with us. We appreciate your decision to leave a comment and value your contribution to the discussion. It's important to note that we moderate all comments in accordance with our comment policy to ensure a respectful and constructive conversation.

Rest assured that your email address will remain private and will not be published or shared with anyone. We prioritize the privacy and security of our users.