Install Hadoop Multinode Cluster using CDH4 in RHEL/CentOS 6.5

If You Appreciate What We Do Here On TecMint, You Should Consider:

  1. Stay Connected to: Twitter | Facebook | Google Plus
  2. Subscribe to our email updates: Sign Up Now
  3. Use our Linode referral link if you plan to buy VPS (it starts at only $10/month).
  4. Support us via PayPal donate - Make a Donation
  5. Support us by purchasing our premium books in PDF format.
  6. Support us by taking our online Linux courses

We are thankful for your never ending support.

Kuldeep Kulkarni

I'm Kuldeep Kulkarni - Crazy about Linux, Hadoop etc open-source technologies!! By profession I'm Senior system engineer and hadoop administrator in well known IT industry since 2011. Always enthusiastic about sharing my knowledge via blogs :)

Your name can also be listed here. Got a tip? Submit it here to become an TecMint author.

RedHat RHCE and RHCSA Certification Book
Linux Foundation LFCS and LFCE Certification Preparation Guide

You may also like...

42 Responses

  1. Nitish says:

    Getting Error in Step 11 : Create HDFS /tmp and MapReduce /var Directories

    sudo -u hdfs hadoop fs -mkdir /tmp
    Error is:-
    mkdir: Call From master/192.168.56.101 to master:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused

    The hdfs-site.xml is configured as:-

    dfs.namenode.name.dir
    file:///data/1/dfs/nn,/nfsmount/dfs/nn

    dfs.permissions.superusergroup
    hadoop

    dfs.namenode.http-address
    192.168.56.101:50070

    The address and port on which the NameNode UI will listen.

    Can you please assist me in understanding what I missed.

    Telnet isn’t working as no process is listening on 8020 port.

    Regards
    Nitish

  2. harini says:

    @Lyle Gilbert I have same issue as u had.Can u please help me fix it?

  3. Govind says:

    HI sir, can we also have this setup using Ansible.? please.

  4. Lyle Gilbert says:

    I apologize for my ignorance in advance, but when i follow your instructions i can get to
    sudo -u hdfs hadoop fs -mkdir /tmp
    and all i get is the following error
    mkdir: Call From master/172.21.181.108 to master:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
    I do a netstat -tulpn and do not see anything listening on that port. what config should i modify?

  5. Ravi says:

    Helle Kuldeep,
    Thanks for this article. I’m trying to install on Centos7, few of the commands are outdated :(
    Do you have an video of making the cluster ?

    Thanks
    Ravi

  6. manish says:

    hello Kuldeep.
    Thanks for the instructions above. I am new in hadoop + linux world. It worked very well for me untill this step.

    Next, copy ‘mapred-site.xml‘ file to node machine using the following scp command.
    [root@master conf]# scp /etc/hadoop/conf/mapred-site.xml node:/etc/hadoop/conf/
    mapred-site.xml 100% 200 0.2KB/s 00:00

    I didnt have mapred-site.xml file and thus i created it using vi . I copied the xml you provided.
    However when i am trying to SCP this file to the node: i get an error :
    ssh: Could not resolve hostname node: name of service not known. Lost connection.

    What should i do in order to fix this?

    Thanks
    Manish

Got something to say? Join the discussion.

Your email address will not be published. Required fields are marked *

Join Over 300K+ Linux Users
  1. 177,942
  2. 8,310
  3. 37,548

Are you subscribed?