Install Hadoop in Ubuntu 14.04.
- Install Ubuntu in VMware or Virtual Box.
- Update Ubuntu after Installation is done ---> "$ sudo apt-get update".
- Install OpenJDK for ubuntu as a default one ---> "$ sudo apt-get install default-jdk".
- Check Java Version ----> "$ java -version".
- Install SSH ----> "$ sudo apt-get install ssh".
- Install rsync -----> "$ sudo apt-get install rsync". ---------> "Rsync is a fast and extraordinarily versatile file copying tool. It can copy locally, to/from another host over any remote shell, or to/from a remote rsync daemon. "
- Setting SSH (generating private/public key) ------>> "ssh-keygen -t dsa -P ' ' -f ~/.ssh/id_dsa" [DO NOT COPY PASTE].
- Adding SSH keys to Authorized keys ------>> "cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys".
- Go to Hadoop website ---->> "http://www.apache.org/dyn/closer.cgi/hadoop/common/"
- Copy the link address of any mirror links given on above page.
- Download Hadoop ----->> "$ wget -c http://mirror.olnevhost.net/pub/apache/hadoop/common/current/hadoop-2.6.0.tar.gz".
- Go to Download directory from terminal and run the command for extraction ------>> "$ sudo tar -zxvf hadoop-2.6.0.tar.gz".
- Then move the extracted content into Hadoop folder which is under usr/local/hadoop directory ------>> "$ sudo mv hadoop-2.6.0 /usr/local/hadoop".
- Update Java ---->> "$ update-alternatives --config java".
- Edit bashrc file ------>> "$ sudo gedit ~/.bashrc".
- Add the following lines of code to the end of the file ------>>>
- #Hadoop Variables
- export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64
export HADOOP_HOME=/usr/local/hadoop
export PATH=$PATH:$HADOOP_HOME/bin
export PATH=$PATH:$HADOOP_HOME/sbin
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib" - Run this command ----->> "$ source ~/.bashrc".
- Go to directory--------->>> "$ cd /usr/local/hadoop/etc/hadoop"
- Edit Hadoop-env.sh file-------------->> "$ sudo gedit hadoop-env.sh"
- Add following lines in where EXPORT JAVA HOME is written ---------------->> #The java implementation to use.
- export JAVA_HOME="/usr/lib/jvm/java-7-openjdk-amd64"
- Now edit core-site.xml file ------>> "$ sudo gedit core-site.xml". and ADD the following lines of code
- <configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration> - Now edit yarn-site.xml --------->> "$ sudo gedit yarn-site.xml" and ADD the following lines of code
- <configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value> org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
</configuration> - Run the command ---------->>>> "$ sudo cp mapred.site.xml.template mapred-site.xml"
- Edit mapred-site.xml file -------->> "$ sudo gedit mapred-site.xml". and ADD following lines of code.
- <configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration> - Edit hdfs-site.xml ------->>> "$ sudo gedit hdfs-site.xml". and ADD following lines of code
- <configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:/usr/local/hadoop/hadoop_data/hdfs/namenode</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:/usr/local/hadoop/hadoop_store/hdfs/datanode</value>
</property>
</configuration> - Run "cd" command.
- Run following commands to make directory of data node and name node.
- $ sudo mkdir -p /usr/local/hadoop/hadoop_data/hdfs/namenode
- $ sudo mkdir -p /usr/local/hadoop/hadoop_data/hdfs/datanode
- Run this command ------------>> "$ sudo chown username:username -R /usr/local/hadoop"
- Format namenode ---------------->> "$ hdfs namenode -format"
- Run the Hadoop -------->> "$ start-all.sh".
- Check ---------- >> "$ jps".
- Check in your browser -
- http://localhost:8088/
http://localhost:50070/
http://localhost:50090/
http://localhost:50075/