hadoop偽分布式的安裝步驟

本篇內容主要講解“hadoop偽分布式的安裝步驟”，感興趣的朋友不妨來看看。本文介紹的方法操作簡單快捷，實用性強。下面就讓小編來帶大家學習“hadoop偽分布式的安裝步驟”吧!

1. 解壓縮 /opt/software/hadoop-2.8.1.tar.gz 文件 
    [root@hadoop002 software]$ cd   /opt/software/
    [root@hadoop002 software]$ pwd
    /opt/software
    [root@hadoop002 software]$ tar -xzvf  hadoop-2.8.1.tar.gz
    [root@hadoop002 software]$ ll
    total 208784
    drwxr-xr-x.  6 root   root        4096 Nov 10  2015 apache-maven-3.3.9
    -rw-r--r--.  1 root   root     8617253 Apr 23 15:14 apache-maven-3.3.9-bin.zip
    drwxr-xr-x.  7 root   root        4096 Aug 21  2009 findbugs-1.3.9
    -rw-r--r--.  1 root   root     7546219 Apr 23 15:14 findbugs-1.3.9.zip
    drwxr-xr-x. 10 root   root   4096 Apr 23 16:36 hadoop-2.8.1
    -rw-r--r--.  1 root   root   194976866 Apr 23 15:40 hadoop-2.8.1.tar.gz
    drwxr-xr-x. 10 109965   5000      4096 Apr 23 15:26 protobuf-2.5.0
    -rw-r--r--.  1 root   root     2401901 Apr 23 15:15 protobuf-2.5.0.tar.gz

2. 添加軟連接  
    [root@hadoop002 software]$ ln -s /opt/software/hadoop-2.8.1 /opt/software/hadoop
    [root@hadoop002 software]$ ll
    total 208784
    drwxr-xr-x.  6 root   root        4096 Nov 10  2015 apache-maven-3.3.9
    -rw-r--r--.  1 root   root     8617253 Apr 23 15:14 apache-maven-3.3.9-bin.zip
    drwxr-xr-x.  7 root   root        4096 Aug 21  2009 findbugs-1.3.9
    -rw-r--r--.  1 root   root     7546219 Apr 23 15:14 findbugs-1.3.9.zip
    lrwxrwxrwx.  1 root    root   26 Apr 23 15:41 hadoop -> /opt/software/hadoop-2.8.1
    drwxr-xr-x. 10 root    root   4096 Apr 23 16:36 hadoop-2.8.1
    -rw-r--r--.  1 root   root   194976866 Apr 23 15:40 hadoop-2.8.1.tar.gz
    drwxr-xr-x. 10 109965   5000      4096 Apr 23 15:26 protobuf-2.5.0
    -rw-r--r--.  1 root   root     2401901 Apr 23 15:15 protobuf-2.5.0.tar.gz


3.  設置環境變量
    [root@hadoop002 software]$  vi /etc/profile
    export HADOOP_HOME=/opt/software/hadoop
    export PATH=$HADOOP_HOME/bin:$FINDBUGS_HOME/bin:$PROTOC_HOME/bin:$MAVEN_HOME/bin:$MYSQL_HOME/bin:$JAVA_HOME/bin:$PATH

    --  環境變量生效
    [root@hadoop002 software]$ source /etc/profile


4. 修改軟連接所屬的用戶和用戶組
    [root@hadoop002 software]$ chown -R hadoop:hadoop hadoop/*

5. 將hadoop-2.8.1文件的所屬的用戶和用戶組 修改為hadoop
    [root@hadoop002 software]$ chown -R  hadoop:hadoop hadoop-2.8.1
    [root@hadoop002 software]$ chown -R  hadoop:hadoop hadoop-2.8.1/*

6.  切換hadoop用戶
    [root@hadoop002 hadoop]# su - hadoop
    [hadoop@hadoop002 ~]$ cd /opt/software/hadoop
    [hadoop@hadoop002 hadoop]$ ll
    total 148
    drwxr-xr-x. 2 hadoop hadoop  4096 Apr 18 14:11 bin
    drwxr-xr-x. 3 hadoop hadoop  4096 Apr 18 14:10 etc
    drwxr-xr-x. 2 hadoop hadoop  4096 Apr 18 14:11 include
    drwxr-xr-x. 3 hadoop hadoop  4096 Apr 18 14:10 lib
    drwxr-xr-x. 2 hadoop hadoop  4096 Apr 18 14:11 libexec
    drwxr-xr-x. 2 hadoop hadoop  4096 Apr 18 14:11 sbin
    drwxr-xr-x. 3 hadoop hadoop  4096 Apr 18 14:10 share

    bin：可執行文件
    etc：配置文件
    sbin： shell腳本  啟動，關閉hdfs，yarn服務

7. # 刪除txt文件
    [hadoop@hadoop002 hadoop]$ rm -f *.txt

8. 配置hadoop用戶的ssh信任關系
    [hadoop@hadoop002 hadoop]$ ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
    Generating public/private rsa key pair.
    Created directory '/home/hadoop/.ssh'.
    Your identification has been saved in /home/hadoop/.ssh/id_rsa.
    Your public key has been saved in /home/hadoop/.ssh/id_rsa.pub.
    The key fingerprint is:
    29:06:4c:1a:a4:74:b4:f4:c2:43:8a:07:68:00:fa:e4 hadoop@hadoop002 
    The key's randomart image is:
    +--[ RSA 2048]----+
    |Bo+=.         |
    |=+*=o         |
    |=.+=o.        |
    | =  o.   .      |
    |  E   o S      |
    |     . .         |
    |                 |
    |                 |
    |                 |
    +-----------------+

    [hadoop@hadoop002 hadoop]$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
    [hadoop@hadoop002 hadoop]$ chmod 0600 ~/.ssh/authorized_keys

    [hadoop@hadoop002 hadoop]$ ssh hadoop002 date  # 第一次回車需要輸入yes
    The authenticity of host 'hadoop002 (192.168.90.164)' can't be established.
    RSA key fingerprint is 3a:51:6d:9b:94:d3:91:bf:fd:ab:da:0a:5b:8c:f2:6c.
    Are you sure you want to continue connecting (yes/no)? yes
    Warning: Permanently added 'hadoop002 ,192.168.90.164' (RSA) to the list of known hosts.
    Tue May 22 10:46:32 CST 2018

[hadoop@hadoop002 hadoop]$ ssh hadoop002 date # 不需要回車輸入yes，即OK
    Tue May 22 10:46:37 CST 2018

9. 配置文件
    [hadoop@hadoop002 hadoop]$ pwd
    /opt/software/hadoop
    [hadoop@hadoop002 hadoop]$  vi etc/hadoop/core-site.xml
    
        
            fs.defaultFS
            hdfs://localhost:9000
        
    


    [hadoop@hadoop002 hadoop]$  etc/hadoop/hdfs-site.xml

   
        dfs.replication
        1
   

10.   格式化 和 啟動
    # 添加環境變量
    [hadoop@hadoop002 hadoop]$ vi etc/hadoop/hadoop-env.sh
    export JAVA_HOME=/usr/java/jdk1.8.0_11

    #查看hdfs命令的路徑
    [hadoop@hadoop002 hadoop]$ which hdfs
    /opt/sofeware/hadoop/bin/hdfs

    #格式化
    [hadoop@hadoop002 hadoop]$  bin/hdfs namenode -format
    ........
    18/04/19 12:53:42 INFO common.Storage: Storage directory /tmp/hadoop-hadoop/dfs/name has been successfully formatted.

    # 啟動hdfs
    [hadoop@hadoop002 hadoop]$ sbin/start-dfs.sh
    Starting namenodes on [192.168.90.164]
    192.168.90.164: namenode running as process 15421. Stop it first.
    localhost: datanode running as process 15523. Stop it first.
    Starting secondary namenodes [0.0.0.0]
    0.0.0.0: secondarynamenode running as process 15717. Stop it first.

    #通過 jps查看進程
    [hadoop@hadoop002 hadoop]$ jps
    15523 DataNode
    15717 SecondaryNameNode
    15421 NameNode
    16877 Jps

11  修改dfs啟動的進程以hadoop002啟動
    #修改前的啟動
    [hadoop@hadoop002 hadoop]$ sbin/start-dfs.sh 
    Starting namenodes on [192.168.90.164]
    192.168.90.164: starting namenode, logging to /opt/software/hadoop-2.8.1/logs/hadoop-hadoop-namenode-localhost.localdomain.out
    localhost: starting datanode, logging to /opt/software/hadoop-2.8.1/logs/hadoop-hadoop-datanode-localhost.localdomain.out
    Starting secondary namenodes [0.0.0.0]
    0.0.0.0: starting secondarynamenode, logging to /opt/software/hadoop-2.8.1/logs/hadoop-hadoop-secondarynamenode-localhost.localdomain.out


    配置 在hadoop002上啟動
    三個進程：
    namenode：         192.168.90.164    bin/hdfs getconf -namenodes    etc/hadoop/core-site.xml        
    datanode：         localhost         using default slaves file     etc/hadoop/slaves  # 修改slaves文件里面的內容為hadoop002
    secondarynamenode： 0.0.0.0


    #修改文件：etc/hadoop/core-site.xml:
    
       
            fs.defaultFS
            hdfs://hadoop002:9000
        
    

    # 修改slaves文件里面的內容為hadoop002
    [hadoop@hadoop002 hadoop]$ vi  etc/hadoop/slaves
    #localhost
    hadoop002

    #修改文件：etc/hadoop/hdfs-site.xml 
    添加：
    
       
                dfs.namenode.secondary.http-address
                hadoop002:50090 #默認是0.0.0.0:50090
        
        
                dfs.namenode.secondary.https-address
                hadoop002:50091#默認是0.0.0.0:50091
        
    

    0.0.0.0 是一個非常特殊的IP，代表的是當前機器的IP。

    # 配置好后，再次啟動
    [hadoop@hadoop002 sbin]$  sbin/start-dfs.sh
    Starting namenodes on [hadoop002]
    hadoop002: starting namenode, logging to /opt/software/hadoop-2.8.1/logs/hadoop-hadoop-namenode-hadoop002.out
    hadoop002: starting datanode, logging to /opt/software/hadoop-2.8.1/logs/hadoop-hadoop-datanode-hadoop002.out
    Starting secondary namenodes [hadoop002]
    hadoop002: starting secondarynamenode, logging to /opt/software/hadoop-2.8.1/logs/hadoop-hadoop-secondarynamenode-hadoop002.out

    到這里就安裝完成了。

到此，相信大家對“hadoop偽分布式的安裝步驟”有了更深的了解，不妨來實際操作一番吧！這里是億速云網站，更多相關內容可以進入相關頻道進行查詢，關注我們，繼續學習！