您的位置:金沙游乐场85155 > 大数据库 > Hadoop 2.x伪分布式环境搭建详细步骤_数据库其它

Hadoop 2.x伪分布式环境搭建详细步骤_数据库其它

发布时间:2019-12-22 22:49编辑:大数据库浏览(170)

    本文以图文结合的方式详细介绍了Hadoop 2.x伪分布式环境搭建的全过程,供大家参考,具体内容如下

    Hadoop 2.x伪分布式环境搭建步骤:

    1、修改hadoop-env.sh、yarn-env.sh、mapred-env.sh

    1、修改hadoop-env.sh、yarn-env.sh、mapred-env.sh

    方法:使用notepad++打开这三个文件

    方法:使用notepad++(beifeng用户)打开这三个文件

    添加代码:export JAVA_HOME=/opt/modules/jdk1.7.0_67

    添加代码:export JAVA_HOME=/opt/modules/jdk1.7.0_67

    2、修改core-site.xml、hdfs-site.xml、yarn-site.xml、mapred-site.xml配置文件

    2、修改core-site.xml、hdfs-site.xml、yarn-site.xml、mapred-site.xml配置文件

    1)修改core-site.xml

    1)修改core-site.xml

      fs.defaultFS hdfs://Hadoop-senior02.beifeng.com:8020   hadoop.tmp.dir /opt/modules/hadoop-2.5.0/data 
    

    <configuration>
      <property>
        <name>fs.defaultFS</name>
        <value>hdfs://Hadoop-senior02.beifeng.com:8020</value>
      </property>
      <property>
        <name>hadoop.tmp.dir</name>
        <value>/opt/modules/hadoop-2.5.0/data</value>
      </property>
    </configuration>

    2)修改hdfs-site.xml

    2)修改hdfs-site.xml

      dfs.replication 1   dfs.namenode.http-address Hadoop-senior02.beifeng.com:50070 
    

    <configuration>
      <property>
        <name>dfs.replication</name>
        <value>1</value>
      </property>
      <property>
        <name>dfs.namenode.http-address</name>
        <value>Hadoop-senior02.beifeng.com:50070</value>
      </property>
    </configuration>

    3)修改yarn-site.xml

    3)修改yarn-site.xml

      yarn.nodemanager.aux-services mapreduce_shuffle   yarn.resourcemanager.hostname Hadoop-senior02.beifeng.com   yarn.log-aggregation-enable true   yarn.log-aggregation.retain-seconds 86400 
    

    <configuration>
      <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
      </property>
      <property>
        <name>yarn.resourcemanager.hostname</name>
        <value>Hadoop-senior02.beifeng.com</value>
      </property>
      <property>
        <name>yarn.log-aggregation-enable</name>
        <value>true</value>
      </property>
      <property>
        <name>yarn.log-aggregation.retain-seconds</name>
        <value>86400</value>
      </property>
    </configuration>

    4)修改mapred-site.xml

    4)修改mapred-site.xml

      mapreduce.framework.name yarn   mapreduce.jobhistory.webapp.address 0.0.0.0:19888 
    

    <configuration>
      <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
      </property>
      <property>
        <name>mapreduce.jobhistory.webapp.address</name>
        <value>0.0.0.0:19888</value>
      </property>
    </configuration>

    3、启动hdfs

    3、启动hdfs

    1)格式化namenode:$ bin/hdfs namenode -format

    1)格式化namenode:$ bin/hdfs namenode -format

    2)启动namenode:$sbin/hadoop-daemon.sh start namenode

    2)启动namenode:$sbin/hadoop-daemon.sh start namenode

    3)启动datanode:$sbin/hadoop-daemon.sh start datanode

    3)启动datanode:$sbin/hadoop-daemon.sh start datanode

    4)hdfs监控web页面:http://hadoop-senior02.beifeng.com:50070

    4)hdfs监控web页面:http://hadoop-senior02.beifeng.com:50070

    4、启动yarn

    4、启动yarn

    1)启动resourcemanager:$sbin/yarn-daemon.sh start resourcemanager

    1)启动resourcemanager:$sbin/yarn-daemon.sh start resourcemanager

    本文由金沙游乐场85155发布于大数据库,转载请注明出处:Hadoop 2.x伪分布式环境搭建详细步骤_数据库其它

    关键词: