Hadoop安装

草稿installhttps://help.aliyun.com/zh/ecs/user-guide/build-a-hadoop-environmentvim /opt/hadoop/etc/hadoop/core-site.xmlconfigurationpropertynamehadoop.tmp.dir/namevaluefile:/opt/hadoop/tmp/valuedescriptionlocation to store temporary files/description/propertypropertynamefs.defaultFS/namevaluehdfs://localhost:9000/value/property/configurationvim /opt/hadoop/etc/hadoop/hdfs-site.xmlpropertynamedfs.replication/namevalue1/value/propertypropertynamedfs.namenode.name.dir/namevaluefile:/opt/hadoop/tmp/dfs/name/value/propertypropertynamedfs.datanode.data.dir/namevaluefile:/opt/hadoop/tmp/dfs/data/value/property如果不手动指定Hadoop 默认会将临时数据放在/tmp/hadoop-${user.name}目录下创建用户组和用户groupaddtestuseradd-m-gtesttest# 这种方法允许该用户在本地运行进程但切断了远程访问(推荐)echoDenyUserstest/etc/ssh/sshd_config# 设置密码passwdtestvisudotestALL(ALL)NOPASSWD:ALLchown-Rtest:test /opt/hadoop/Hadoop 的启动脚本是通过SSH登录到各个节点即便只有 localhost 一个节点来启动服务的ssh-keygen cat ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys环境变量的隔离当执行start-dfs.sh时它会启动多个 SSH 会话。这些会话通常是Non-interactive Shell它们不会加载/etc/profile或~/.bashrc。Hadoop 的自救机制hadoop-env.sh是 Hadoop 运行环境的“宪法”。只要在这里定义了无论通过什么方式启动Hadoop 都能准确找到 Java。$vim/opt/hadoop/etc/hadoop/hadoop-env.shexportJAVA_HOME/opt/java/jdk-11.0.30exportHADOOP_HOME/opt/hadoopexportHDFS_NAMENODE_OPTS-Xms256m -Xmx512mexportHDFS_DATANODE_OPTS-Xms256m -Xmx512mexportHDFS_SECONDARYNAMENODE_OPTS-Xms256m -Xmx512mhive 和 Hadoop 的版本保持一致都使用JDK8并且都使用非root用户