В порядке ли моя файловая система Hadoop с сообщением о неверной конфигурации?

Я пытаюсь установить hadoop на свой ноутбук в учебных целях с помощью учебника из пункта tutorialspoint. Я запустил start-dfs.sh:

Ожидаемый результат был:

10/24/14 21:37:56
Starting namenodes on [localhost]
localhost: starting namenode, logging to /home/hadoop/hadoop-
2.4.1/logs/hadoop-hadoop-namenode-localhost.out
localhost: starting datanode, logging to /home/hadoop/hadoop-
2.4.1/logs/hadoop-hadoop-datanode-localhost.out
Starting secondary namenodes [0.0.0.0]

Но я получаю:

mike@mike-thinks:/usr/local/hadoop/sbin$ ./start-dfs.sh 
Incorrect configuration: namenode address dfs.namenode.servicerpc-address or dfs.namenode.rpc-address is not configured.
Starting namenodes on []
localhost: mkdir: cannot create directory ‘/usr/local/hadoop/logs’: Permission denied
localhost: chown: cannot access '/usr/local/hadoop/logs': No such file or directory
localhost: starting namenode, logging to /usr/local/hadoop/logs/hadoop-mike-namenode-mike-thinks.out
localhost: /usr/local/hadoop/sbin/hadoop-daemon.sh: line 159: /usr/local/hadoop/logs/hadoop-mike-namenode-mike-thinks.out: No such file or directory
localhost: head: cannot open '/usr/local/hadoop/logs/hadoop-mike-namenode-mike-thinks.out' for reading: No such file or directory
localhost: /usr/local/hadoop/sbin/hadoop-daemon.sh: line 177: /usr/local/hadoop/logs/hadoop-mike-namenode-mike-thinks.out: No such file or directory
localhost: /usr/local/hadoop/sbin/hadoop-daemon.sh: line 178: /usr/local/hadoop/logs/hadoop-mike-namenode-mike-thinks.out: No such file or directory
localhost: mkdir: cannot create directory ‘/usr/local/hadoop/logs’: Permission denied
localhost: chown: cannot access '/usr/local/hadoop/logs': No such file or directory
localhost: starting datanode, logging to /usr/local/hadoop/logs/hadoop-mike-datanode-mike-thinks.out
localhost: /usr/local/hadoop/sbin/hadoop-daemon.sh: line 159: /usr/local/hadoop/logs/hadoop-mike-datanode-mike-thinks.out: No such file or directory
localhost: head: cannot open '/usr/local/hadoop/logs/hadoop-mike-datanode-mike-thinks.out' for reading: No such file or directory
localhost: /usr/local/hadoop/sbin/hadoop-daemon.sh: line 177: /usr/local/hadoop/logs/hadoop-mike-datanode-mike-thinks.out: No such file or directory
localhost: /usr/local/hadoop/sbin/hadoop-daemon.sh: line 178: /usr/local/hadoop/logs/hadoop-mike-datanode-mike-thinks.out: No such file or directory
Starting secondary namenodes [0.0.0.0]
The authenticity of host '0.0.0.0 (0.0.0.0)' can't be established.
ECDSA key fingerprint is SHA256:/s7X8QMliB6FVx5bde5AaCycprQ/B+NtcTXrInrXxJM.
Are you sure you want to continue connecting (yes/no)? no
0.0.0.0: Host key verification failed.

Поэтому я попытался с помощью команды sudo:

mike@mike-thinks:/usr/local/hadoop/sbin$ sudo ./start-dfs.sh 
Incorrect configuration: namenode address dfs.namenode.servicerpc-address or dfs.namenode.rpc-address is not configured.
Starting namenodes on []
localhost: starting namenode, logging to /usr/local/hadoop/logs/hadoop-root-namenode-mike-thinks.out
localhost: starting datanode, logging to /usr/local/hadoop/logs/hadoop-root-datanode-mike-thinks.out
Starting secondary namenodes [0.0.0.0]
The authenticity of host '0.0.0.0 (0.0.0.0)' can't be established.
ECDSA key fingerprint is SHA256:/s7X8QMliB6FVx5bde5AaCycprQ/B+NtcTXrInrXxJM.
Are you sure you want to continue connecting (yes/no)? yes
0.0.0.0: Warning: Permanently added '0.0.0.0' (ECDSA) to the list of known hosts.
0.0.0.0: starting secondarynamenode, logging to /usr/local/hadoop/logs/hadoop-root-secondarynamenode-mike-thinks.out

Неправильная конфигурация поставила меня в тупик ...

Затем я попытался запустить пряжу:

mike@mike-thinks:/usr/local/hadoop/sbin$ sudo ./start-yarn.sh 
[sudo] password for mike: 
starting yarn daemons
starting resourcemanager, logging to /usr/local/hadoop/logs/yarn-root-resourcemanager-mike-thinks.out
nice: ‘/usr/local/hadoop/bin/yarn’: Permission denied
localhost: starting nodemanager, logging to /usr/local/hadoop/logs/yarn-root-nodemanager-mike-thinks.out
localhost: nice: ‘/usr/local/hadoop/bin/yarn’: Permission denied

Я сделал chmod +x на yarn

mike@mike-thinks:/usr/local/hadoop/sbin$ sudo ./start-yarn.sh 
starting yarn daemons
starting resourcemanager, logging to /usr/local/hadoop/logs/yarn-root-resourcemanager-mike-thinks.out
localhost: starting nodemanager, logging to /usr/local/hadoop/logs/yarn-root-nodemanager-mike-thinks.out

, но у меня не было доступа http://localhost:50070

Я повторил попытку и теперь мне нужно бежать против моих демонов:

mike@mike-thinks:/usr/local/hadoop/sbin$ sudo ./start-yarn.sh 
starting yarn daemons
starting resourcemanager, logging to /usr/local/hadoop/logs/yarn-root-resourcemanager-mike-thinks.out
localhost: nodemanager running as process 8183. Stop it first.
mike@mike-thinks:/usr/local/hadoop/sbin$ sudo kill 8183
mike@mike-thinks:/usr/local/hadoop/sbin$ sudo ./start-yarn.sh 
starting yarn daemons
resourcemanager running as process 9513. Stop it first.
localhost: starting nodemanager, logging to /usr/local/hadoop/logs/yarn-root-nodemanager-mike-thinks.out
mike@mike-thinks:/usr/local/hadoop/sbin$ sudo kill 9513
mike@mike-thinks:/usr/local/hadoop/sbin$ sudo ./start-yarn.sh 
starting yarn daemons
starting resourcemanager, logging to /usr/local/hadoop/logs/yarn-root-resourcemanager-mike-thinks.out
localhost: nodemanager running as process 10058. Stop it first.

И все же я могу перейти на страницу «Все приложения Hadoop» на http://localhost:8088/:

Hadoop All applications page [тысяча сто двадцать-пять]

0
задан 6 December 2017 в 17:25

1 ответ

У меня (пользователь archlinux) была такая же проблема в Hadoop 3.0.0. Но я знаю некоторые соображения об этом.

Итак, попробуйте

  1. запустить команду jps и проверьте наличие «NameNode»
    , если вы не можете найти NameNode, вам следует повторить свой Hadoop.
  2. выполнить команду telnet localhost 8088/50070 и проверить соединение
    Я не могу подключить порт 50070, но могу подключить порт 8088 ... (Я не смог найти это решение)
0
ответ дан 6 December 2017 в 17:25

Другие вопросы по тегам:

Похожие вопросы: