MapReduce does not connect to HDFS


Configuration environment Hadoop environment is really fatal, and unexpected problems can arise at any time, such as

Failing this attempt.Diagnostics: Call From Hadoop 001/ to Hadoop 001:8020 failed on connection exception: Deny connection; For more details see: Refused

It’s a strange problem. All the configurations are OK. The problem is on ipv6.

Just disable the IPv6 reboot of the system

vi /etc/default/grub
# cat /etc/default/grub
GRUB_CMDLINE_LINUX="ipv6.disable=1 crashkernel=auto rhgb quiet"
grub2-mkconfig -o /boot/grub2/grub.cfg 

Recommended Today

Hadoop MapReduce Spark Configuration Item

Scope of application The configuration items covered in this article are mainly for Hadoop 2.x and Spark 2.x. MapReduce Official documents…Lower left corner: mapred-default.xml Examples of configuration items name value description mapreduce.job.reduce.slowstart.completedmaps 0.05 Resource requests for Reduce Task will not be made until the percentage of Map Task completed reaches that value. mapreduce.output.fileoutputformat.compress false […]