Virtual distributed graphics and text tutorial by configuring Hadoop with VMWare

Time:2021-4-19

1、 Experimental environment

number project Software and version
1 operating system CentOS6
2 Environment software VMware 12
3 JDK environment jdk1.8.0_181
4 Hadoop Hadoop2.8.5
5 Native operating system Win10 professional

2、 Prepare Linux Network Environment

1.0 click the VMware shortcut, right-click to open the file location and double-click vmnetcfg.exe ->VMnet1 host only > Modify subnet IP setting network segment: 192.168.150.0 subnet mask: 255.255.255.0 > apply > OK

在这里插入图片描述

在这里插入图片描述

在这里插入图片描述

在这里插入图片描述

Back to windows — > open the network and sharing center > > change adapter settings > > right click VMnet1 > > Properties > > double click IPv4 > > set windows IP: 192.168.150.110 subnet mask: 255.255.255.0 > > click OK

在这里插入图片描述

在这里插入图片描述

在这里插入图片描述

在这里插入图片描述

On the virtual software my computer select virtual machine right click settings network adapter host only OK

在这里插入图片描述

在这里插入图片描述

3、 Configure the host name

Modify host name (permanent modification)
Modify hosts file

在这里插入图片描述

在这里插入图片描述

在这里插入图片描述

在这里插入图片描述

4、 Modifying IP in CentOS

Modification via Linux graphical interface (highly recommended)

Enter the Linux graphical interface, right click the two small computers at the top right, click Edit connections, select current network system eth0, click Edit, select IPv4, select method as manual, click add, add IP: 192.168.150.110, subnet mask: 255.255.0, gateway: 192.168.150.1, apply

在这里插入图片描述

在这里插入图片描述

5、 Turn off firewall permanently

View firewall status

在这里插入图片描述

Stop firewall

在这里插入图片描述

Turn off firewall permanently

在这里插入图片描述

6、 Decompress JDK

在这里插入图片描述

Modify the environment variable and source it

在这里插入图片描述

在这里插入图片描述

在这里插入图片描述

7、 Decompress Hadoop

在这里插入图片描述

8、 Modify configuration files (5)

First: Hadoop- env.sh , under / cloud / hadoop-2.8.5/etc/hadoop

Modify about 27 lines, depending on the line number. You can view the file, press ESC, enter a colon, and then enter set Nu to get the line number


export JAVA_HOME=/usr/java/jdk1.8.0_181

在这里插入图片描述

在这里插入图片描述

Second: core- site.xml

<configuration>
		<! -- specify the communication address of HDFS namenode -- >
		<property>
				<name>fs.defaultFS</name>
				<value>hdfs://itcast01:9000</value>
		</property>
		<! -- specifies the storage path of files generated by Hadoop runtime -- >
		<property>
				<name>hadoop.tmp.dir</name>
				<value>/cloud/hadoop-2.8.5/tmp</value>
		</property>
	</configuration>

在这里插入图片描述

Third: mapred- site.xml.template Need to rename: MV mapred- site.xml.template mapred- site.xml

<configuration>
<! -- inform framework MR to use horn -- >
<property>
		<name>mapreduce.framework.name</name>
		<value>yarn</value>
</property>
		</configuration>

在这里插入图片描述

在这里插入图片描述

At the end of this, you can input and modify the name of the XML file


mv mapred-site.xml.template mapred-site.xml

Fourth: HDFS- site.xml

<configuration>
<! -- set the number of HDFS copies -- >
<property>
		<name>dfs.replication</name>
		<value>1</value>
</property>
		</configuration>

在这里插入图片描述

Fifth: Yarn- site.xml

<configuration>
<! -- MapReduce? Shuffle -- > is the way to get data
<property>
	<name>yarn.nodemanager.aux-services</name>
	<value>mapreduce_shuffle</value>
</property>
		</configuration>

在这里插入图片描述

在这里插入图片描述

9、 Add Hadoop to environment variables


vim /etc/profile

	export JAVA_HOME=/usr/java/jdk1.8.0_181
export HADOOP_HOME=/cloud/hadoop-2.8.5
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin
	
		source /etc/profile

在这里插入图片描述

10、 Format HDFS (namenode) to format the first time you use it (forget the screenshot)


hadoop namenode -format

11、 Start Hadoop

Start all
Under / cloud / Hadoop / SBIN /
Enter. / start- all.sh

在这里插入图片描述

And then keep typing in the password and yes

在这里插入图片描述

12、 Enter JPS to see if the configuration is successful

There are six things in the red box, which are indispensable. Otherwise, something is not configured well. According to the experience of the surrounding students, either the network is not configured well, or the above five configuration files are not written well. I really can’t find them. The most simple, rough, troublesome and effective way is to install the machine from the beginning. I succeed in one breath.

在这里插入图片描述

Enter the browser in CentOS

Enter the IP port number 50070 of CentOS from the browser to see the figure below

在这里插入图片描述

Enter the IP port number 8088 of CentOS from the browser, as shown in the figure below, successful

在这里插入图片描述

The above is the whole content of this article, I hope to help you learn, and I hope you can support developer more.

Recommended Today

Libp2p RS version 0.3.0 introduction

V0.3.0 released on 4.23, usingAsyncRead & AsyncWriteTo replace ourReadEx & WriteEx & SplitEx; SimplifiedKad/DHTImplementation logic. modify ReadEx & WriteEx & SplitEx: At first we tried to useasync-traitTo define their own IO operationsTraitFor more pure useasync/awaitTo write code. withReadExFor example, it is roughly as follows: #[async_trait] pub trait ReadEx { async fn read(&mut self, buf: &mut […]