I’m a little nervous when I write a column for the first time. Ha ha, how to start? En… That’s it. I’m working on Hadoop recently. Then I found a video tutorial of shangxuetang on big data. Unfortunately, the video was not complete. It started directly from the middle, and the environment was not built. I had to listen to the class and grope for it. Fortunately, I was a rookie to finish it. Let me sum up for encouragement. Here we go.
1. First of all, we don’t have so many people. Grandfather Mao can’t afford to buy multiple computers, let alone servers, so we can only build them under virtual machines. Speaking of learning, it must be in the virtual machine pull-down. The virtual machine I use here is: VirtualBox. The download of this direct Baidu has, the installation of words, the next step is OK.
3. Next, install your Ubuntu under the virtual machine. Start the virtual machine as shown in the figure below. However, I have installed five virtual machines (in the red circle), and the VirtualBox just installed is not available. That is, there is nothing in the red circle. Forgive me for my nonsense. Above:
3.1. Click the light blue new button in the upper left corner to create a new virtual machine. At this time, three input boxes will appear. The first is the name of the virtual machine, the second is the type of the virtual machine, and the third is the version. Select the corresponding one and it will be OK. Finish the next step. As shown in the figure below:
The following are some virtual machine settings that can be changed or not changed. You can directly select the next step to the end, and then you will see the virtual machine you just created in the left column of the VirtualBox main interface (the one in the red circle is the one just created)
However, this virtual machine has not been installed with a system, so it cannot be started. Let’s start installing the Ubuntu 14 system on the node5 virtual machine.
4. Select the virtual machine that you want to install the system (light blue means that it is selected. The operation of this virtual machine can only be performed if the following operation is selected, otherwise it will be wrong). Then click the settings button in the upper left corner, then click the storage button in the top bar, and then click empty under the controller ide. The interface at this time should be as follows:
Then click the small CD button in the yellow circle on the right, select your Ubuntu image, and then click OK to return to the
VirtualBox main interface. Click the start button to start the virtual machine. Then we enter the virtual machine operating system installation process. First of all, you need to select the operating language. English is recommended. There are all kinds of strange problems in Chinese. At this time, we need to pay attention to the problem of switching between the virtual machine and the host computer. Under windows, ALT + Ctrl is used. Under the Mac, press and hold the command key, move the mouse to move the mouse out of the virtual machine. To enter, you can click the virtual machine (sometimes you need it, sometimes you can move it out directly, maybe because of the problem of stuck). Then press the Enter key until it is shown in the following figure:
Here you need to enter the host name, and then the keyboard tab key to move the cursor to continue, and then you need to enter the full name of a user for your virtual machine. After that, it is the account number of the user, followed by the password. These are optional, not Chinese. The next step is to continue until you see the following figure:
At this point, you need to use the keyboard Shift + Tab key to move the red cursor to yes, and then press enter. Next, as shown in the figure below, you need to move the cursor to yes, then press enter, and then wait a long time. A check box will pop up in the middle, and it will be OK to default directly,
5. After completing the above, you should see the login reminder of the virtual machine, as shown in the following figure:
After successful login, we will switch the user to the root account, enter sudo passwd to set the password for root, and then enter Su root, and then enter the password of root to switch to the root account.
6. Next, we need the virtual machine to be able to link to the network, because we need to download and install some software. VirtualBox defaults to the network NAT of the virtual machine, and nat only allows the virtual machine to link the host and the external network, The host computer is not allowed to link to the virtual machine, and the virtual machine can not communicate directly. Therefore, I set the network as bridged adapter bridge. In this mode, there is an unsafe factor, that is, external machines can also link to virtual machines, so learn to use it for testing. Operation: select network in setting, and change the attached to item to bridged adapter, Then restart the virtual machine and enter ifconfig to see that the virtual machine has an independent IP. Then we need to modify the configuration file: VI / etc / SSH / sshd_ The permitrotelogin attribute under config is yes, which allows remote login of root account. Then the virtual machine also needs to install openssh client and openssh server before the host can link to the virtual machine. Type the following command to install: sudo apt get install openssh client / openssh server, enter: PS – e | grep SSH; to confirm whether ssh server is started, you can enter sudo / etc / init.d/ssh start to start. Now the host can log in to the virtual machine: SSH [email protected] 。 You can modify the hosts file of the host, instead of entering the IP of the virtual machine every time.
7. Type the following command to copy the Hadoop installation package to the virtual machine: SCP Hadoop 1.2.1 tar.gz [email protected] :/root
8. Password free login between virtual machines:
8.1: Secret free login generates public key and private key: SSH keygen - t RSA 8.2: SSH root @ IP "MKDIR. SSH; Chmod 0700. SSH" of the password free login machine If there is a. SSH file under the root of the login machine, you can omit it. Pay attention to the permission setting of the file 8.3 copy the public key generated by the host to the login. SSH file: scp ~/.ssh/id_ rsa.pub [email protected] :.ssh/id_ rsa.pub (password required) 8.4 create authorized under / root /. SSH / of the login side_ If the keys file exists, you can omit this step 8.5 cat /root/.ssh/id_ rsa.pub >> /root/.ssh/authorized_ Key (set ID_ rsa.pub The content of is appended to authorized_ In keys, be careful not to use >, otherwise the original content will be cleared, making other people unable to use the original key to log in) 8.6 back to the host, you can log in to the logged in terminal without password
9. The specific configuration of Hadoop single node can be seen directly herehttp://hadoop.apache.org/docs/r1.2.1/single_node_setup.html
Or a fully distributed multi node systemhttps://segmentfault.com/a/1190000005744663