Host and DNS configurations play a very import role in communication among the components of Hadoop and HBase. So, we need to have the host domain name forward and backward resolvable. This section will cover the different methods to consider for host configurations.
First, we set a hostname for the machine; we change the hostname of the node by changing it into a hostname file, which is found in the /etc/ directory. We can use a command or edit the hostname file.
The following command sets the new hostname for the system, but if we want to make it persistent, we need to open the hostname file from the /etc/ directory and change it in the file:
sudo hostname <name we need to give for the system>
Have a look at the following example:
sudo hostname slave1.infinity.com
Open the file with the following command:
vi /etc/hostname
Alternatively, you can directly use the following command to give a hostname we want to set, for example, infinity1.techinfinity.com:
echo [hostname] > /etc/hostname
Likewise, we can set the hostname in this file for all the nodes we need to have in our cluster.
After changing it in the host file, we need to open the hosts file in the /etc/ directory, which is used for file-based networking. When we open this file using vi /etc/hosts, we will find something like the following lines:
127.0.0.1 localhost 127.0.1.1 <some default name>
We need to change these lines to:
127.0.0.1 localhost #127.0.1.1 <some default name> 192.168.0.2 slave1.infinity.com
Use ifconfig to get the IP address assigned to your Ethernet adapter. You can use ifconfig –a. This will display all the addresses assigned to your machine; just check for the Ethernet. It is better that you assign a static IP to your system because dynamic IP addresses tend to change on reboot. After assigning an IP address and hostname to the file, we add all other machines' IPs and hostnames to all host files in all the machines in the cluster so that they can communicate with each other. Follow this method to set a static IP to the machine:
- Type in the following command to open a file:
sudo vi /etc/network/interfaces - Once the file is opened, make the following changes:
auto eth0 ifaceeth0inet static address 192.168.0.2 netmask 255.255.255.0 network 192.168.0.0 broadcast 192.168.0.255 gateway 192.168.0.1 dns-nameservers 192.168.0.1
- Make changes according to your network settings. You can also make changes using GUI in Ubuntu. Have a look at the following screenshot for reference:
- In the Network Connections window, click on Edit... and enter the information accordingly.
- After making these changes, we can reboot the system or restart the networking services. These changes should be made in all the machines in the cluster. Keep in mind that you need to provide a unique name for all the machines. Restart the networking service using the following command:
sudo /etc/init.d/networking restart - So, once we finish configuring a
hostsfile, we will have ahostsfile on different systems, as follows:127.0.0.1 localhost #127.0.1.1 <some default name> 192.168.0.2 slave1.infinity.com 192.168.0.3 slave2.infinity.com 192.168.0.4 slave3.infinity.com 192.168.0.5 slave4.infinity.com 192.168.0.6 slave5.infinity.com
Once we successfully set up the hostname and IP address correctly, a lot of problems can be avoided during cluster setup.
For DNS-based host configurations, we need to have the IP addresses of all machines and equivalent hostnames. Then, the IP address and hostname pair must be added to the DNS server. For this, we need to have a DNS server that resolves the host to IP. Setting up a DNS server is out of the scope of this book, refer to https://help.ubuntu.com/community/Servers#DNS or https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Identity_Management_Guide/Working_with_DNS.html for details.