Thursday, January 6, 2011

SSH Setup For HBase

SSH Setup For HBase




SSH passwordless login
 Login with password
 Login with passphrase
 Login without password
HBase
 Configure HBase
 Start HBase
References




HBase can be used as datastore for Nutch 2.0.

This is a tutorial to get started quickly with a standalone instance of HBase. I did not find it so straightforward on the original guide, hence this blog entry.



SSH passwordless login


We are going to setup the SSH environment for HBase. Personally I just use a standalone instance of HBase on a single machine, so the server and client are the same. I use Debian as Linux OS.

$ whoami
alex

First, if trying to login with no instance of the SSH daemon running, I get this error:

$ ssh alex@localhost
ssh: connect to host localhost port 22: Connection refused

You want to setup an SSH server. I installed the Debian package:
# apt-get install openssh-server

This will start it automatically. In case you need that later, this is how to start it:

$ sudo /etc/init.d/ssh start
Starting OpenBSD Secure Shell server: sshd.

Login with password


Now you can login to the server by using your regular linux user password:

$ ssh alex@localhost
The authenticity of host 'localhost (127.0.0.1)' can't be established.
RSA key fingerprint is b6:96:06:e1:fb:f1:9f:23:40:32:ac:cb:ac:c9:bc:12.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'localhost' (RSA) to the list of known hosts.
alex@localhost's password:  
Linux maison 2.6.32.24 #1 SMP Tue Oct 5 08:36:28 CEST 2010 i686

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
You have mail.
Last login: Sun Jan  2 13:32:46 2011 from localhost
$ exit


On the client, you want to generate an SSH key:

$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/alex/.ssh/id_rsa):    
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/alex/.ssh/id_rsa.
Your public key has been saved in /home/alex/.ssh/id_rsa.pub.
The key fingerprint is:
47:79:b7:e8:e4:25:1b:8d:f6:0a:86:36:67:fa:1d:94 alex@maison
The key's randomart image is:
+--[ RSA 2048]----+
|                 |
|           .     |
|          o . .  |
|         . . * . |
|        S . E +  |
|         o * *   |
|        + = = .  |
|       . * o o   |
|        ... o    |
+-----------------+
$ chmod 755 ~/.ssh


Copy the public key to the SSH server.


$ scp ~/.ssh/id_rsa.pub alex@localhost:.ssh/authorized_keys
alex@localhost's password: 
id_rsa.pub                                                                                                                                       100%  393     0.4KB/s   00:00


Now on the server:

$ su alex
$ chmod 600 ~/.ssh/authorized_keys



Login with passphrase


From the client, login to the server with your passphrase:

$ ssh alex@localhost
Enter passphrase for key '/home/alex/.ssh/id_rsa': 
Linux maison 2.6.32.24 #1 SMP Tue Oct 5 08:36:28 CEST 2010 i686

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
You have mail.
Last login: Wed Dec 29 15:57:24 2010 from localhost
$ exit

On the client, start an SSH agent to avoid typing your passphrase in the future:+

$ exec /usr/bin/ssh-agent $SHELL
$ ssh-add
Enter passphrase for /home/alex/.ssh/id_rsa: 
Identity added: /home/alex/.ssh/id_rsa (/home/alex/.ssh/id_rsa)
$ nohup ssh-agent -s > ~/.ssh-agent
nohup: ignoring input and redirecting stderr to stdout

Login without password


Now login to the server. It should be passwordless.

$ ssh alex@localhost
Linux maison 2.6.32.24 #1 SMP Tue Oct 5 08:36:28 CEST 2010 i686

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
You have mail.
Last login: Sun Jan  2 13:41:59 2011 from localhost
$ exit


HBase


Now that we have setup SSH, we can configure HBase.

Configure HBase


Change HBase properties in conf/hbase-site.xml according to your needs. For example you can change the directory where the data is stored.

<configuration>
  <property>
    <name>hbase.rootdir</name>
    <value>file:///home/alex/hbase/hbase-${user.name}/hbase</value>
  </property>
  <property>
    <name>hbase.tmp.dir</name>
    <value>/home/alex/hbase/hbase-${user.name}</value>
  </property>
</configuration>


Start HBase


$ bin/start-hbase.sh 
localhost: starting zookeeper, logging to /home/alex/java/ext/hbase-0.20.6/bin/../logs/hbase-alex-zookeeper-maison.out
starting master, logging to /home/alex/java/ext/hbase-0.20.6/logs/hbase-alex-master-maison.out
localhost: starting regionserver, logging to /home/alex/java/ext/hbase-0.20.6/bin/../logs/hbase-alex-regionserver-maison.out

These 2 Java processes are now running on the machine:

  • org.apache.hadoop.hbase.zookeeper.HQuorumPeer
  • org.apache.hadoop.hbase.master.HMaster



References


1 comment:

  1. Thanks for the post, it worked perfectly on Ubuntu 11.04!

    ReplyDelete