Monday, October 8, 2012

Setting up the Message Passing Interface (MPI)

After configuring SSH with public/private key pairs, and setting up aliases for in /etc/hosts for each PI node, one more useful thing is to update the command prompt to make it easy to know which machine you are working with.  I altered the .bashrc profile on each machine like this:

# ~/.bashrc: executed by bash(1) for non-login shells.
# see /usr/share/doc/bash/examples/startup-files (in the package bash-doc)
# for examples

# JB - Put the machine name in the prompt to make it easier to know which
# machine we are running on.
export PI_NODE=R1


...

if [ "$color_prompt" = yes ]; then
    PS1='${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@$PI_NODE\[\033[00m\] \[\033[01;34m\]\w \$\[\033[00m\] '
else
    PS1='${debian_chroot:+($debian_chroot)}\u@$PI_NODE:\w\$ '
fi

...


Not only does this make the node name apparent in the prompt, it gives us a handle on each machine to use if we want to write tools:

pi@R1 ~ $ echo $PI_NODE
R1
pi@R1 ~ $


Now it's time to install the message passing software - MPI-2.  I'm going to build this form the source, just to get the latest version.  Thanks to Phil Leonard for the pointers to everything.
  1. sudo apt-get install fort77
  2. wget http://www.mcs.anl.gov/research/projects/mpich2/downloads/tarballs/1.4.1p1/mpich2-1.4.1p1.tar.gz
  3. tar zxfv mpich2-1.4.1p1.tar.gz
  4. cd mpich2-1.4.1p1
  5. sudo ./configure
  6. sudo make
  7. sudo make install  
Step 1 was to install the Fortran compiler (fort77), which is no big deal.  Then the rest of it built and installed without any problem.  As Phil mentions in his blog (install option 3), this takes a while - at least a couple of hours, so you may want to plan accordingly.

I got this to build and install properly on my first Pi node, so now I'm off to replicate this on the other machines.  I can install of these machines in parallel of course, but it will still take some time.

Wednesday, October 3, 2012

My First MOOC Class - Neural Nets

Yesterday I took my first online class in the neural networks class offered by Coursera.  I took a set of machine learning classes when I was working on my masters degree at Rensselaer, and I always found neural networks interesting.  Unfortunately our course material never required us to actually create a network, so neural nets were never more than an academic interest for me.

Fortunately, I recognized the name of the instructor for this course when I read the description.  Geoffrey Hinton is a leading name in the area of neural networks, and he was referenced many times in the readings that I had for my masters classes years ago.  This is my chance to actually lay hands on a neural network, and learn from a leading thinker in the area.

My experience with Massively Open Online Courses (MOOCs) is pretty limited right now, but my first impression is positive.  The Coursera site is well constructed and easy to use.  I would recommend it to anyone thinking about taking a class.

The long term goal here is to really learn how to implement neural networks, and then move them onto the Raspberry Pi bramble I'm in the middle of creating.  These networks are inherently parallel beasts, so it should be an interesting exercise.

Monday, October 1, 2012

Making This Parallel Pi Environment Manageable

This seems like a good time to re-cap some configuration steps I've taken, because I don't think I've summarized them very completely so far.  Once the systems are all physically mounted and cabled, all of the SD cards have had the system image installed and they are inserted into each machine, attach a keyboard and monitor to one of the machines, and power it up to perform initial configuration.  You should see the main raspi-config setup screen.  These are the settings that I have completed:
  1.  Expand rootfs.  This is per the recommendations, and will allow the whole SD card to be used.
  2. Configure keyboard.
  3. Change password.
  4. Set timezone.  This will be important if there is a need to debug things.  Timestamps are useful, so having all of the machine set to the correct timezone is necessary.
  5. SSH - enable.  This is very important, because this allows us to reach each of the machines without having to attach a keyboard/mouse/monitor.
  6. Boot behavior - boot to desktop.
  7. Update.
Once this is done, the raspi-config utility should leave you at a command prompt.  Issue a "sudo shutdown -r now" command to re-boot the machine.  When it comes up the next time, there will be a bit of a delay as the system re-blocks the root file system to take advantage of the full size of the SD card.  This should only take an extra minute or two though.  You should then see the Raspian desktop.  Initial configuration is now complete.  Shutdown the machine, move the keyboard and monitor to the next machine, and repeat the config until all of the machines are ready.

Once initial config is complete, you do need to assign fixed IP addresses as I mentioned before.  The steps to making this work really depend on the local network that you have available, so it's not much use to go into detail about how to do this.

One very helpful and common setup step is to edit /etc/hosts, and set up a name for each of your systems.  This way you don't have to use IP addresses every time you need to touch a machine:

# My Stuff
192.168.1.151    RsPi1    RsPi_1    R1    r1
192.168.1.152    RsPi2    RsPi_2    R2    r2
192.168.1.153    RsPi3    RsPi_3    R3    r3
192.168.1.154    RsPi4    RsPi_4    R4    r4
192.168.1.155    RsPi5    RsPi_5    R5    r5
192.168.1.156    RsPi6    RsPi_6    R6    r6
192.168.1.157    RsPi7    RsPi_7    R7    r7
192.168.1.158    RsPi8    RsPi_8    R8    r8


Now that there is an 8-headed monster here to deal with, the challenge becomes one of dealing all of these machines in a simple and efficient manner.  If you want, you can log into each machine via ssh or telnet, and have a console up for each one to control it.  This is very tedious though, since you'll most likely want to do the exact same thing on every machine most of the time.

Fortunately, there is some additional setup that we can perform that allows us to skip the login and password overhead for every action that we want to take.  By using ssh and identity keys, it's possible to make things a lot simpler.  Here is a good HOWTO for setting up ssh identity keys in general:

http://sshkeychain.sourceforge.net/mirrors/SSH-with-Keys-HOWTO/SSH-with-Keys-HOWTO-4.html

I pretty much followed these instructions as presented, but here are the details of my installation.  Note that I am running all of this from an Ubuntu system that is not part of the pi network.  This could be done from Windows or Mac, but there is extra work to setup the environment for things like /etc/hosts, and commands like ssh.
  •   Create a public/private key pair by running ssh-keygen in my home directory. I created a type 2 DSA key:
frodo@ubuntu:~$ ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/home/frodo/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/frodo/.ssh/id_dsa.
Your public key has been saved in /home/frodo/.ssh/id_dsa.pub.
The key fingerprint is:
fa:59:71:1e:ef:a8:e7:b6:66:b0:9d:6f:54:1a:6b:14 frodo@ubuntu
The key's randomart image is:
+--[ DSA 1024]----+
|                 |
|              E  |
|               . |
|              o .|
|        S . o. = |
|       .  .+ o=  |
|      .   .+.+.  |
|       . o. Bo.  |
|        o .B++o  |
+-----------------+

I didn't use any passphrases, because this is the point to setting all of this up - avoid the prompts for passwords or phrases when logging in or issuing commands on the pi machines.
  • Looking in the .ssh directory shows the public and private keys I just created:
frodo@ubuntu:~/.ssh$ ls -l
total 12
-rw------- 1 frodo frodo  672 Oct  1 09:19 id_dsa
-rw-r--r-- 1 frodo frodo  602 Oct  1 09:19 id_dsa.pub
-rw-r--r-- 1 frodo frodo 1998 Oct  1 08:58 known_hosts


  • I copied these over to a a pair of files to make the names more descriptive, and create a backup.  Save these away somewhere, because you can't get them back if they are lost:
frodo@ubuntu:~/.ssh$ ls -l
total 20
-rw------- 1 frodo frodo  672 Oct  1 09:19 frodoTheBrave_dsa
-rw-r--r-- 1 frodo frodo  602 Oct  1 09:19 frodoThebrave_dsa.pub
-rw------- 1 frodo frodo  672 Oct  1 10:23 id_dsa
-rw-r--r-- 1 frodo frodo  602 Oct  1 10:23 id_dsa.pub
-rw-r--r-- 1 frodo frodo 1998 Oct  1 08:58 known_hosts


Note that it's important to keep the the original public (id_dsa.pub) and private (id_dsa) file names, since the private key is used by ssh on the originating machine to match up with the public key that we are about to copy over to the remote pi machine.  In other words, don't re-name these files, just make copies of them

  • Now move the public key over to one of the target machines using scp:
frodo@ubuntu:~/.ssh$ scp frodoThebrave_dsa.pub pi@r1:./frodoThebrave_dsa.pub
pi@r1's password:
frodoThebrave_dsa.pub                                      100%  602     0.6KB/s   00:00   


I have given my target machine a name of "r1" (in /etc/hosts), and the default user name on each of the pi images is "pi", hence the "pi@r1:" address of the remote file.
  • Now log onto the remote machine as user "pi" (using ssh or telnet) to put the public key in the right place.  If you look at the pi's directory, you can see the key that was just copied over:
pi@raspberrypi ~ $ ls -l
total 20
drwxr-xr-x 2 pi pi 4096 Aug 15 20:56 Desktop
drwxr-xr-x 3 pi pi 4096 Sep 18 20:55 Documents
-rw-r--r-- 1 pi pi  602 Oct  1 10:10 frodoThebrave_dsa.pub
drwxrwxr-x 2 pi pi 4096 Jul 20 14:07 python_games
drwxr-xr-x 2 pi pi 4096 Sep 18 20:55 Scratch

  • Create a .ssh directory if one doesn't exist, cd into it, and create a type 2 key file.  This is what will hold the keys for all of the remote users who may want to execute commands on this machine as user pi.  Here are the necessary commands
pi@raspberrypi ~ $ mkdir .ssh
pi@raspberrypi ~ $ cd .ssh
pi@raspberrypi ~/.ssh $ touch authorized_keys2
pi@raspberrypi ~/.ssh $ chmod 600 authorized_keys2
pi@raspberrypi ~/.ssh $ ls -l
total 4
-rw------- 1 pi pi   0 Oct  1 10:13 authorized_keys2
-rw-r--r-- 1 pi pi 666 Sep 28 08:27 known_hosts

  • Now put the public key into the key file, clean up the public key that we copied over, and everything should be ready on the Pi machine:
pi@raspberrypi ~/.ssh $ cat ../frodoThebrave_dsa.pub >> authorized_keys2
pi@raspberrypi ~/.ssh $ cd ..
pi@raspberrypi ~ $ ls -l
total 20
drwxr-xr-x 2 pi pi 4096 Aug 15 20:56 Desktop
drwxr-xr-x 3 pi pi 4096 Sep 18 20:55 Documents
-rw------- 1 pi pi  602 Oct  1 10:10 frodoThebrave_dsa.pub
drwxrwxr-x 2 pi pi 4096 Jul 20 14:07 python_games
drwxr-xr-x 2 pi pi 4096 Sep 18 20:55 Scratch
pi@raspberrypi ~ $ rm frodoThebrave_dsa.pub
pi@raspberrypi ~ $ ls -l .ssh
total 8
-rw------- 1 pi pi 602 Oct  1 10:14 authorized_keys2
-rw-r--r-- 1 pi pi 666 Sep 28 08:27 known_hosts


Note that the authorized_keys2 file in .ssh should now be the same size as the original public key - 602 bytes in this case.  Of course, if this key file contains multiple keys, it will be larger than the public key we just added.  The point is, the key file is a repository of keys for each remote user that will have access to this machine (r1) using this userID (pi).
  • Now we are ready t execute a command from the remote machine, and have it run on the target pi machine without being prompted for a password:
frodo@ubuntu:~$ ssh -2  pi@r1 date
Mon Oct  1 11:58:51 EDT 2012


It's magic, and it's more secure than using passwords, since we're now using RSA encrypted keys to authenticate between the machines.


As with the other setup steps, repeat for all of the other machines so that the whole environment is now set up to use this kind of infrastructure.  This is what we will build our toolset upon.