Hanne's blog

Very brief blog post (but > 140 characters...)

After buying Sun server and realizing that they are NOT using LSI RAID controllers and MegaCli will not work, do not despair.

Yes it is Adaptec. Or a Sun-ified Adaptec called Storagetek, whatever.

Do what you would otherwise never do: Look on the CD that came with the server. (Or download from Sun's download page http://www.sun.com/servers/x64/x4450/downloads.jsp, but these non-understanding people requires you to register and I can never remember what password I used. Geek appeal -1).

You are looking for something called Storman. The package on the cd is called asm_linux_x64_v5_30_17509.rpm (or x86 if that stuff still exists and it can be alien'ed to debian with no hacks necessary at all:

alien asm_linux_x64_v5_30_17509.rpm

The package storman_5.30-17510_amd64.deb comes out of it and can be installed with

dpkg -i storman_5.30-17510_amd64.deb.
apt-get install libstdc++5.
chmod u+x /usr/StorMan/arcconf.

Now try the following commands:


/usr/StorMan/arcconf GETCONFIG 1 AL
/usr/StorMan/arcconf GETCONFIG 1 Ad


If you have more than one RAID controller I assume the next one has number 2.

The rest is easy. My little RAID monitoring cron entry:
10 * * * * /usr/StorMan/arcconf GETCONFIG 1 Ad |grep "Controller Status" |grep Optimal >>/dev/null || /usr/StorMan/arcconf GETCONFIG 1 AL | /usr/bin/mail myemailaddress@example.com -s "Servername RAID problem!!!"

This has been tried on a Sun Fire X4170 and a Sun Fire X4450 running Debian Lenny.

My Dell support man said I had to upgrade the firmware on my perc6/e controller just to make sure that old firmware was not the reason it threw disks offline.

And he sent me this link:

http://support.euro.dell.com/support/downloads/format.aspx?c=uk&l=en&s=gen&deviceid=12436&libid=46&releaseid=R216021&vercnt=5&formatcnt=0&SystemID=PWE_R900&servicetag=&os=WNET&osl=en&catid=-1&dateid=-1&typeid=-1&formatid=-1&impid=-1

Which has files for DOS/Windows, Redhat / SuSE Enterprise Linux, and ... Windows.

Soooo I know that my Debian Lenny is not a supported platform. And yes this is a choice we have made. It saves us a lot of custom compiling and stuff in our main area - bioinformatics - and cost us a little hassle when our hardware gives trouble.
So I am not whining that Dell does not support non RPM Linux. I am making a note to self, and others, about how I went about to upgrade the firmware anyway.

First of all I should mention that there seems to be a Linux Dell firmware project, which has at least Ubuntu in it, check out http://linux.dell.com/repo/community/. This is not supported by Dell. I cannot tell if it works since I did not use it.

My solution was more back to basics. In short I downloaded a boot CD image with some DOS boot images on it, added the DOS files from the Dell download to it, booted from it and ran the update.

The stepwise explanation is here:

1. Download UBCD (Ultimate Boot CD): http://www.ultimatebootcd.com/download.html
2. Mount it, and copy the contents to somewhere else:

   mount -o loop ubcd411.iso /media/cdrom
   mkdir myubcd
   cp -a /media/cdrom/ myubcd/
   

3. Download the "Hard-Drive" installation package from http://support.euro.dell.com/support/downloads/download.aspx?c=uk&l=en&s=gen&releaseid=R216021&SystemID=PWE_R900&servicetag=&os=WNET&osl=en&deviceid=12436&devlib=0&typecnt=0&vercnt=5&catid=-1&impid=-1&formatcnt=5&libid=46&typeid=-1&dateid=-1&formatid=-1&fileid=307242
   -this is a self extracting exe file.
4. Find a Windows box to unpack it. If you don't have a Windows box, this is the time to call a friend
5. Copy the files that you unpacked to a subdir in your myubcd folder. I called mine dell-fir:

   cp -r path-to-unpacked-stuff/dell-fir myubcd/

6. Create new bootable iso image:

   cd myubcd
   mkisofs -b isolinux/isolinux.bin -c boot.catalog -J -no-emul-boot -boot-load-size 4 -boot-info-table -o ../myubcd.iso .

7. Burn a CD, f.ex.

   wodim dev=/dev/scd0 myubcd.iso
   

8. Boot from the CD. Choose Dos/Linux Boot Disks, then LZ-DOS Boot Disk.
9. When booting into DOS is done, copy the files from the CD to the RAM disk, and run update.bat:

   q:
   copy -r t:dell-fir .
   update.bat

10. It should now display a page allowing you to cancel, or to continue with the upgrade. If you choose to continue, it does the firmware upgrade on the perc6/e controller, if it can find one.

I do not know if copying the files to RAM disk is necessary, but it wants to write some log files and it can't do that unless it has write access. Life was so much easier back in the floppy days (in some ways)...

Needing a Gui remote login for the students

When I started working here at BINF, we were located in an old building and we had our entirely own rather primitive network. But the classrooms and our servers were on the same net - our net. And when our teachers wanted the students to connect to a KDE session on one of our Linux machines, they just told them to install an X server on their Windows or Mac laptop, and connect to one of our workstations, which had all the right programs installed.

It didn't work right out of the box for everybody but that was the situation I inherited, though only for one semester.

We have about 30 students needing to connect at the same time during our courses.

When we moved to a grand new building where the network in the class rooms are in the hands of the IT department of our institute - now our faculty - we had to find another solution.

The first year we taught people to login to one of our workstations with ssh via our login server, start a vnc server on some port, set up an ssh tunnel via our login server, and then connect with vnc. While this worked, it was complex and required a lot of support.

FreeNX nightmare

So last year we heard about FreeNX and decided to try to go with that. This was in one way a success and in one way a nightmare. It was much simpler to set up than the VNC thing since we could use the ssh port and the ssh protocol directly, and let people run on our login server.

The FreeNX server, however, was a nightmare to setup. We did manual little hacked bug fixes in the scripts, we had character set problems we couldn't resolve, but the real problem was that suspend/restore of sessions didn't work and a dead session would clutter people's home directories with large hidden files and at some point we would have to log in to the server and kill sessions manually because it couldn't handle more dead sessions.

Neatx

So this year, we wanted a solution that was basically the same but actually worked. Luckily my student aid came over Google's neatx project. It looks like those people weren't happy with FreeNX either, so they made their own implementation, in a much nicer, cleaner way.

It is not a polished, packaged easy to install project, and there are some things they haven't implemented. But it works. Now I'm risking this claim a bit early, after only 2 days with the new students on the servers, but it actually seems to work. It can suspend and resume sessions. It stores them locally on the server (this can probably be configured).

People can log in, they can work, they can log out. No nightmare at all. And after finding the trick, installing the server is not very complicated at all.

Here is my installation notes and links to documentation.

It is worth mentioning that this year, we are running on a much bigger server. This year we use our 8 core 32 GB RAM server, but it get a load of approx 1.78 at most with 30 users connected, and with a few local users, too.

When they start running heavier programs on it, off course it will be more loaded, but not because of Neatx in itself.

Installing neatx on Debian Lenny

Project page: http://code.google.com/p/neatx/

Download it by checking it out from svn:

svn checkout http://neatx.googlecode.com/svn/trunk/ neatx-read-only

Then follow the instructions in the INSTALL file for installing dependencies. Most of the dependencies comes with Debian and can be installed with apt-get / aptitude:

apt-get install python-pexpect python-simplejson python-gtk2 python-docutils python-gobject python openssh make automake autoconf gcc xauth xrdb netcat

Also install KDE and/or GNOME so people have a GUI to run with neatx.

Now comes the hard part: Install nxagent from http://www.nomachine.com/sources and get it to run.

Documentation: http://www.nomachine.com/documentation/building-components.php

Get the following packages:

• nxcompext-3.3.0-4.tar.gz
• nxcompshad-3.3.0-3.tar.gz
• nxproxy-3.3.0-2.tar.gz
• nxagent-3.3.0-13.tar.gz
• nx-X11-3.3.0-6.tar.gz
• nxauth-3.3.0-1.tar.gz
• nxcomp-3.3.0-4.tar.gz


mkdir NX
cd NX
tar xvzf ../*.tar.gz
cd nx-X11
make World
make install


Now the trick: You actually need to do with your LD_LIBRARY_PATH as it says in the documentation. Not add to your LD_LIBRARY_PATH, but replace your LD_LIBRARY_PATH. Which you would not want to do globally. So you can test by setting a new LD_LIBRARY_PATH in a shell, and start nxagent.

Now install neatx:

neatx-read-only/neatx/
./autogen.sh
./configure
make
make install
useradd --system -m -d /usr/local/var/lib/neatx/home -s /usr/local/lib/neatx/nxserver-login-wrapper nx
install -D -m 600 -o nx /usr/local/share/neatx/authorized_keys.nomachine ~nx/.ssh/authorized_keys
cp /usr/local/share/doc/neatx/neatx.conf.example /usr/local/etc/neatx.conf


In my case I set the export LD_LIBRARY_PATH="/usr/local/src/NX/nxcomp:/usr/local/src/NX/nxcompext:/usr/local/src/NX/nx-X11/exports/lib" in
/usr/local/lib/neatx/nxnode-wrapper and /usr/local/lib/neatx/nxserver-login-wrapper.

It might be smarter to do it somewhere else but this works.

After reboot of the server after a cooling system outage I had a problem: The old sessions were not deleted and NX tried to resume them instead of creating new ones. Users got “Internal error” when they tried to reconnect to a session.

Solution: Put this line in /etc/rc.local:


# Delete all NX sessions after a reboot, they're dead anyway
rm -rf /usr/local/var/lib/neatx/sessions/*


Client

Download the newest client from http://www.nomachine.com/select-package-client.php .

Starting the nx server

You don't have to start anything. If nxagent works and you have followed the instructions, and you have ssh access to the machine, it should work now.

Troubleshooting: Try to login with ssh manually and get rid of any hostkey issues in advance.

Built this, in case anyone else needs it:

http://people.binf.ku.dk/hanne/files/r-base-core_2.9.0-1~lennycran.0_amd64.deb

Use on your own risk, I am probably breaking every rule on package maintenance.

I tried to use the cran repository on new Lenny installation, it has newer versions of everything than Lenny, but the binaries was only i386.

I installed the source from

deb-src http://ftp.sunet.se/pub/lang/CRAN/bin/linux/debian lenny-cran
with
apt-get source r-base-core
and built it with
debuild -us -uc
without making any changes at all. Found method here: http://www.debian-administration.org/articles/20

I probably should add something in a change log somewhere? Well I didn't. Teach me and I shall do better next time.

I think there used to be 64 bit R cran packages? Perhaps I should offer to compile them... but then I better learn the rules first.

[root@whatever ~]# yum list yum-rhn-plugin
Traceback (most recent call last):
  File "/usr/bin/yum", line 28, in ?
    import yummain
  File "/usr/share/yum-cli/yummain.py", line 29, in ?
    from yum import _
ImportError: cannot import name _

At first I though this was because there was no yum repository. But there is not supposed to be! The Redhat enterprise registration somehow takes care of this, I think /etc/sysconfig/rhn/up2date has something to do with it.

The solution was to log into redhat network and download a new yum RPM and install it the manual way

rpm -Uvh yum-3.2.19-18.el5.noarch.rpm

and then it worked again. The old version was yum-3.2.8-7.el5 and I found the package by searching for yum in the "Packages" box near the top at the redhat network web page when I was logged in.