postmortem

[Rick Moen]

First, Bob:  Congratulations on the good job you've done of documenting 
what state your machine was left in.

begin Bob Blick quotation:

> In the /home/httpd/cgi-bin directory there was a program called "...". 

Very typical.

> User "stan" is a valid account of a new user who has never logged in, but
> in his directory there is a new directory "stacheldrahtV4" and it's been
> compiled. Looks like a nasty program with all sorts of devious features.

That's the name of a distributed denial of service (DDoS) tool, based on
the earlier Tribal Flood Network and Trinoo toolkits.
http://staff.washington.edu/dittrich/misc/stacheldraht.analysis

Suffice it to say that your machine was almost certainly to be used to
attack other sites and networks.

> wget http://packetstorm.securify.com/distributed/tfn2k.tgz

And _that_ is some Tribal Flood Network variant.

> I scanned the source of a lot of these programs, and it looks like
> they got into the machine by exploiting rpc.statd.

That's quite plausible.  The last remote root exploit against rpc.statd
I recall was in July 2000.  You may have left a known, documented remote
root exploit open for six months -- maybe more, if the bad guys
exploited an even earlier exploit.

> Never knew what that program was for....

Bzzzt!  One infers that you were running it, and didn't know what it was
for.  Wrong.  Bad move.  Don't run network daemons whose purpose you
don't know.  If you don't know why you're running a given daemon
process, switch it off until you have a good reason.

If you don't know how to determine what network daemons you're running,
yank the network cable until you've studied _Running Linux_ well enough
to determine that.

Monitor security-alert mailing lists (or equivalent) for any advisories
about the network daemons (and kernel versions) you _do_ elect to run.
When you hear about remote security exploit or DoS attacks against that
software, disable the daemons until you've applied the security patch 
or upgrade required to close the hole.

> I haven't deleted anything or reformatted yet, so if anyone has any
> suggestions of more things to do I can still do it.

I _do_ hope you intend a complete rebuild, yes?

http://www.cert.org/tech_tips/win-UNIX-system_compromise.html
http://www.cert.org/tech_tips/unix_configuration_guidelines.html

And don't forget that all of your users' existing passwords have
been compromised.  You need to set them all up with totally new
passwords.  You might also consider chmoding /usr/bin/passwd to 0700 
for a while (after the rebuild), to drive home to the users that they
_must not_ re-adopt their compromised passwords.

Quoting the intruder's .bash_history:

> pico egg.conf
> pico corrupt.conf 

Figures.  LAM3R.  ;->

Typical script kiddie behaviour.

-- 
Cheers,                                      Re-elect Al Gore in '04.
Rick Moen
rick at linuxmafia.com