[NBLUG/talk] Servers rebooting every hour

Troy Arnold troy at zenux.net
Tue Feb 5 22:24:40 PST 2008


On Tue, Feb 05, 2008 at 09:43:24PM -0800, Bob Blick wrote:
> Yep, no sleeping here. By the way, the two machines are 100 miles apart,
> and no other machines are rebooting. Period is more like 60 minutes and
> 35 seconds between reboots. I'm thinking some kind of rootkit that is
> flawed and didn't remove its reboot-after-60-minutes script.
> 
> It'll be a long night, but if anyone hears anything, please post it.

Unless they're especially sophisticated, rootkits are pretty easy to find.

For deb based distro's like Ubuntu, debsums [-s] can help with this.  If
you seriously suspect a rootkit, then of course you can't trust the output
of debsums unless, of course, you use a statically compiled debsums from
elsewhere.  The 'chkrootkit' package can help too.

Still, I'd put my beer on a script or a process that isn't behaving as
intended.  For instance, a couple weeks back on a web application that I
babysit, a database table containing FAQ entries would mysteriously lose
its data.  I first suspected SQL injection or something else nefarious, but
the cause turned out to be a ruby script that I'd repurposed for
another task, while carelessly neglecting to remove the line containing:
dbh.execute('delete from FAQ') 

d'oh!

So, before you pull your hair out looking for that amazingly stealthy
rootkit, checkout the files in /var/spool/crontabs and/or shutdown
crond/atd... and hell, maybe even recursively grep the filesystem for
'reboot'

good luck.

-t




More information about the talk mailing list