command history is also easy to alter if you know what you are doing.
Re the command history, is there any way around them wiping it - e.g. piping all .bash_history entries to an append-only store?
You can also use a global bash configuration that would log all commands [1] entered into any bash shell to a central log, which could be shipped off-server simultaneously.
[0]: http://rkhunter.sourceforge.net
[1]: http://askubuntu.com/questions/93566/how-to-log-all-bash-com...
However, rkhunter has additional logic to specifically seek out rootkits and malware-like behavior, and is more specifically targeted to system file modifications. Combining it with unhide (to compare actual processes running with those visible from userspace) provides a reasonable assurance that nothing nefarious is going on.
They're all part of a spectrum, however. I use scanners like aide alongside rkhunter as well, but I'm the sort of guy that will spend a day tuning the config of aide to avoid constant false alarms.
I'm curious why you think owning a DC makes you less qualified to respond? Presumably because someone that senior is less in touch with day-to-day security operations?
(Just kidding.)
-> disable root being able to login inside your sshd_config file. Make sure PermitRootLogin no
-> rename the root account too so if they are using an exploit based on user authentication then perhaps they won't be able to elevate to root.
-> disable password based logins and go to cert based auth. This will shutdown brute force attacks.
-> lower MaxAuthTries in sshd_config to something like 1 or 2 to help slow down attackers.
-> change the port you listen on. A little security through obscurity while not very effective might slow future casual port scanners that are testing a single port. In practice I've seen this really eliminate a lot of reconnaissance or farming type activities.
-> Make sure you're openssl and openssh are at the latest stable releases.
-> If the perp is coming from the same IP, you can use an iptables rule to block the netblock. Again, not perfect but may help slow things down.
-> grab shell history of all the user accounts on the box. Example ~/.bash_history. This is more reconnaissance but may be helpful if they are sloppy - you might see what they were doing on the box.
-> look for any modified files or new ones. Obviously logs will show up in the list but you're looking for things that should not be there. Example: find / * -mtime -60 -print where 60 is how many days ago you want to go back.
-> look at chron file to see if any timed delay bombs exist.
-> look at ps -aux output for any running processes that don't make sense
-> look at iptraf for any suspicious traffic to IPs you can't reconcile.
Good luck!