Of course, security issues these days are more often the result of misconfiguration but if you're doing something simple like a single box with a localhost only MySQL, Apache/Nginx and Rails/Python/PHP or the like then it's pretty straight forward. Don't really even need a firewall.
What I would suggest is locking down SSH to not allow root login and to require key authentication and deny password auth. So much automated SSH password guessing bot spam out there.
http://news.ycombinator.com/item?id=1025520
I think a new admin needs to read up on locking down ports in iptables, Bastille, snort, filesystem fingerprinting and some checklists:
http://www.mnxsolutions.com/blog/apache/securing-your-server...
http://blog.dhananjaynene.com/2009/10/configuring-a-secure-u...
Something I've these -- who is responsible for security updates? I assume it's the deployer, which can often be someone who may not have much experience as an sysadmin. What's the best way to keep a system secure? Cron + Apt-get update?