For popular CMS applications you can normally find some helpful .conf file to get you started.
You will also need to setup php-fpm if you are running a PHP flavoured 'LAMP' stack.
Will it be noticeably faster than my current (tuned) Apache setup?
However, as well as speed there is also the amount of load that can be sustained. Nginx wins on this front as well as with speed.
As mentioned, run it on another port and see for yourself. When you are happy with it turn off apache on the chkconfig, set nginx to the other port, stop apache and run nginx -s reload to swap over the port. Yep, you can get the whole thing installed, tested and deployed without reboot or downtime!
At least concerning Drupal, a lot of plugins require changes to the .htaccess file(s), and at least when I tried, this was hard to translate to nginx.
But there are some .htaccess to nginx converters, so I'm not sure if that is a big issue at this time.
And as others have said, you might find the nginx approach to be much more comfortable than .htaccess. I did.
There's only a few modules that require changes to the default ".htaccess" (basically just overriding Apache default logic) of Drupal, like the module Boost. But since Nginx can serve as both a application server and reverse proxy, you can have the best of both worlds, without doing hack-y things like Boost, which basically generates a lot of static files.
You don't want to use those converters, as they usually rely heavily on ifs, which is a bad thing, when things normally can be handled pretty well with the "try_files"-directive.
[1] I've been using it in my stack for a couple of dozen Drupal sites for about two years now.