work_mem is a per-sort setting. So if you have 20 users connected, running a single query with 4 sorts each, that's 80 x work_mem. If it's set to GB anything, that's going to get out of hand very fast.

I recommend this guide.. it's a good start: http://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Serve...