Maybe keep the keys only in memory in each application? Pull them from some very well monitored service once at startup? Anyone who gained access to the server would have to check the program's memory to get them.