It's more complicated than just turning off swap. Even without swap, when you run extremely low on memory the page cache will be squeezed to nothing and you'll thrash. And I don't think there's a way to set a minimum page cache on Linux.
So in other words, I would not rely on the OOM killer ever kicking in.
In a situation with more memory use over time a small swap can act as a canary, letting you know you're nearing a performance drop, while a swapless server could suddenly hit a molasses wall without dying.
If you want a canary, why not just directly monitor the page cache size?
Keep in mind that if your swap is not huge, it will very often fill will cold data and you won't have swap thrashing despite running out of memory.
Good point about measuring the page cache directly.
There was a fair amount of interesting engineering to make swap effective.. For example, for the code (text) segment, which was read-only, you'd use the file itself (e.g. a.out) as the swap for the text (code) portion, and if you'd try to `rm a.out` while it was running, you'd get a "text file busy error".
Often times it's better to have a dead system (you do have HA and auto-failover, right?) than to let difficult-to-track-down "slowness" into real time systems.