Nobody died, which gave every person I guarded the opportunity to pursue an anti-boring life in whatever way they see fit.
There are a lot of ways to have fun at a pool without flotation devices that endanger lives. I can provide a list if you'd like.
You could introduce a rule that nobody can speak or make any noises that aren't a direct result of swimming, so that when someone is in trouble it's easier to hear them and/or someone else alerting to the problem.
Both of those would still be running the pool, both of them would make it easier to keep people from drowning, and it wouldn't be impossible to have fun. It would be much less fun, but not impossible.
But you've drawn a line that those rules would be going too far, whereas banning flotation devices isn't too far. You've balanced in your mind safety vs. entertainment to see what's worth doing and what isn't. I'm not arguing with your decisions, I know nothing about keeping swimmers safe, I'm just saying in response to "my job was to guard lives, not provide entertainment" that if you really didn't care about anything other than saving their lives you could be much stricter and make the pool much less fun, but a tiny bit more safe.