I have a fence.
A friend's son was visiting, also four years old, vanished for just a second and suddenly I thought Oh god, the pool. Sure enough there he was stuck outside the fence trying to get to the pool but frustrated that the latching mechanism can only be operated by someone at least 5 feet tall.
I wouldn't let a small child go into a pool alone. Pool safety is life and death. Get a fence.
I don't know why my comment above was downvoted, but we did consider buying a house with a pool when our kids were young, and we decided that the risk was ETOOHIGH, especially given that we have a wonderful community pool in the neighborhood.
Pools are expensive to operate and dangerous to have children around -- your own as well as your guests'.
Of course luxury homes with their own pool in the backyard put rather a big strain on safety around your own home. Put a good fence around it, I guess.
...and safety is relative to this child's capability. My parents put me through extensive swimming lessons from a young age precisely so they could let me play unsupervised in the ocean. I'd been a half mile out to sea alone by the time I was 10. Turns out I wasn't as unsupervised as I thought; my mum was freaking out but, unable to swim, couldn't do anything about it!
But if the child can't swim, no alone pool time for them.
Sea, though, can be tricky. Half a mile out to sea, currents can be very different. I know that I as a kid once floated on a tiny inflatable boat quite a bit out to sea, and my dad swam after me to drag me back. I thought I could get back on my own, but my dad clearly wasn't entirely convinced.
You would. Let me give you the scenario: you're home alone with the three kids, you've been chasing them around, doing laundry, cleaning up spilled grape juice, telling Jenny to stop cutting Tommy's hair, etc. Finally, you think everyone is down for a nap. You turn on the game. 5 minutes later, 5 minutes, you think "It's too quiet...". You get up and walk around for a couple minutes to find 4 year-old Sally's door open. No Sally. Where's Sally? Sprint around the house, run down to the kitchen, look in the back yard, and she's face down in the pool. You immediately get her out, desperate. You realize you have to separate from her to call 911. She's been unaccounted for by now for 12 minutes.
The paramedics get a breath back, but anoxic brain injury has set in. She dies, tubes in every orifice, 3 days later.
I have seen this play out more than once. My parents had a pool. I was a lifeguard, have made rescues. I was also on swim team, I'm in the Navy now, and I'm a physician. I surf, I dive, I do open ocean swimming and triathlons. I've helped rescue a diver in pulmonary edema. I think I wouldn't leave my kid unattended, but I know I might.
I've met the parents. They wouldn't let a small child go into a pool alone either.