I could never forgive myself if a child drowned in my pool and I could have prevented it so I always have everything locked down.
I came home from work early one afternoon and found two girls around 12 in my pool swimming. They had climbed the fence. I ask them nicely to leave and never come back after finding out who their parents were. I called their parents later and they seemed rather put out that I called them about it. My parents would have beat me half to death for doing that when I was young but their parents didn't see much wrong with it.
I have since taken out an "umbrella" policy that protects me a little more for about 100 dollars a year. It gives me a little more sense of security just in case something would ever happen.