Poolbee,

The solar cover is designed with the bubbles so that it actually reflects the sunlight to the water, acting as a solar collector directly on your pool. It also helps retain the heat that your heater has been generating.
Leaving your solar cover on at night is a definate "MUST DO" as this is the period of time of your greatest heat lost.
Leaving your solar cover on during the day, especially when it's sunny out is also a "MUST DO", and removing it when you actually go to use your pool.
It was pointed out by Phantom Andy that he's better with the cover off, when there is no wind and hotter ambient air temps than water temps.
This will change from morning to afternoon and unless you know when to do this from habit, and are around, you're going to even out the overall effects of the heating/cooling argument.

All in all, the solar cover is designed to help retain the heat in the pool. As Carl said, With it on, period! I agree. Take it off to go swimming.

Another good rule of thumb, point your return jets downward. Your pool is best heated from the bottom up. That way you're not dealing with only the top few inches of water being heated, but the rest being much cooler. Well pointed out information ePoolUser.

With kids, I don't think it really matters what temperature it is as long as their lips turn purplish/blue and it's shaking a million miles a minute! It's amazing what they can tolerate and what we REQUIRE!