Poop in Pool

After looking at a post with pictures of droppings from a caterpillar in a pool from 2013 it seems this is what we have been having a problem with for the past two weeks. We have a lot of trees around our pool and some limbs do hang over the pool. I know cutting some limbs back should get rid of the problem to a degree., but any idea how to eliminate the caterpillars. From the amount of droppings collecting in the pool it has to be a large number. We have an inground pool and live in NC. We have gotten up in the middle of the night to try and see where this problem is coming from, but not had any success.
Thank you for any help