something is dropping clay in my pool!!

We live in red clay country - if you scratch the ground around here you'll find clay. This year we are finding little blobs in the pool, with quite a bit accumulating between vacuumings. There are little spots of clay everywhere around the house (noticeable on the wooden decks, and the concrete drive and paths), so it is a widespread phenomenon and not something specifically pool related (like a raccoon washing his paws might be). So far our thoughts have been mud dauber wasps (but we haven't seen a whole lot of those; no more than previous years) or swallows transporting mud for nests (but they should have been finished that by now!) Anyone have any ideas on the culprit?

Alison