I have the same problem, but I know my problems come from the street behind our house. It's a low area where a creek runs just behind our yard and then crosses under street. All the drainage from rain/wind puts dirt/sand in that low spot, then cars fly by @ 30 to 40 mph and that throws it up in the air where it comes down in the pool. I know because I put a white filter sand in the pool and this is dirt brown. Next house will have IG pool w/o road behind it!
I bet yours is blowing into pool also, since the DE should filter it.