I think I finally (after 6 weeks of stress) figured this thing out. It is not the water inflow that is causing the particles to get into the float assembly. Dirt is coming in through the overflow opening on the autofill when it rains heavily. Took a big storm while I was at the pool over a weekend for me to figure this out. My inflow line is clean, despite what the pool people have been telling me (makes sense as the line is now double filtered, from well to whole house filtration to special pool-centric metal trap and sediment trap. If rest of house gets clean water, never made sense for pool water to be bad, especially when garden hose (unfiltered water) seemed cleaner than the autofill generated water. All the metal in the pool must be coming from the soil bed that the autofill sits in. Key indicator was dirt in the pipes that feed from autofill to pool and spa.

Now if I could only get back the hundreds of dollars spent on metal trap etc...