If it's not spreading, and if it's settling in the same crevices, then you've got dirt, just like mine. There is an amazing amount of crud blowing around in the air around here, and it really becomes evident when you look at your pool. Might want to check and make sure that you don't have sand blowing from your filter when you turn the pump on )you'd see it from the returns as soon as you power it on). Vaccuuming to waste is probably the quickest and best way to get it all out. Alternately, you can brush the pool daily to keep it stirred up so that the filter gets it.

Janet