Could it be that the slight bit of dirt is just in suspension while pool/spa is in use and then when all quiets down again, it resettles back to the floor?