What are teses tiny rust-like stains?

[poolbee]

I have been finding tiny specks of brown/rust color on my steps(plastic), in the skimmer boxes and around the white aluminum coping. What might be causing these? They are only as big as a grain of sand to start, are raised a bit and are very difficult to remove. I have been scraping the worst ones off with my fingernail and they don't come off easily. Ive tried the Mr. Clean Magic Eraser and Softscrub with minimal success.
I have a vinyl liner pool, SWG with no mineral additive. CAL is 210 and my copper tested at <.1 at the pool store.
Would anyone know what it is and a good way to remove it?