Go in and look at the bags. 99% chance that they'll tell you it's "filter sand", no matter whether it is, or not.

Is your water clear enough for you to see whether the DE came back out into the pool?

Also, you said the sock was dark brown. Was that dark like chocolate or coffee, or dark like a rusted tin can? Dark like coffee + well water points to manganese. Do you have a treatment system on your house water, especially a "greensand" filter?