Sounds like just everyday dirt, pollen, dust, etc that settles in all pools. If it feels slimy, or is growing on the walls. then you need to look at possible algae growth, but if it's just settling on the bottom, usually in the same places, then it's more likely just airborne "stuff" that lands in the pool. I have a couple of handfuls of the same brownish stuff that settles in the same places in my pool floor, too--nothing you can really do except keep it stirred up so hopefully the filter will get it, or vacuum it to waste.