if you have to scrub a bit to remove teh "silt" then I highly doubt it's settled silt - please read:
http://www.poolforum.com/pf2/showthread.php?t=6954
it should explain what you should do.