The fur and stuff should still be no reason to change sand. Sounds like a lot of annual, unnecessary work. My sand is on the 10th season and is the original. I do remove the valve assembly each Fall and look at the sand and have never seen anything unusual that would make me consider replacing it. Even removed the sand one time to do some work on the bottom end and just put it back. If you really feel you must then I'd consider just removing the top few inches and replacing just that much.
Al