Could be. Post specs about your pump and filter and let one of equipment gurus see what they think. 20 may be normal for your filter's clean pressure and if so, someone please correct me. But, that seems high for a clean filter pressure. My clean filter pressure is more like 10 or 12. With a sand filter, a pressure increase of 5-10 is indicative of the need to backwash. So, the fact that a 5psi has a diminished flow isn't farfetched.
You say that it happens after you run the Polaris for any extended period of time. You should see a pressure increase when you vacuum, especially if you are vacuuming up much debris. That would be a sign that the filter is doing its job --- removing debris from the water.
Bookmarks