Well I think I might be getting some more clues. I have manually washed the filter a few times already so I don't think that is the main issue. As I was vacuuming to waste this morning, I noticed that there was barely any suction. It would come and go. When I turn the system to suck water entirely through the skimmer (which is where the vacuum hose goes in) there is literally no suction. If there is a blockage or issue in the line from skimmer to pump, this could be the air and the dirt going back into pool, etc.

Question: Can you snake a pool pipe or do a camera to see if there is something in the line which is causing any of this? I'm thinking this could be creating a problem with the DE getting to the filter properly when I recharge it.