It's possible that your pump is overdriving your filter--what size pump do you have?

also--when you try vac'ing with a hose and bag, try putting a handful of cottonballs in the bag--sometimes that will help trap some of the smaller stuff.