Sounds like you are on track.

Successfully vacuuming fine dust like that requires a well-functioning filter, a good vac head, and patience.

If you are having trouble with the dust blowing through the filter, adding DE to the filter via the skimmer, just prior to vacuuming, should solve that problem.

A roller vac will allow you to vac up the dust, without stirring it up the way a brush vac will do. The Rainbow (now, Pentair) ProVac has been the model we've always used ourselves, and have placed at the large commercial pools I have serviced. There are multiple sizes; do NOT get the larger models unless you have EXTREMELY strong suction and flow at your skimmer. We currently use the 22" model -- but we attach it to a dedicated 1 HP vac pump. Here's the model you want:
Pentair R201276 214 14" Pro Vac Series Vacuum Head @ Amazon
Regardless of vac head, when vacuuming fine dust you must move slowly. I've only rarely been able to get teen aged lifeguards to do that without standing over them, constantly saying, "Slow down, you are stirring up the dust. Easy. Focus on smooth slow movements." etc, ad nauseum. But, that's what it takes. And, if you have a good vacuum head, a well functioning filter, and vacuum correctly, you can do a complete pool surprisingly quickly. We can do a 45 x 75 pool that way in 2 hours, so even allowing for the smaller 14" head, you should be able to complete an entire residential pool in less than an hour!