Arg-h!

My bad. You said "IN your pool" . . . I read "AROUND your pool".

Rough plaster is usually caused by a combo of low pH and poor finishing. If it was ALL rough, low pH would be the likely primary cause. Rough in spots points to poor finishing as primary. A rubbing brick can help. I'd recommend staying away from the Chinese 'in-stock' bricks going by brands like "Anvil" and "Bon". Instead, go with US pro brands Marshalltown or Goldblatt: the difference is only $3 or so. Keep in mind that these are VERY brittle, and WILL break if you drop them onto a hard surface -- US-made or not!