Removing loose plaster is ESSENTIAL, regardless of how you re-finish.

I understand why you might not want a rough surface - it also tends to make algae more of a problem. But why have you ruled out simply replacing your old plaster ('Marcite' = marble dust + white cement) with new plaster? Properly applied, that is a very desirable surface.

Diving boards are a risk . . . but mostly if you let teenage boys (or drunk adults) use them unsupervised. Most of the broken neck / back injuries have come from a larger athletic male who was showing off. Reportedly the guy who became a paraplegic, and whose lawsuit bankrupted the NSPI (National Pool and Spa Institute) preceded his final dive with the words, "Watch me do a suicide dive!".

In my opinion, diving boards are usually safer than slides, but when diving board injuries do occur, they can be catastrophic.

You know know your pool, and your swimmers.
+ When you dive, do you have to be very careful to avoid hitting the bottom? If so, take it out.
+ Do you have unruly teens or adults using your board? Again, if so, take it out.
+ Otherwise, if it were me, I'd probably keep it.