The calcium does not disappear on its own. It is lowered through water dilution so rain overflow or significant backwashing. Evaporation and refill will only increase it by the CH that is in the fill water since calcium is not volatile.

As for sitting on the bottom of a plaster pool, calcium hypochlorite is not acidic (like Trichlor pucks) which would otherwise be detrimental to plaster, but the concentrated chlorine might be a problem for colored plaster. For white plaster, it's probably OK, but I'd try brushing it to mix it up after adding it or you can try pre-dissolving it in a large bucket first.