Read the page http://pool9.net/alk-step/

That page explains how to do -- on purpose -- what you are currently doing accidentally.

Bottom line is, it's not possible to maintain a standard pH level AND standard carbonate alkalinity levels.