If you can, shut the main power off to the breaker panel when you work on it. If you just turn the breaker off (that your changing) it is still safe to remove the wire and replace breaker but you would be better off killing power to the whole panel as the adjacent (breakers) wires will be energized. Make sure you buy the same brand (i.e. ITE Siemens, Cutler Hammer, GE, Square D, etc) since the panels are designed for specifc brand breakers (its also required by code not to mix brands).