I have NEVER brushed my walls, actually, I just noticed the other day that my brush is in desperate need of a garbage can from hanging in the sun most times.......
The only cleaning of walls I've ever done is the scum line that appears every spring - I really don't think brushing a necessary component of pool maintenance - maybe if you got an algae outbreak, but even then, when I did have that years ago, I still didn't bother to brush and got the pool cleared up in no time flat.