Expanding primer pockets is a common cause of brass cases wearing out.
However, it is heavy loading that gradually stretch them. I've loaded on the heavy side with a 270 W. and some would be discarded because of expanding pockets.
For years my favourite load with a 30-06 and 180 grain bullets, which I also used in competetive shooting, was 50 grains of Norma 203. When I later got a chronograph, I found the bullets were actually making the 2700 fps they were supposed to make. Norma reloading charts are extremely accurate on the velocities they give.
I used both Norma and Dominion brass and they went something like 20 reloadings, without the primer pockets expanding.
And yes, annealing will certainly extend the life the necks will go, without cracking.