I have RCBS, Lee and Redding shell holders for just about every cartridge I load for because of this problem.
For the specialty stuff, I make up my own to suit the brass on hand.
Your brass itself, may be on max specs. I find PRIVI brass is very prone to this, as is some of the newer Winchester brass.
Just cleaning up one side of a SHH will cause other irritating problems, especially if you're an accuracy freak. The increasing of the width of the lips needs to be done uniformly all the way around.
Another problem I've noticed with some brass/SHH combinations is that the base of the cartridge won't bottom out on the floor of the SHH. All of this, leads to crooked ammunition. Not a big deal with a worn bore but as I said, if you're an accuaracy freak, it makes a difference.