What's so magical about a punch set? And let me guess, you've never used that particular set from PA. How many hundred roll pins do you remove a week?
I have a couple different sets from PA, steel and brass, and they work just fine. And, they aren't supposed to be made of super hard steel. They should be softer than the material of the hole to prevent damage. That's also the purpose of brass punches. The set includes have is fairly hard/tough steel, and the shafts haven't wanted to bend.
Look at the US made ones from Brownells, they have a pin projecting from the centre of the tip to centre the punch on the pin.