I wrapped the slide in painters tape and laid it across the open jaws of a vice. Hit the punch so hard that I saw sparks come off the pin. It didn't budge. So, I said to hell with it and drilled and tapped the hole without removing the firing pin. Touched up the now shiny end of the pin with Birchwood Casey Super Blue, and called it a day.
I had mounted a dovetail mounted optics mount, but no matter what I did it wouldn't say in position after 50 shots or so, the two 3mm screws that secure it to the dovetail piece would eventually loosen up. So I drilled a third hole at the front of the mount, counter sunk it and drilled and tapped the slide for a 6-48 screw. Hopefully that will secure it. Put blue Loctite on all screws, and will take it to the range tonight and give it a workout.
If I ever had to remove that pin, I will try using an arbor press to press it out.