You'll make a mess, trying to punch that much material out of the way of the punch.
Easier, and more effective, to set up a drilling jig that will hold the bullet and align and guide the drill bit, if you really feel the need to hollow point a cast slug. Drill a hole big enough to allow the displaced material room to move back into the hole, when the punch is driven in.
Make a few, then measure them and try them out. No use ruining all your bullets, only to find out after that they are not fit to shoot.
The next step up is a swaging die. More money than you want to spend, I'm betting. Got a lathe? You can make the stuff. Gotta be able to hold some pretty close tolerances.
Cheers
Trev