You could try to peen the burr back in place. Not sure what the issue would be with removing the burr? The material is already displaced?
R.
Yeah, this, IMO.
In a case like yours where the access is really easy, use the shank of a drill bit that is a close fit. IIRC, 7/32, or a Number 1 or 2 as fits, a large -ish flat nose punch, and a hammer.
Secure the barrel so you have two hands to work with, insert the drill shank and tap it sideways with the punch and hammer as required to start moving metal around.
You will only accomplish so much, but you can make it better than it is pretty easily.
This is exactly what a chamber iron does, it moves the metal back in to the space it came from.
Except the chamber iron is a tool that allows fairly easy access without having to remove barrels from actions and that sort of stuff.
Finish off with a fine file, some 320 or finer wet or dry wrapped around a stick, a reamer, if you have one, or whatever your comfort level allows.
If you have to be told not to hit it like you are pounding nails, maybe get it to someone that understands why not.
Cheers
Trev