You could try JB Bore cleaner. It is a mild abrasive, but won't hurt your bore or choke. Smear some on your bore brush, then work the brush back and forth advancing an inch or so at a time until you reach the muzzle. Yes, it is hard on the brush, and you should use the best quality brush you can find, I like the ones made by "Plenty of Patches". It could take a few applications, but it should cleanup the rust. Patch the barrel out with regular solvent to remove the paste, then oil and patch dry when the problem is resolved.
If the rust has penetrated into the barrel wall, the gun might have to go out to a smith who can polish it with a hone, similar to a cylinder hone.
Edited to add . . .
I reread your post and I misunderstood - I was thinking the rust was on the inside of the barrel. Try some very fine steel wool, that should do it.