Don't give up on the .50 yet. Try some rust solvent product, there are lots of them out there. Let it soak, then, as you say, get at it with brushes, steel wool, etc. If need be, you can try lapping it. The rusty spot is just in front of the ball, when loaded, and may prove to be more of a pain than a real problem. If the rest of the bore is ok, it may still shoot not too bad, but you will have to be real careful that you get the ball seated right down on the powder every time. The rough spot will make it tough to get the ball seated correctly, especially if the bore is dirty after a few shots. If that is a problem, you can help yourself out by wiping between shots. One technique is to load your powder charge, then run a tight wad down on top of it. Then you wipe with a damp patch, followed by a dry one, then load the patched ball. OTOH, it may be screwed. Only one way to find out...
The CVA .32 is a fun little rifle, and not bad quality, IMHO. IIRC, you mentioned in another post that you are set up to cast, so you are good to go. I would buy a proper mold, rather than try to use buckshot. The buck will likely work, but I think, is almost exactly .32, so you would need to use such a thin patch that you would likely have trouble with burning through the patch. If you get a .31 or .315 mold, then Bob's your uncle. Have fun. I bet you'll have another ML before too long :-D
Check out this site. http://www.blackpowderrifleaccuracy.com/ Dutch has a lot of good info that will get you shooting pretty well way quicker than you can figure it out on your own. Think about sending him $20 for his "book" ( actually it is a home printed pamphlet, but it is really, really helpful). As always, YMMV
Glen