Thinking some more about how I messed this up - You will want to consider how you are going to keep the hand held tool going dead straight back and forth as you polish - if you make the hand tool with small contact area for the emory, is very easy to make "waves" in the race-way - that is not helping. Some believe that cycling a bolt straight back and straight forward is difficult - without user induced wobble up and down, especially at the rear end - is actually a "piece of cake" compared to setting up and polishing a raceway dead straight. Need to plan for similar cutting / polishing pressure, all the way - same at start as at the finish of the stroke. Can do it "in a rush" like I did, and that is basically second or third class results from that.
I think if I was to try again, I would do like trying to scrape out a barrel channel - use a transfer marker, soot, or whatever, and try to find the "binding spots" and just work to polish or scrape there. As mentioned, smooth or "rough" feel of a bolt in it's raceway can be partially on the user's technique. Can also be really poor quality machining, poor heat treat, and so on. Also to be expected that the "rough" feel might not be within the raceway at all - could be in the bolt shroud contact, on edge of an external extractor, or other place - so I would think the "test" would be to start with a completely stripped bolt body first. Then add back one piece at a time to find where is the drag actually occurring?