Oil in the bore will cause some variation in the bullet's velocity and may also change the way it engages the rifling. Either of those conditions will affect its flight path and point of impact. As Double Gun said, after a thorough cleaning, it's best to check you zero after a fouling shot or two out of a cold bore, which is the best way to replicate a situation you'd encounter when needing it on a hunt.
If you have a blued steel rifle, your bore will still be blued steel, regardless of the finish applied to the exterior. You can paint a chromoly rifle with Tremclad or Krylon to make it more rustproof. The bore will still be as rust prone. Exterior rust is just a cosmetic eyesore. A severely rusted/pitted bore affects accuracy and makes the gun useless.
Another huge advantage to stainless is that it's not a finish, but a steel. Even if scratched, it will be equally rust resistant. No finish can compete with that, except perhaps stainless that's been Cerakoted.