Hi,
Sorry, but if it's hitting high already, you don't need a file...you need a higher front sight. If you handload, go to a lighter bullet. As stated already, increasing the velocity will also bring POI down, but at the cost of added recoil. Going to a lighter bullet, and jacking the velocity up as needed should get you somewhere you can live with.
You have discovered that single actions are somewhat finicky about how you grip them. You are likely snugging your gun down onto the rest fairly well, which accounts for it shooting lower than when you go freehand. It will likely change again if you shoot it one handed, depending on how you grip it when you shoot two-handed. For longer range shooting, I've found that sitting on the ground, with my back leaning against something (a tree...), and resting my forearms inside my knees gives me pretty much the same POI as when I'm shooting standing, and steadies things nicely. YMMV.