Personally, I've had bad luck with Federals - well bad luck with the premiums anyhow. I'd test out a Remington and a Winchester load too, and different bullet weights. For example I have a .30-06 that really only seems to like 165 grains.
I find that my 'bench shooting', i.e. with the back of the stock supproted with a sandbag is better than my prone with a bipod shooting. It is likely you are a better shot than I am, but if it'd been me shooting those 2.5 inch groups I'd bet the gun and load combination is good for 1.5 inch, which a lot of people would be happy with for a magnum hunting rig.
RG
<><