There is no difference between model with the exception of the material used (stainless versus chromoly steel) The "milspec" has a nicer stock and bedding system, and the barrel has 5 grooves instead of 4 or 6.
They all shoot about the same - serioulsy. The actions are exactly the same, and the barrels are of exactly the same quality from the cheapest SPS to the most expensive 700.
What most fail to appreciate (and this question is asked alot) is that hunting rifles and target rifles have diametrically different requirements...
Target gun:
Long heavy contoured barrel, tight twisted to use long high-BC bullets
Barrel contour to resist high heat from sequential firing.
Stock that is ergonomically suited for the position from which it is fired
They are heavy!
They have high-quality scopes with very high magnification
and special reticles
Hunting Rifle
Light
narrow contour, short barrel as few shots are taken consecutively no need to worry about heat
stock designed for low weight and off-hand firing
low power scope with hunting-type reticle
So... you will have to balance between these requirements. a 700P or a 5R milspec would be good platforms as their stocks are decent. They are heavy to haul in the bush. The SPS Varmint is also ok, but the stock is so-so quality.
The 700 is the safest and strongest action out there, and there is a reson it is used as the basis for almost every custom action out there. They a well-made and safe action.
There simply is no one-size-fits all rifle. Hunting and target shooting requires compromises.
