This is the target in question.
Is the upper, left round a flier or not?
Load is 42.0 IMR 4895, CCI primer, 168 gr Speer BTHP match bullet, IVI brass. C.O.A.L. is 2.820"
I'd say that if you heated up your barrel by firing 5 quick rounds into the back stop, and then proceeded to fire five, 5 round groups in a row, loading fairly quickly and firing the battle rifle as quick as you can shoot accurately, as it would be fired in battle, I'm thinking your groups will fill out the mean radius with your flyer. It probably won't get any bigger than your flyers mean radius(the flyer I'd call the exrteme side of the spread, but I'm thinking it would be considered the rifles practical accuracy potential.
A lot of rifles will put 2 rounds very close together, one that comes to mind to me is a sporting rifle the BLR61 in .308Win, two rounds very close and then the third round strays, not a flyer like we like to call it, but the actual mean radius if we continue to fire the rifle at a controlled rate of 5 or more rounds.
These so called flyers show up in my mind in rifles that have mechanical shifts during heating and usually won't go much past on continued firing at a controlled accurate rate. If you want to see 5 rounds in a tight group everytime with your decent hand loads, then you will need a medium to heavy barrel, free floated and the action properly bedded. Here's where your frustration will end.
Even two round groups are okay in my mind in sporting rifles because really the first round from a cold clean barrel is the most important and is what you need to recheck a few times prior to your hunt, and in the field may be followed by a still controlled and accurate second round,...and if it aint down by then its probably on the run and you're starting to throw them down range now on a running target. So its more of your instinctive shooting than MOA at this point.
Take the M-14 for what it is and check out the US Army's accuracy requirements for it using ball NATO ammo, unless you have it set up as a NM rifle by an experienced M14 smith. I think if the rate of fire was slow enough to not have it tip over that mechanical shift that causes the first flyer, it would stay in that nicer group range you believe the rifle is capable of shooting. Your handloads will be better for sure than 762NATO ball especially Canadian 1970's IVI flavour, but it still can never be a HB M70 Winchester or M700 or any other bedded and free floated rifle.
For another example of the mechanical trip I call it, my Remington 7400 in .30/06 is decently accurate. I just a week ago rezeroed after moving my scope rearward and after a few rounds to tune at 200 yards, I fired a 3 shot 2" group of the bench rest from a dirty but ambient barrel. I shot 2 clays on the back stop at around 215 yards. So the old auto is go to go as a deer rifle.
The group however if you look closely, will always be 2 rounds into an inch at 200 and then the flyer lol making it 2" or sometimes 3" . If I fired a 5 shot group it would fill up the 2" to 3" group and open slowly more from the heating and mechanical shifting from the gas system and fore end.