I remember seeing a thread from a while ago where people were discussing this problem with M14/M305 style rifles. It turns out that it was always the last round in the mag that caused a flier -- because all of the other shots had upwards pressure on the action (from the rounds left in the mag being pushed up by the follower), and the last round didn't have the same pressure, there was a noticeable difference in performance on that last shot.
If you're shooting this type of rifle, try replicating the problem without ever emptying your mag. If the problem doesn't re-appear, then you've found the culprit.