I've had barrel mirage hurt my shooting, especially when testing ammo at 100y (and don't forget, barrel mirage is quite rare for outdoors shooting, it can only happen in nearly-dead-calm winds).
I've never had mirage from the range hurt my shooting. I've shot in fairly thick and muddy looking mirage, with the target looking distorted and wobbly, but I've never been able to attribute a poor score to mirage.
The experiences I talk about are fullbore shooting from 300y-1000y, with 20X and 25X scopes. In fullbore, your shot position is indicated by a large spotting disc, so it is not necessary to be able to resolve bullet holes (in fact, in fullbore it is impossible to see bullet holes under ordinary conditions, even at 300y with the very best optics)
Disclaimer: I'm not a BR shooter, but I do know that they can be quite unhappy with heavy mirage conditions, cycbb486's post above is pretty representative of what you'll hear a BR shooter say. In BR, they use smaller bullets (typically 6mm), shoot on light-coloured targets (mostly white, with thin black scoring lines), and rely on being able to see of shot hole in order to decide on how they're going to fire the next shot. They also shoot rifles that are more accurate than fullbore rifles, and usually at shorter ranges (so that the wind changes they are doping, while just as complex and difficult as a fullbore shooter's task, are much smaller and more subtle - so there are sight picture things, and wind things, that a BR shooter sees, that no other shooters ever have to deal with).
