I'm going to see if I can annoy as wide a segment as possible with my answer...
I have a number of braked rifles, and a number without, in a wide variety of power ranges.
When it comes to recoil, brakes do make a difference, but it isn't an important difference to me. Dealing with recoil is more about stance and shooting posture. Get these right, and anyone can shoot almost any rifle (within limits). Get it wrong, and even relatively light recoiling cartridges can wear you down, especially after continuous and repeated shooting.
For my mind, the big advantage of a brake is in keeping the muzzle flat when shooting - preventing the natural tendency for muzzle rise and lateral swing (to the right for right handed shooters, left for left handed shooters). This makes it far easier to keep the scope on target to verify your shot, and also allows for a quicker follow up shot if necessary - especially with semi-autos. (Yes, yes, I know - 1 shot, clean kill, every time, right? Except sometimes not, and anyone can flinch.)
The downside is noise, especially on shorter rifles. And it isn't just noise, but the concussion wave - which can be jarring and uncomfortable.
So, for me, it works out to a blend.
Bolt action rifles, especially with open sights, I'm not going to brake. The benefits don't outweigh the downside. I'm going to need to re-acquire while cycling the action anyway. And even with a scope, I'm opening my weak eye as I fire and will be able to see the effect of the shot outside the scope.
For high volume shooting (think varminting), or cases where a follow up is more likely and/or a case where a follow up is potentially a safety issue (hog hunting, or other dangerous game), I'm going to lean towards a semi auto with a brake.
When it comes to shooting from a bench... Courtesy prevails. I'm fortunate enough that out club seldom has all the benches full, so simply moving an extra bench over makes it tolerable enough for the other people on the line. At the same time, if someone is blasting away with a braked rifle on the bench beside me, *shrug*, oh well. I have good ear pro and it doesn't bother me much. I also have some shorter barrelled "handy" rifles without brakes, that realistically are just as nasty, in terms of noise and concussion, as my rifles with brakes, and yet those don't seem to draw the complaints out.