I usually can resist the urge to put my two cents in, but since in this case, I have one of each I'll tell you what I think.
I have a single shot 338 lapua, Barnard action Robertson stock and 28" Lilja Barrel, with a nightforce 12-42 benchrest. My fifty is more or less the same gun, Barnard GP single shot action, 34 inch trueflight barrel in a robertson stock, also wears nightforce NXS.
I like them both, they are totally impractical and only toys, they are so heavy and large that they are useless for anything but prone shooting. That being said they are a pile of fun as long as you don't try to justify their existence as anything practical. I mean I have hunting rifles that could put meat on the table..not these guys.
There are downsides that apply to both
1)in order to take advantage of either one's potential, you really have to spend a pile of $ on optics.
1a)the guns themselves are horrendously expensive $5000 to $10000 and literally the sky is the limit. You can't get a rifle in fifty for less than $5000 I guess you can get into a M700 338 lapua in the $1500 range but most like hs precision are a $3500 touch.
2)you aren't welcome at the local range with either in all likelyhood so if you don't have access to some big stretches of Crown Land you may not be able to use them at all.
3)you have higher reloading costs, much higher on the fifty, in fact shooting factory ammo for the fifty is all but impossible due to american import restrictions. There are lots of ways of sourcing components through the EE but you have to accept that you'll be buying a new press and that a pound of powder doesn't go very far on fifty ammo, or .338 lapua for that matter. You need a new press, king size, kiss another $1000 goodbye. Primers for a fifty are a B*stard to find, I have about a thousand cause I'm scared to run out.
4)The guns are MUCH louder and have much more muzzle blast.
When I started on my fifty project I had no idea how much it was going to end up costing, I am well over $10 000 now for the gun and the stuff I needed to use it. I started with a phone call to Robertson composites and a deposit, by the time I realized I needed a new press, had to get a custom gun case, and needed a $2000 scope as well as a $2500 action etc etc it was pretty pricy. I waited a while and then a while more....but it was sure beautiful when it arrived.
Do you want a gun as much as a quad, or dirtbike etc etc? Unless you are a rich guy, that's what you are giving up..
For me the answer is YES, but you have to go in with your eyes open.
I only use mine for banging gongs way out there. and learning both rifles is something I am still doing and it is a fun pursuit. There is a lot to know and a lot of ways to miss that far out.
If I could only keep one it would be the fifty, there is something about a muzzle blast that pushes your chest in....You get a sunny summer afternoon and a few friends together with a quad running out to the target, the spectacular terminal ballistics which are an order of magnitude cooler than your 300 mag and you've got a pretty good afternoon.
Inevitably one of your redneck buddies shows up with a piece of plate steel, or a length of railway track, or an axle from a Cat and says do you think you could shoot through this??
good times!
hope that helps.