You're going to get enough advice to keep you shoveling for a week.
As far as I'm concerned, the dominating factor in felt recoil is
stock fit....and about 97% of shooters are using stocks that don't fit.
The second factor is
weight. The greater the mass, the less the felt recoil.
It's that simple - a lesser caliber, in a light rifle that doesn't fit you will hurt like hell.
A big bore, in a stock tailored to fit
you, won't be bad at all, and the heavier it is the less you'll feel it.
I'll illustrate this with a little story. I once had an M-65 Tikka in .338 Win.
Mag. Now, I'm not exactly a little guy, but that thing made my eyes go fuzzy every time it went off....I was down right scared of it. A friend of mine came a long, and wanted to buy it for his wife! She was about 5'4", and weighed maybe 100 pounds soaking wet. I told him he was nuts, but after lots of cajoling (it was a very pretty rifle, after all!) I decided that if she tried it...if she could put three rounds through it without physical harm... that I would sell it.
She did...and kept right on shooting it! It fit her, and she loved it, and she shoots it to this day. I still cringe every time I see it.
At the same time, I have a Sako L61R in .338 that has been chopped and channeled down to less than 7 pounds. It's my peep sighted, never-out-of-my-reach truck gun. Everybody else looks at it like it's the reincarnation of a Govt. mule, but I can shoot under an inch with it all day, off the bench, prone, any which way at all....simply because it fits.