When I got my Savage (in .30-06), I took it out to the range with a couple boxes of 150 grain. Sat down, got my bench set up, loaded the rifle, got on target, and took a shot...
And I was amazed by the recoil. I'd fired an SKS and a couple other firearms prior, but this was my first gun, and the first one with such a big cartridge. The first shot was ok, but it hurt... the second shot was worse, as I started expecting the recoil. It just got worse over the course of 30 rounds. Why I subjected myself to that, I don't know. Before I took it out to the range again, I got a Limbsaver and put that on. I was a bit sketchy at their claims of "50% reduced recoil!", so my first shot with it on I was anticipating the pain... and it never came. It was like being whacked in the shoulder with a pillow, rather than several pounds of wood and steel reacting to a projectile with a muzzle velocity of 3000 fps.
Right now, I've got no issue with flinching due to pain, although noise is another issue. I use the Limbsaver and shoot 150 grain bullets... I find that I can shoot 150's all day without an issue, but after my shoulder starts hurting after just a few 180's. As has been mentioned by other people, the weight of the stock makes a huge difference... I've got a wooden stock on mine, so it's pretty heavy. I was offered a couple shots on someone else's .30-06 with a synthetic stock, and I was surprised at how much it hurt. The action was fantastic, and the gun felt really good, but after shooting one round (even with a recoil pad) I couldn't do any more... I was using my own ammunition too.
Try using 150's, or even 125 grain reduce recoil loads. Get a good recoil pad. If you've got a synthetic stock, try out a rifle with a wooden stock, or vis versa. If you've got a synthetic stock, and find you prefer the weight of a wooden one, then trade stocks with someone or fill yours with lead shot. Most importantly, I would suggest you get a proper feel for your rifle before you start worrying about your groupings. As you get used to it, your shooting will get better, and the more you like your rifle, the more you'll practice. If worse comes to worst, either buy another smaller caliber, or try to find someone to trade with you.