.
Might I suggest you gas check a few, load them with different powders to test them, and actually FIRE them FIRST to see what they are doing. Develop a good load and you might be surprised. It seems today that many people are too enamoured with the theory of rifle shooting, and forget that time spent on the range is the real way to see what something is doing.
As suggested, use a bit harder alloy. Lyman #2 is 1 in 10 alloy and is the standard for rifles. LEE makes a hardness tester that you might find handy.
Lead cast bullets are something that takes a lot more development to find an accurate load. Primers, powder, BULLET LUBRICANT, and loading procedures all affect them, much more than jacketed bullets. They are not something that you can just cast and shoot and expect the same results as jacketed bullets.
Also, people tend to try to drive them too fast. If you keep them under 2000 fps you will be all right, but over that requires in most cases, special techniques, lubricants or bullet designs. Paper jacketed, (paper patched) bullets can be driven faster, and even as fast as a lot of factory rounds. People also expect bullet design to give the same results as metal jacketed bullets. For example, I have a H&G .314 diameter mould that is the same shape as the 172 grain 30-06 military bullet. Because of the pointed spritzer shape, people think they can drive this bullet up into the 2600-2700 fps range, and then wonder why they end up spreading them all over the target.
This bullet was designed as an indoor gallery and target load. It works well up to 300 yards IF YOU LOAD IT PROPERLY. Drive it at 1600-1700 fps and it gives amazing groups, better than .303 Mark VII ball out to 300 yards, and will give a good account of itself even out to 600 yards. It was my target loading for years -- all you had to do for longer ranges was to test fire it so that you could find out the appropriate sight yardage settings to set on your sights. (300 yard ranges were in the 700 yard sight setting ranges.) You are still shooting at 300 yards, the sight picture is relatively the same, but it was a LOT cheaper than buying factory ammo.
Try the old C.E. Harris cast bullet loading for Military rifle calibres and original bullet weights. It is 13 grains (thirteen grains) of Red Dot powder. You might have to vary it a half grain plus or minus but it seems to work well with most military calibres of the medium (.308/7.62) or larger (30-06) sizes. Do NOT USE IT in the smaller cases such as the 7.62x39 or 5.56 NATO.