OP, what you propose is perfectly safe..........pulling bullets and changing to a different one has been going on for decades.........don't vary more than 5 gns + of nominal weight and of course you can go down in weight. What you will end up with for ballistics and accuracy is anybody's guess but there is no safety issue what-so-ever. I have seen this done with several cartridges like 303 Br, 8X57 and 30-06 ball ammo. Just pull the ball, replace with a soft of the same shape (more or less) and go sight 'er in. They shoot surprisingly well sometimes. I had one buddy that did this with 303 all the time and did not reload so couldn't size the cases. He used girlfriends nail polish to "glue" the new bullet in place. What can I say it worked just fine and he did a lot of shooting for next to nothing. Don't think he ever bought a box of factory softs..........ball ammo all the time and pulled and replaced for hunting.
If I'm not mistaken way back when, there was a small ammo company that did this for a living..........I'll be damned if I can remember the name of it, but they offered 303, 8mm, 30-06 and a couple others with "sporterized" legal hunting bullets. A lot of the Brit was Berden primed as I recall as was the 8mm likely, I never had a 8mm back then. I am pretty sure they were Canadian as they used CIL bullets as I recall, but they weren't local to Calgary. H4831 (Bruce) would likely know what I'm talking about and possibly Ganderite and Eagleye has heard of them as well.
Anyway OP, given that you are not using up military surplus ball ammo and have factory loaded softs for the Wby I see no gain or advantage in changing bullets. Pulling a 100 gn and changing to an 85 gn will not shoot the same as your 85 gn handloads. So you actually gain nothing and go to a bit of trouble to do it. I also would not be surprised if the 100 gn factory loads shoot as well as your handloads or even better. All the same applies to your '06.