Lefty, I hear what you're saying and I have sympathy.
On the other hand, I've loaded for a couple of 7s over the years. (In fact, that's how I learned about reloading).
My first rifle was an old Sako. With 61.5 grains of IMR 4831 and a Hornady 162 BTSP, I was breaking about 2,850, but accuracy was excellent. Before the barrel finally gave up the ghost, I was getting 2,850 to 2,900 with 175 Hornadys pushed by 64 grains of H4831SC.
I also loaded for a buddy's Sako. I was shocked to find that 160 Partitions pushed by the same 61.5 grains of IMR 4831 was clocking 3,050 out of his rifle. This just goes to show how wide the velocities can be with different rifles in this cartridge. (Incidentally, this has been the magic combination for accuracy in my experience in a 7 Mag. If this load (with a decent 160-grain bullet) doesn't print small groups, then something is wrong with the rifle.)
My question for you is what is your seating depth? In my experience, Remington 700s have looooong throats, and bullets need to be seated way, way out if you hope to reach the classic 0.020" off the lands. Seating depth can make a significant difference in velocity and accuracy, and that might be a factor in your case.
However, your velocities (with the stated charges) seem very low, by any standard. If I were you, I'd tinker a bit more, but after a while it's just not worth it anymore.
The truth is that even with the velocities you're getting, the rifle will work on any game animal -- but as you've recognized, what's the point? You already have another proven rifle that performs at this level.
If you want to re-barrel the Remington, I'd still suggest a 7mm Remington Magnum. I wouldn't let this experience sour you on the cartridge. It just plain works, and with a bit of careful loading, you should be able to safely reach 2,950 to 3,000 with a 160-grain bullet.
At the end of the day, though, it's all about what turns you on and gives you confidence. If you think your rifle is anything less than the Angel of Death, for whatever reason, it will not work for you in a hunting situation. That's why these days I'm enamoured with the .300 Winchester Magnum and that's where I plan to stay for a while (as far as "magnums" go, that is).