As Dennis pointed out, you don't lose a lot of weight. Maybe 2 oz on a long action. I've not seen any evidence that it is detrimental to accuracy but it stands to reason it could be.
I've probably done a dozen or so but, in every case, there were enough other alterations ( new barrel or new stock or both), there was no longer any basis for comparison.
Like most weight loss techniques, the purpose is as much to LOOK light as it is to be light! Regards, Bill