I've been reloading since 1970, casting since about 1974,worked in the world's largest lead mine for a couple of years in the early 70's, and handled and shot god knows how many rounds with lead bullets over much of the latter part of that time. Never tested for abnormal blood levels during any of that time.
However, I also make a point of never putting my hands anywhere near my mouth while casting or handling bullets - a good practice taught in the mine. And I have spent a limited amount of time on indoor ranges, where airborne lead particles are present from both lead in primers and bullets hitting backstops.
My guess is you either never kept your hands away from your mouth or you spent a lot of your time working/shooting on an indoor range. Elevated lead levels until recently was a very, very common problem with staff and shooters who spent a lot of time on indoor ranges. Many of those ranges have been closed and new ones built, or the old ones heavily modified to provide ventilation that dramatically eliminates the amount of airborne lead on the range.
Won't do a lot of good unless properly fitted. Moreover, I'd pay more attention to the quality of the ventilation and cleanliness of that range. If the contamination is so high I have to wear a mask in hopes of avoiding elevated Pb blood levels... I think I'll pass.
Buddy was well outside his scope of practice if he started lecturing you about absorption of elemental lead through the skin. Elemental lead is not absorbed through the skin. To quote from OHSA:
Lead can be absorbed into your body by inhalation (breathing) and ingestion (eating). Lead (except for certain organic lead compounds not covered by the standard, such as tetraethyl lead) is not absorbed through your skin. When lead is scattered in the air as a dust, fume or mist it can be inhaled and absorbed through you lungs and upper respiratory tract. Inhalation of airborne lead is generally the most important source of occupational lead absorption.
However, if wearing vinyl or latex gloves stops people from sticking their hands in their mouth, eating sandwiches while reloading, etc, then that is probably indirectly effective in preventing them from getting leaded.