The guy at work doesn't have the whole story, so he's wrong. As Dogleg suggested, the thing that guy thinks he is talking about is the exit pupil. This is determined not just by the size of the objective lens, but also the magnification. Divide the objective lens size by the magnification to discover the exit pupil. If the exit pupil is larger than the maximum dilation of your pupil, there is some wasted light. With variable power scopes, some settings probably have a larger exit pupil than you can fully use, and some will be smaller, actually restricting the light. But that doesn't mean they are no good on those settings.
As migrant hunter said, the typical maximum dilation of the pupil for most people is about 7mm, so an exit pupil larger than that is giving you more light than you can use. That means it's silly to put a 50mm objective lens on a 4x scope because you are carrying weight and bulk for no good purpose, but if that scope is variable power, that big bell is giving you light you can use on the higher settings, so then it does make sense.
However, as we get older, our maximum pupil dilation usually gets smaller, part of the general aging process that means most of us will eventually need glassses, if we live long enough. Typical maximum pupil dilation in your 40s is about 5mm, so if your optics produce an exit pupil bigger than that is wasted on you. But you can still use all the light your 50mm lens can transmit at 10x, so if you feel a 50mm or 72mm lens isn't too much trouble to carry, go ahead.
(Remember also that better quality glass and lens coatings can enhance light transmission, so you may get a scope that is more than bright enough for your use without going to the biggest size of objective lens.)
And if you are trying to take a shot at last light and the sight picture is a little too dark to feel sure, try turning the magnification down to the lowest you think you can use, and you may find there is enough light after all.