I must be misinterpreting the OP's question, but I thought he was referring to cartridge collecting, not rifles in particular chamberings.
Be that as it may, when I first started collecting cartridges 40+ years ago, I wanted everything, every different cartridge, every headstamp, every different bullet in each load, then different primers, &c., &c., &c. Fast forward a number of years & I finally came to the conclusion that that just wasn't going to happen in my lifetime.
I first specialized in Canadian, British and European cartridges, then solely British. After a time, they got darned expensive & harder to find, especially as the older folks' collections got scattered and fewer and fewer people were collecting.
Probably the easiest [& cheapest] would be to start with current cartridges [hint, follow C.O.T.W. divisions] & go from there. The cartridges are relatively easy to locate & much cheaper than the older ones and if you find that your interest wanes, you're not out an awful lot of money that cannot easily be recouped.
I might (& quite likely am) off the OP's intended topic, but that's my $0.02Cdn.