My father's philosophy was to always have enough ammunition that you can reload something your potential enemy may drop. Enough to fight your way to more ammunition. So, I found stocks of 5.56, .303, 7.62 x 39, x51, x54R, 30-06, .30 Carbine, 7.9 Kurz, 7.9 Mauser, and assorted odd balls like Japanese, French, Swiss, Italian and historic. He also had most of the common pistol cartridges, which coincidentally are SMG ammunition. Did he genuinely need lots of random ammo? No, but it made him feel better.
For an average gun owner, I'd suggest the Holy Triumvirate of .22LR, 12-ga, and something suitable for big game. That's not a clear answer, but a realistic one.