I think it has a lot to do with the specific material of the springs, would not surprise me if cheap springs can get damaged by prolonged compression.
However, I have heard anecdotes of a guy loading some mags during WWII and his son shooting them a few years ago and they functioned normally.
I have noticed among my own magazines a certain change in pressure between the ones I have left loaded and the ones I left empty, but it seems to be a minor change that has since stopped, mostly it just means I'm not tearing my thumb quite as raw on the mags that have been left loaded. But that might be all in my head.
If there is truth to it, you'd probably be looking at years or decades. A bigger issue would be if you are constantly loading and unloading a mag, such as when practicing drills.