My instinct would be to throw a set of calipers on those mags. It's hard for me to imagine a scenario in which a Colt lower has a significant variance in internal dimensions of the magwell...I assume they mill out a billion of them at a time, and if it slipped in the clamp the whole thing would be screwed.
IIRC the internal width of a TDP magwell is 0.898", +/- 0.015". But I have no idea whether that is pre- or post-anodize, now that I think about it.
I looked around a bit to see if I could find leaked documents on mag tolerances but I couldn't find any. Funny, seeing as the TDP for the M4 got leaked a few years ago.
At any rate my best guess is that you happen to have the first mags that came off the dies, and the last lower that came off the mill, and the tolerances just add up badly.