I think Canuck223 and Freedom nailed your method for finding the cause. When it's a very light drag and the magazines are stainless you may not get a drag mark. Removing the stuff that could cause the drag and then replacing them one item at a time will let you zero in on the part that is the issue. It's not that you're looking for a fault in the gun itself, it's just a way to get all the obstructions out of the way so you can see and focus on where the interference is coming from.
Easy hit on an excellent possible cause as well. If the stop is catching on the lip it'll hold the mags but you won't see any external scuffing. Removing the slide and barrel and just checking with only the slide stop lever in place would be a good method for checking this aspect. If it's catching then a little tweak on the stop rest may save the day.