No disadvantage. No performance changes. In fact, some guys use the same load regardless of large or small primers.
As noted, don't get them mixed. There was a recent story on here about a guy getting primer detonation by trying to cram a large primer into a small primer hole.