They are vastly different in hand, especially if using the small back strap on the M&P. I wasn't able to deal with Gen 3 Glock, but Gen 4 no backstaps worked well for me. I have an SR9 and don't think the Ruger American or 9E were improvements at all. SR9 is a great pistol.
They're virtually identical in size (girth, width, lop etc.). If size is a concern and the OP wanted to stay with a double stack polymer the Ruger 9E/SR9 would be the best choice. The Ruger 9E is a great handgun. I just bought my second 9E from IRG's for $540 CAD all in. When in comes to price/value the 9E cannot be beat IMO.