I have found the same thing. I have shot my dad's S&W 4006 a lot as well as our friends G22. Having shot a G17, and a BHP just recently, I found them a lot snappier than the other guns in .40S&W and it seemed to take longer to realign the sights quickly after each shot due to the jerky muzzle flip. My accuracy was a lot better with the .40's as well (same with .45). I couldn't hit jack with the G17. I think I handle the 'recoil pattern' as someone called it, of the .40 better than the 9mm. I guess I find the .40 right in between the harsh snap of 9mm, and the gentle push of .45.
Anyone else find this? Maybe I need to shoot more 9mm....