Are you trying to make a specific power factor, or are you just shooting for the enjoyment?
I have not noticed a difference in feel or accuracy.
The chrono has noticed a difference in velocity, in the range of what 4n2t0 showed in his post. I found the small pistol primers were 10-20 fps slower when the powder charge, OAL, and bullet remained the same. To meet power factor you will probably end up adding 0.1 - 0.2 gr of powder when using small primers. I did not try anything like enlarging flash holes or whatever; I just tested large primers and small primers in the brass that I had on hand.