Noooo no no no. Small pistol primer cups are not built for the higher pressures that rifle rounds generate. One time I didn't have any Remington 7 1/2 small rifle on hand and tried the 6 1/2 small for some load work with my .223. With low end and mid range loads not even close to max, I was cratering, and even pierced two primers. The cups on the 6 1/2's are not as tough, as the 7 1/2's. So I think the small pistol primers would be not only inviting disaster into your house, but letting him (or her) raid the fridge, and run up your phone bill.
Pistol primers also have a shorter height by about .006" so ignition mightn't be reliable. In black powder .32-20 loads I would give it a whirl. But if worse comes to worse, try to trade somebody primers.
Oh cool, I didn't know the exact working pressures. Thanks brnolvr.