I have a 32 cal Dixie Poor Boy (I think that is what it is) and it is fairly fussy about what charge it is accurate with. 25 - 30 grains of 3F seems to give the best group and with amounts less than that the accuracy drops off quite markedly. At 36 cal, I doubt that you have a recessed breach but with very small charges, the ball might block the flash channel to the powder. Try using 10 grains and bench the gun to see what your group size is, as well as work your way up in charge and compare group size
cheers mooncoon