A properly sized soft lead ball/patch combo will show the weave of the patch material around the equator of the ball so that as the ball is propelled out of the barrel it will be made to spin by following the twist of the rifling. If the ball is a hard alloy such that the patch does not grip the ball then the ball, because of it's inertia, will not spin or spin less as it is propelled down the barrel and thus will not shoot as accurately as a properly paired ball and patch. Same effect as a patch/ball combo that doesn't fit tight enough or shooting a patched ball through a smoothbore barrel. Unlike a conical bullet that takes the rifling over the entire length of it's parallel sides, a round ball contacts the rifling only around it's narrow equator and will strip if the rifling twist is too fast: anything faster than about 1:50 whereas modern in-lines and some slug guns had 1:28 or so twist. Wheel weights and the like work as well as pure lead balls in smoothbores but not with barrels intended for patched balls, in my experience.
If you think that a lead ball has a lousy ballistic coefficient, I can only imagine how inefficient, and less accurate a glass marble would be...