Don't be put off shooting that gun! It's what is was designed for and it is a tremendous thrill. Generally, it's not shooting that harms old Colts but lack of care and maintenance, especially cleaning. But before you shoot, all the usual caveats: make sure the gun is sound enough to shoot and that you know it in and out (take it completely to pieces, replace suspect springs etc with the modern repro parts widely available for Colts), make sure you know a lot about BP shooting generally, and be prepared to spend an hour or more cleaning each time, etc. There's lots to learn on specialist forum sites such as The High Road.
I shoot a Colt 1849 (1862 serial no), and every time I do it looks better for it. I shoot 315 ball (but prob would do 320 if I had some) over a wonderwad and 12-15 gr Goex 3F, with a No 10 cap. I clean using warm water and then liberally cover everything with liquid wonderlube.
Some common issues I've had with shooting antique colts:
- Worn/pitted bore. Solution: just enjoy the huge thrill of shooting a 150 yr old gun and forget about accuracy.
- Worn nipples, causing loose fit of caps. Solution: pull nipples and replace with modern reproductions if you can; if not, live with it, as I do, pinching caps to fit and not using the worst nipples (useful to have one empty chamber anyway for resting the #### as safety).
- Enlarged nipple hole, causing added blowback, forcing blown cap into gap below hammer. Again, you can live with this. Tensing the hammer spring by wedging a small shard of metal under it while screwing it in can give the hammer more resistance, but you get used to checking where those blown caps have gone between shots.
Congrats on the gun and hope you enjoy it!