Is your idea of "signs of use" a gooey tremclad mess? Because that's what you end up with after oil and solvent come in contact with conventional paint. Been there and done that. If you never clean with solvents or use oil or grease I'm sure you'll be fine.......
Strangely enough, I have been known to properly clean and lubricate my firearms. My clear-coated, Krylon'd N14 is doing quite nicely, thank you.