That pin was there to placate the powers that be and to discourage those who would attempt to add a safety sear or DIAS to the Colt, it is of little concern to most CDN owners. Your gun has to following exclusive features.
Two screw takedown/front axis pin
Large hammer and trigger pins
Sear block
Despite these features you have an original Colt, don't mess the gun, RRA makes a decent two stage trigger. Leave the gun alone and shoot the crap out of it. Buy another rifle for modification.