The story is usually told as follows:
Early in the US official involvement in Vietnam, say 1965, the first XM16E1 rifles were being handed out to ARVN troops. They were also deployed with some first line US forces, namely 173rd airborne and several different special forces groups. Most ARVN units were carrying wood-and-steel traditional rifles, mostly the M1 Garands and Carbines.
In this time frame, a commissioned officer with the NVA was captured or killed. Several of his papers were captured and sent off for translation. One of the translated papers was a general order, being broadcast to all regular infantry in the NVA reading something to the effect, "do not engage soldiers armed with the Black Rifle, it makes terrible wounds and causes great suffering".
The Americans who captured this message were quite tickled to know the enemy was so scared of their rifles, and the name has stuck, certainly for the M16 family, and by association, many other modern mouse guns.
The term "green rifle", I think comes about because some people who don't know the story behind the term "black rifle" take it very literally, and think, "hmm, my rifle isn't black, it's green", and modify the phrase to match their own gun.