First off, you have to be aware that military units don't just look up the H&K website and select a rifle for service and put in a purchase order. Military units consult with H&K in order to build a weapon specifically for the user requirements of the said unit.
The HK416 was designed to provide a very specialized unit with an enhanced capability that incorporates the familiar control features of the Direct Gas Impingement 'legacy system'. This wasn't something that HK just dreamed up and decided to build.
The engineering that HK Germany have put into the 416 is quite amazing. That engineering doesn't come cheap either. HK have been building piston guns for a long time and is part of the reason why they have been able to produce such a good product with the 416. This wasn't their go on the merry-go-round!
The SCAR program had nothing to do with the development of the HK416 as far as I am aware. The SCAR was developed around the concept of a multi-caliber/ multi barrel length carbine. One gun that could provide Special Operations with the flexibility to change weapons configurations per mission requirements. The goal posts for the SCAR development was moved a number of times before 'they' wanted two weapons, a SCAR Light and a SCAR Heavy.
FN clearly have the machine gun market cornered but the FN SCAR leaves a lot to be desired for me. Some recent changes to the SCAR reflect some of its deficiencies for a weapon that is suitable for Special Operations.
I have shot the SCAR and think that it does not handle as well as weapons built using the AR15/ M16 platform. The SCAR is overly large, particularly the forward hand guards, preventing the use of modern shooting techniques. I personally don't see a bright future for the SCAR. Especially when the HK 416/417 have been able to successfully fill that void.
The HK 416 is in service and fielded by the Norwegian Army. HK made some very specialized modifications for that project and some that have flow on to the U.S market with the release of the 416A5.
The 416C (PDW) is an interesting version. The stock for me does not lend itself for good check weld but for HK I guess it was their way of saying that they can build just about anything!
As for foreign militaries and the weapons they have selected, it is very difficult to change something that is already in the system. The 416 was developed for the U.S market and the Germans already had a weapon in their inventory. Trying to change that platform is like changing the M16 to a G36 for the U.S Army!
These are my thoughts only.
The HK416 is the best short stroke carbine on the market today and just like any weapon, if you treat it right it will treat you right.
I am yet to replace any parts on my HK416's due to usage. I have replaced a 'C-Clip' on the rod that retains the ejection ports cover. This has been fixed with the A5 model 416's as HK have changed the design.
I have replaced a firing pin spring after one of our students who was used to a direct gas impingement system inadvertently left it out during the reassemble! The weapon will still function without it but it's better to have all the parts in place.
I would stock up on the few items that you might need over time. The bolt will need to be replaced after about 15k (or shoot it til it breaks) the other common maintenance technique!
I would have an extractor, firing pin spring, some C-clips, a new bolt and thats about it for the upper.
If you are not shooting an HK lower, you may have to replace a fire control group every so often. Due to the higher than normal gas pressures and speed of the BCG during the cocking phase, the bolt hits the hammer very hard, which in tern strikes the disconnector. The disconnector is the part that typically breaks after time. If you buy an HK Two-Stage Trigger, this problem is solved by the bridge plate that sits over the disconnector protecting it from hammer strike.
There are some cool new products for the HK 416 series now from replacement rail assemblies from Remington, Geisselle etc. You will need one of the low-pro HK gas blocks IOT fit a newer type rail on though. The low-pro block does not feature the sling mounts allowing the narrower rails to go right on.