A worn out/ damaged drill will usually make a hole over size.
From what I see, they need much better machining technique that is more robust.
If their process is that un reliable, they need to 100% check all dimension, with gauge, CMM or whatever.
This will be very costly.
I don't know how much they know about machining, quality system etc. but IMO they really have a long long way to go. That is just 6061. Not Titanium or Incolnel or heat treated steel or anything 3 dimension.
Hire some more qualified engineers with solid experience and academic background in mass production, machining, mechanical properties etc will help.
A lot more than just QC....