I'll transcribe an answer I received from an engineer at Diemaco a number of years ago when I asked the same question; (He talked alot, I took notes and my writing sucks so bear with me

)
By quality, I will assume you mean the 3 design components normally associated to building a balanced firearm?
They are, of course, Accuracy, reliability and durability.
Quality will usually translate into direct cost. But what is it that you want to achieve? If you are looking for just overall quality, then buy the best parts for the money. That approach will cost you more than what the end product might deliver. Or simply focus on the areas that you want the gun to be more than average at performing.
As a footnote to these points; keep in mind that even manufacturers who claim "Milspec" tolerances have acceptable variences within their component line. For example, to have an upper receiver/carrier pan tolerance that is Milspec at an acceptable varience level of +.0004, using a bolt carrier from another manufacturer/subcontractor at an acceptable varience level of +.0009 may not make for a very reliable or accurate rifle. Varience issues can (sometimes) be eliminated by acquiring parts from the same manufacturer. Although some might disagree when looking back at Bushmasters issues of many years ago.
(Never looked into that statement to any great length.......they did have lots of manufacturing issues)
a. Accuracy requires focus on (typically) the barrel, bolt/carrier, trigger group, upper receiver tolerances and optics/sights.
b. Reliability requires closer tolerances on minor lower receiver components and their relationship to upper receiver tolerances, and upper/lower alignment. Sloppy tolerances between upper & lower components almost always creates reliability issues.
c. Durability. Not normally a focus for the average civilian shooter, but parts not built to proper specification (or between manufacturers/subcontractors) will wear prematurely and begin to affect both reliability and ultimately accuracy. Durable parts (properly treated & with acceptable variences) should be acquired from experienced manufacturers, and not "parts kits" in a plastic sandwich bag from some unknown factory.
So, after all his rambling..........
I'd say upper receiver, bolt/bolt carrier & barrel