The way I see it, when you get a "factory gun" (non-AI/Sako/etc) you may or may not get a barrelled action that is capable of shooting to your accuracy expectations. You also probably won't get exactly what you want in regards to barrel length, twist, contour.
When you move up to a "semi-custom" (or a high end factory gun like AI, etc) it's very unlikely that the gun won't be up to your accuracy expectations. In the case of something like a Defacto, it comes with a high end barrel (IBI) so it should shoot as well as any other action with an IBI barrel on it (provided quality gunsmithing, which should be a given). Same thing for any of the "more affordable" custom actions that are built to tolerances tight enough to accept prefit barrels sight unseen (Nucleus, Ruckus, Origin, etc) with a quality prefit barrel from a reputable gunsmith. What you often give up with a "semi-custom" vs a full on custom is your choice of barrel length, contour, some calibers or specific reamers as generally a bunch are manufactured in the same configuration to keep costs down with those savings then passed on to the consumer. For instance, the Defacto appears to only be available with a 24" barrel.
A custom will be exactly what you want, down to the action, coating on action, barrel contour, caliber, reamer used, etc. Built a-la carte, hence the increased cost.