PL.....May I suggest you might like to do some reading on internal ballistics. It's not that we are just giving you the "old boys club" standard line, it's that the answers are quite complex and although one may understand the whys, which I think I do, its not that easy to reiterate to another always. There are some very good books on the topics of internal and external ballistics........The Bullets Flight by F.W. Mann with 3 updates is a good place to start. The original treatise on the topic is a book by James Atkinson Longridge entitled Internal Ballistics. It was written in 1889 but is worth reading for it's initial understandings into internal ballistics.
I have 1/2 a dozen or more but I cannot remember titles or authors right at this minute and I think the books may still be packed in a box I have yet to open, from the move, or I would send them to you. I'm sure there are newer more up to date books on the topic, however the principles of spitting a projectile from a tube by igniting solid fuel and using the expanding gases hasn't really changed much in a few hundred years. The ability to measure certain aspects of it's acceleration and pressures have, but not the basic principle.
I find this field and external ballistics fascinating and used to read every book I could find or buy, now that I think of it some the books I read were lent to me, which would explain why I can't find them. Once you get a feel for why certain powder and bullet combinations (weights) do what they do, it opens a whole new realization and understanding of burn rate relationships to case capacities, bore diameters and bullet weights. Possibly it's just me, but I do not find this understanding easy to explain to others...........
I'm going to give it a bit of a go here though. I'm going to use some over simplifications which some may take exception to but I think you will get the basic understanding.
The reason slower powders give the highest velocity regardless of barrel length (within reason) is because you put more of it in a case, therefore you have started with more energy. If you assume that all gunpowder has the same overall energy per grain (which it more or less does) then you can see that more gunpowder in a given case has more energy to give. The differences between different burn rate gunpowders is at what rate they release this energy during their transition from solid to gas. The defining factor as to how much powder one can put behind a given bullet weight is known as the peak average pressure which has been designated for all commercial cartridges in NA by an organization known as SAAMI. I won't get into the finer points of SAAMI, you can Google it for all it's information and structure. The peak avg pressure is the limiter and is also a time sensitive factor which is determined by the powder burn rate. Faster burning powder consumes more of it's mass and changes it to gas in a shorter time frame than a slower, more inhibited powder, therefore it takes less time to reach it's maximum avg pressure (MAP) meaning a shorter steeper pressure curve in the barrel. This creates a shorter thrust stroke against the base of the bullet even though the MAP is the same as slower powders. This MAP is also reached using less powder thus having less energy to accelerate the bullet.
In doing some testing using extended flash tubes I was able to move the pressure curve even further down the barrel of a rifle thus giving a longer thrust stroke again and increasing velocity. By lighting the powder at the base of the bullet and keeping the powder in the case longer while burning as well as removing the weight of the powder from the initial projectile mass I was able to lengthen the burn pressure curve of extremely slow powders. This also has the advantage of less sandblasting effect on the rifle throat as the powder isn't being forced from the case through the funnel of the neck and sandblasting the leade. But I digress............
The length of time and barrel in which the MAP happens is one of the factors in setting up what is known as barrel nodes. Bullet weight, time of travel (in barrel) and muzzle pressure are other factors in the production of barrel nodes. The more uniform the barrel nodes from shot to shot the more consistently the bullet will exit, thus the greater the potential for accuracy. One of the best indicators of this consistency is the extreme spread (ES) of the velocity at which the bullet exits the muzzle. It has been my experience that in very short barrels (16 1/2" is very short for a rifle) that the lowest ES is usually achieved with more mid rate burning powders than with the slowest acceptable powders for a given cartridge. This is also dependent though on another factor known as expansion ratio of a cartridge.
The expansion ratio is the rate the volume in the barrel (behind the bullet) expands as the bullet moves down the barrel. (I'm terrible at explaining expansion ratio) . It is easy to illustrate though.........take a 458 WM, as the bullet moves approximately 2" down the barrel the volume left behind the bullet is approximately doubled. The powder must produce it transformation to energy much faster to use this greatly expanded area in which it is allowed to burn. Therefore the optimum powders in a 458 must be relatively fast burning. As I'm sure you can see the opposite situation would be a 6.5X300 Wby or 7mmRUM or a 22-284 which have a huge capacity for powder with a very small hole through which the expanding gases may escape (while pushing the bullet of course). These cartridges have a very low expansion ratio and are what are considered overbore cartridges. These cartridges require a very slow powder so as not to hit MAP before adequate acceleration has occurred. These cartridges also produce the longest pressure curve of all due to the fact that the bore is so small in relation to the amount of energy one can store behind the bullet (case capacity).