You need to assess your loading procedure to determine where the bottlenecks occur, then take the necessary steps to address those bottle necks. I've pretty much given up on the hand priming tools and now use an RCBS bench mounted priming tool. IMHO, these are not only faster, but they provide a better feel and produce more uniform results. While on the subject of priming, which I consider part of case prep, I'm now in the habit of prepping much more brass than I intend to use over the short term. Brass prepping consumes the majority of the time dedicated to handloading, although admittedly not everyone puts in the same amount of effort with respect to sorting, cleaning, primer pocket uniforming, flash-hole deburring, trimming and deburring and so on. Still, if you prep some brass in advance of loading, the actual time loading will result in more ammo ready to shoot. This is particularly true if you want to try something new spur of the moment, and you only need to throw a few loads together. That's not the time you want to deal with an hour's worth of brass prep.
When a couple of steps can be reduced into one, that represents a time saving, although it might require an expenditure that you would rather spend on components, glass, or even another rifle. For example I was really frustrated with the time it required to trim and deburr brass. My solution was to purchase a Giraud case trimmer, which not only makes trimming faster, more precise, and with better uniformity; it also deburrs the case mouth at the same time that it trims it. Another example is when lets saying you are loading cast bullets, you need to expand the neck, seat the bullet, then crimp, 3 separate steps, right? But these steps would be much faster if you had used a turret style press, and simply had to advance each die over the charged casing without taking it out of the shell holder. The only problem here is that the only turret press worth a damn (at least in my opinion) is Redding's T-7, which no one will accuse of being an economically priced press.
Tazzy's comment about the Lyman 55 powder measure makes me think of another point. If your powder measure doesn't have micrometer adjustments, purchase the upgrade if one is available, otherwise, purchase a powder thrower with micrometer adjustments. The savings in time to initially set up is huge, and if minute adjustments are needed once you're under way, it only takes a moment to make them.
As a final word, don't be so concerned about saving time that the quality of the end product suffers. Handloading is only partially about saving money, its also about making better ammo.