I've got a few presses, and I'll use each for different purposes. Almost all of my precision rifle ammo comes off of the single stage press. I also use a Lee hand press a lot for various case prep excercises, and often for bullet seating: if I size and prime up a bunch of brass ahead of time, I can use the hand press at the range to do some load work right at the bench.
I have a turret press in the garage, it's an old one, beefy and very over-built. I actually don't use this one too much anymore, but it really works well for any 'heavy' processes that need a lot of elbow grease. Case forming, that sort of thing.
Then I've got a progressive set up, in 38spl. I like the Lee 1000 - it is a bit more finicky than the Dillon stuff, so if you're not mechanically minded, or impatient, don't bother. You've really got to be alert and paying attention to a lot of things at once on this one to prevent jams (which kill your loading rate, while you clear them.) But it works well for me; with the bullet feeder and case collator, I crank out about 700 rounds per hour. And given the reasonable price on this one, I'll buy one for each caliber that I intend to load for, rather than fiddle with setup every time I want to switch calibers.
Then for lower-volume pistol loading, I use a Lee turret press, with auto-index. It's not as quick as a full-on progressive, but it's much faster than single stage, too. If I'm going to be loading less than 4 or 500 rounds in a sitting, I'll probably use the turret press rather than monkeying with a progressive. Set up times for a progressive are pretty high. The auto-index turret press is good for about 200 rounds per hour, maybe a few more if you're using an auto-disk powder dispenser. I like it because I have the option of weighing out charges on my RCBS chargemaster - they measure out in about the same amount of time that it takes me to have a case ready for the charge, so it means that I'm not wasting any time waiting around, but it also means that I'm getting very accurate, weighed charges. So, for maximum magnum handgun loads, this is the set up I'll turn to: I'm rarely loading more than a few hundred of these at a time, and for maximum loads, I just don't trust a volumetric powder measure enough.
The single stage press is solidly mounted to the reloading bench, permanently. All the other presses are bolted to a chunk of lumber about 12" long, 4x6. So I take them out of storage and clamp them to the bench when in use, then take them down and put them away when not in use. Makes for more efficient use of space. And it also means that I can move them to the living room or kitchen if I want; do some loading in front of the TV.
Suffice it to say, the set up that will work for you will greatly vary, depending on your needs and expectations. For me, I'd start with a single-stage. It's by far the most versatile and easy to use; great to learn on. My emphatic recommendation (and I've used a ton of single stage presses) is the Lee Classic Cast, or the Forester Co-Ax (depending on your budget; the Lee is a little over $100, and the Coax about $350). Everything else is either poorly built, or over priced. Some, like the recent RCBS stuff, are both.
Other than a press, you'll need dies (any brand will do, I have lots of all of them, and they all work. RCBS are a good go-to, as are Redding), a powder funnel, shell holders, a scale (I highly recommend biting the bullet on this one and getting an electronic charger/scale like the RCBS 1500). You should have a case trimmer (The Lee set up is elegant and inexpensive). Also a debur/chamfer tool. I use the lock stud from the Lee Case trimmer to chuck the brass into an electric screw driver for trimming, chamfering, and other (optional) case prep like flash-hole deburring.
Also pick up a stuck case remover kit. You'll need it sooner or later. And I really recommend the Lee Safety Primer Feed kit if you do get a Lee Classic Cast press. Otherwise, get a hand priming tool. I have the Hornady one, and it works well; on-press priming sucks ass otherwise.