Strongly suggest that you pick up a copy of ABCs of Reloading. It doesn't have much by way of 'recipes', but gives an excellent introduction to reloading, the types of presses and so forth. It will answer all sorts of questions you didn't even know you had.
Agreed on starting with a single-stage press. Progressives are great at churning out large quantities but are considerably more complicated (and expensive).
Here's my call on what you need (keep in mind that much of this is subjective, so a lot of it is just Ford vs Chevy opinion).
Must-haves
Press
Dies and shell holder for the calibre in question
Scale (which IMO doesn't need to be digital)
Loading block
Components and proper storage for them
A solid bench (which need not be concrete or made out of railway ties, despite what some will say)
Fire extinguisher (you won't need it unless you really need it!)
Reloading manual from a major ammo firm like Hornady, Lyman, Sierra, etc (Don't take any loads off the net until you have some experience!)
Good lighting
Priming gizmo. It can be one that fits on the press or a hand-held, but you will need one
Lube system for cases. My call is Imperial Sizing Die Wax.
Should-haves
Micrometer or calipers (needn't be digital, to my mind)
Stuck case extractor (cases only stick after the store closes on Saturday night)
Spare primer pins to fit your dies (they only break after the store closes...)
Case trimmer (for bottle-neck cases). You'll need it eventually.
Neck chamfering tool
Powder trickler
Primer tray
Nice To Have
Tumbler and media (cases can be cleaned by hand). One real advantage is that shiny brass makes it easier to spot cracks.
Note book to keep records of which loads work and which don't
Small flashlight
Magnifying glass
Powder dippers. As noted, Lee makes a good set.
Nobody makes bad stuff, but you get what you pay for. Lee works, but is lighter in construction. I have a Lee SS press set up permanently for primer extraction and an RCBS press for everything else. Much of it is personal taste.
I have an Excel spreadsheet which will give you payoff times, costs per round, etc based on what you plan on reloading. PM me if you would like it.
Last point - the only bigger gear-whores than reloaders are fishermen and there are all kinds of flashy gimmicks calling to you as you walk down the aisles. Start small and work up or your wallet will kill you.