OK, as someone who just got into reloading, here is what I did.
First, I got a couple of ebooks on my ipad (don't have to wait for them to be delivered): Hornady reloading manual 9th ed. and the ABCs of reloading. Of the two, I found that the latter wasn't all that more informative and I would have been fine with just the Hornady manual. That, as well as Hodgdons reloading data site, should provide you with enough data to start reloading safely.
Then, I built a reloading bench. I made mine 4'x26" since that was all the space I could spare in my garage. The benefit was that I could make it out of a single full sheet of plywood and a bunch of 2x4s, so it wasn't too expensive. It is pretty solid setup. If you want, I can email you the Sketchup file I created when designing the bench.
I was going back and forth on what type of press I wanted. I decided early on that I didn't want a single stage, but it is the cheapest way to go especially if you are not loading a lot of ammo. In the end, I went with the Lee Classic Turret Press kit, since it came with many of the accessories I needed. In addition to the kit, I also got the Lee rifle charging die and double disk kit. Now the kit comes with a scale, but it isn't really good, so I ordered a Gempro 250 from Amazon, which is an electronic scale and much more useful.
I also ordered a Lee Reloader single stage press and deluxe quick trim and appropriate rifle die. The Reloader is probably the cheapest single stage press you can get. I wouldn't do any serious reloading on it, as it is pretty light and I'm not sure how well it will handle heavy use, but for things like case trimming, and maybe decapping it should be fine.
I ended up buying a set of Dillon carbide rifle dies, which is pretty much overkill. I could probably have gotten away with a set of Lee dies which would have been significantly cheaper. The Dillon set comes with a taper crimp die, but I also ended up getting a Lee factory crimp die. There is a bit of controversy over whether crimping is necessary. I've found no appreciable difference in accuracy between the taper crimp, factory crimp, and no crimp.
Other accessories included a Lyman case length/headspace gauge, a funnel, and a kinetic bullet puller. I did not buy a case tumbler, instead I just wash the brass with hot water and some laundry detergent, then dry in the oven. Works well enough but doesn't get the brass looking like new. Everyone raves about stainless steel media tumblers but they are a bit pricey and I would rather use the money to buy more components.
One other thing I got was some liquid lanolin off Amazon, and some 99% isopropanol from London Drugs, which makes a decent and cheap case lube. The Lee kit does include a small tube of case lube as well.
Overall, I'm pretty happy with my setup. I load .223 only, and I can get it to be pretty accurate, close to 1/2 MOA from my AR if I do my bit. Right now I don't load for handgun, mainly because it is next to impossible to find handgun powder, and bulk 9mm ammo is pretty damn cheap. Right now I don't shoot a lot of .45 auto, which is the only other cartridge I would reload, but that may change in the future. One nice thing about the Lee turret is how cheap and easy it is to set up another caliber.
I think, all totaled, I spent under $1000 for everything. Going with a single stage press and cheaper dies (and maybe a cheaper scale) would save you a few hundred dollars, so it might be do-able for about $500 or so.