Start with a copy of The ABC's of Reloading. About $30 from your local gun shop or Amazon.
Then look into an RCBS Beginner's Kit. Comes with everything you need less dies(think carbide only for handgun cartridges) and shell holder. Runs around $500ish plus the cost of the dies(all makes will fit all presses. 7/8-14 is the standard thread.) and shell holder, but there are sales on. Cabela's wants $349.98 and there's a factory rebate.
Buy a Lyman manual too. The kits come with a manual, but the Lyman book is more versatile.
RCBS just because of their warrantee. Bit pricier kit but the warrantee is 'replacement value' forever. Even if you buy used. Used isn't ugly either. Takes a lot of abuse to damage reloading kit.
Your local gun shop is as good a place as any to buy. You can buy equipment from the States, but no components(bullets, primers, powder and cases). Not really great given the current exchanged rate and shipping costs. Ditto for buying components on-line. Shipping will negate any cost savings.