I love when a guy comes on here asking about a new rifle, and everyone suggests he buys the cheapest rifles on the market...
IMHO you should buy the best you can afford, and use it as often as you can. Reliability trumps all else, and it is difficult to achieve reliability with marginal quality. If you have difficulty justifying the cost of a new rifle, look for older, quality used rifles in good condition. Upgrades can be made to your rifle over time once you have it, if the quality of the rifle justifies those upgrades.
Most of the suggestions are for bolt guns with scopes, and that also is my preference, but what is yours? Do you prefer a lever action, a pump, a falling block single shot, or a gas gun? Some cartridges are not available or suitable in some actions, while other actions can handle almost anything. Do you want a scope or is your preference some sort of irons? Will the rifle have a sling and if so will it be a shooting sling or a simple carry strap?
Often the features available for the rifle are more important than the cartridge it is chambered for. If you will be hunting in difficult country, is the weight of the rifle a concern, or are you tough and athletic and pretty much unaffected by any weight up to 10 pounds or so? You haven't indicated what part of the country you will be hunting in, but if it tends to be wet in your location you might find a rifle with a synthetic stock is superior to one with a wood stock. A good wood stock is more expensive than a good synthetic, but I would stay clear of what are referred to as tupperware stocks, these are the inexpensive plastic models that are often sold in sporting goods stores at very low prices. Much is made of stainless rifles for use in wet conditions, but in truth all rifles need to be properly maintained. An abused stainless rifle will not stand up as well as a chrome-moly rifle that is cared for.
With respect to the cartridge, there are some questions you need to consider. Will you handload? Some factory rifle ammunition is horribly expensive, and if limited to this stuff you maybe unwilling to make the investment that is requires to attain good marksmanship skills. Do you have enough shooting experience to handle heavy recoil. If not, anything that ends in the word Magnum, Ultra, or Express is best avoided. What is the longest shot you intend to take on game? Does the trajectory of the cartridge you want make such a shot possible, and does the bullet have enough retained velocity at that range to provide a reasonable expectation of a one shot kill?
To sum all this up, pick a quality rifle within your price range that turns your crank. Don't give up all you want to satisfy what you need, but don't give up what you need to satisfy what you want. Choose a cartridge that reflects the type of hunting and shooting you will do. Make sure the rifle is reliable before taking it afield. Shoot enough under field conditions that you understand what you can and cannot do on demand with that rifle. If you do not handload, consider it. While the initial set up cost can be intimidating, you can make better ammo that is more affordable than is possible with factory ammo; this translates into more shooting with more rewarding results which in turn leaves you wanting to shoot more.