Your first hunting rifle should be much the same as your 20th, except that it should be for general use where the 20th rifle will probably be for specific use. If must possess the power, trajectory, and accuracy to allow you to kill a game animal with a single shot from any angle and within the range of your marksmanship limitations and the limitations of the conditions on the ground.
The rifle must fit you, and that fit must be while you are wearing the clothing you would normally wear in the field. When the rifle is brought to shoulder the butt should not catch on clothing, and when your cheek hits the stock the sights should be immediately in your field of view without having to squirm around to acquire a sight picture. The rifle should not be fatiguing to carry all day. It should be equipped with a sling that can be used as a carry strap and as a shooting sling.
It should have a robust set of iron sights that are easy to use, whether as the primary sighting equipment or as back up to a scope. If scoped, the scope should be small and robust, mounted as close to the bore as prudent in good quality rings, and not extend behind the rear bow of the trigger guard. A low power variable while ideal can involve some mounting difficulty due to the longer eye piece, a low fixed power scope will be less expensive and can still be used effectively out to 300 yards if you have the ability to shoot that far in the first place.
The length of the rifle should be no longer than you can manage under the conditions you expect to be hunting in; most folks do pretty well with a standard length action, a 22" barrel, and 13.5" length of pull which typically is an 8 pound rifle when equipped with sling, scope, and a magazine full of ammo. The choice between stainless vs carbon steel rifles doesn't matter in the real world. A rifle that is cared for will be reliable while a rifle that is abused will let you down. Choose a rifle that catches your imagination.
Choose a rifle that catches your imagination. There is no point in purchasing a bolt gun if your heart is set on a lever action. I would advise against a single shot as your first hunting rifle, as learning to make fast followup shots from the shoulder is an important skill to master.
While your question pertains to cartridges, that is the least important consideration of your rifle. If you can find a rifle that has every element you are looking for, but comes in .30/06 when you wanted a .280 Remington, it is the correct choice. The game animal won't be able to tell the difference and neither will you. My rule of thumb for big game cartridges is that the minimum caliber is 6.5 and the maximum caliber is whatever I can handle. But a general purpose hunting rifle should be able to be shot effectively from all field positions, including prone. Some find this impossible with the .30/06 while others can bang away happily with a .375. If your exposure to shooting is limited, you should probably start with a milder round and a 6.5X55 is a good choice, so is a 7X57. Once you've decided on the rifle and cartridge, you should invest in a modest handloading set up. This will allow you to shoot more per dollar than you could with factory ammo, and the initial set up need not be expensive.
Now that we understand the parameters of your hunting rifle, what does your budget allow? Few of us were able to start with custom rifles or even top line off the shelf rifles. Find a rifle that has the elements that matter. Don't make a purchase based solely on price. If possible, choose a sporting rifle over a surplus military rifle. While there are exceptions, and despite the military rifle's robustness, the action is apt to be rougher, the trigger poorer, and the accuracy less inspiring than what is commonly found in a quality sporting rifle. However, don't turn your nose up at the possibility of choosing a used rifle. The used market often allows you to purchase quality you would not otherwise be able to afford. But again, concentrate on the elements of the rifle that really matter.