For deer and turkey, you could get one of the combo sets from Mossberg or Remington of a 12ga pump with smoothbore and rifled slug barrel. The slug barrel can take out deer up to 100 yards, and in most controlled hunt portions of southern Ontario is all that you are allowed to use. The smoothbore barrel can be used to take turkey, and with other choke tubes can also be used for upland game and waterfowl, probably everything else that you might be inclined to hunt. There are combo packages available in camo finish, which is a big plus for turkey and good idea for waterfowl.
If you hunt in an area where rifle is legal for deer, there's a variety of good, inexpensive carbines for pushing them in the thick bush (i.e. Winchester 94 in 30/30, various SKS carbines), though a more potent round, such as .308 or 30.06, maybe in a scoped rifle could be useful for areas with openings and in the event that you decide to take up bear or moose. A Stevens 200 in .308 or .30/06 with something like a Bushnell 3200 scope would do fine. A Norinco M14 could also work and is available for a decent price, but some hunters don't like the weight. Some people start of buying a less expensive SKS or sporterized Lee Enfield, and keep it as a back up gun if/when they upgrade to a more expensive rifle.
A .22 rifle can also be a good idea. You can purchase a brick of 500 rounds for a bit over $20, and if you practice with even just a couple hundred rounds through the off season (or better yet, even more), you'll just need to fire enough rounds from your centrefire rifle before the season to ensure that she's sighted in correctly, though a couple extra boxes once in a while to confirm your accuracy in various holding positions and ranges would be a good idea, especially at first. There's lots of decent new and used .22s available for anywhere from under $100 for some older used Cooeys to about $2-300 for some nicer but affordable models.
If you end up wanting to hunt more extensively, it's common for many hunters to have two shotguns (i.e. a nice light double or over-under for upland game and a heavier waterfowl/turkey/deer gun), two centrefire rifles (i.e. a short carbine in a lighter calibre, and a scoped, flatter shooting rifle for open country or stand hunting), maybe a third centrefire in a varmint calibre (i.e. .223 Rem) for coyotes, groundhogs and other pests. Add a .22 rifle or two (often a lighter plinker and maybe a heavier target rifle, which better mimicks a larger centrefire), and you'll be set for pretty much anything in the bush. If you choose wisely for your first shotgun/rifle though, you won't be too handicapped if you don't expand your collection and will probably keep them as part of the collective if you do.