In Ontario you're allowed to shoot wildlife that is damaging, or about to damage, your property. Officially bear shot in defence of property are supposed to be reported to the MNR, but unofficially, the local policy is to shoot, shovel, and shut up. I don't think this means you can shoot them for stealing the bird feeder.
As mentioned above, anything from a .30-30 to a 12 gauge is suitable. If I were using a shotgun I'd use Brenneke or Rottweil slugs rather than Forsters. Be aware that if you have to shoot a bear inside your house, the bullet is likely to pass through, and put a big, or very, very big hole in whatever is on the other side, depending on your choice of calibre.
Get a lockable bear-proof container to store garbage in ( an old freezer with a padlock works), don't leave dog or cat food outside and keep the bird feeders out of reach and most bear troubles will be avoided.
A lot of people on here pooh-pooh the idea that bears are dangerous, and I have to say that I've never felt threatened by a black bear. I've had more trouble with stray dogs. Even so, every year it seems someone is killed or mauled by a predatory black bear so I would never say it couldn't happen. Still you're going to feel silly walking around your property with a loaded rifle.