I don't have a choice if I want to be law-abiding. Here in the lower half of Ontario it's probably a good idea as the hunter numbers vs. the land available can lead to some very crowded conditions.
If I had the choice, I would wear it in (hunter) concentrated areas, but if I were hunting somewhere very remote I would not.
In Ontario, things like deer drives are common practice. In situations like this where you have say 10 guys lined up 40 yards apart walking out a patch of bush, its probably a good idea as it helps with visibility (i.e. where your partners are).
Having said all of that, I only wear the minimum amount of orange - a vest and a hat.
I wear my camo clothes under that, but not because I think camo is required necessarily. I buy camo because I turkey and waterfowl hunt as well as bow hunt. Most of the higher quality hunting clothes I have found come in camo anyways, so that's what colour I bought them in.