I'd have no worries at all about using 20 gauge buck for deer, as long as I could put enough pellets in the vitals at the range I was shooting. Wouldn't want to use it to shoot running deer, but it's certainly capable of it, just like any other gun, if the user is up to it. Most aren't, though!
Shooting a moving deer is risky, but of course most hunters have done it or seen it done. They've also seen what goes wrong if you screw it up.
The beauty of buckshot is that you can put it in any old shotgun, even a single shot with a bead sight, and it will work very well inside its parameters. The most affordable way to start deer hunting, and in my opinion, the best, as you've got to learn a lot of other useful skills to compensate for the lack of range. Same as bowhunting, but with less silliness.