I try and do all my smelting when the wind is blowing gently, and not swirling. It has to be a crosswind for me, too. Otherwise it swirls in my face.
Years ago, when I took a machinist course in Winnipeg, we took a field trip to one of the aerospace outfits in the city. At one of the workstations where the guys had to work with nasty fumes, the workers got tired of complaining to management about the fumes not being extracted from the area. So with a roll (or 2 or 3) of duct tape, and some scrounged cardboard these guys made up a vent hood to get rid of the fumes. There might have been some sort of fan sucking the fumes out, too.
About 1 week after these guys rigged up the hood, management replaced their "improvised" setup with an "approved" setup exactly the same configuration as the cardboard, only made out of sheetmetal and rivets and screws and stuff.
No reason why a similar setup couldn't be used in a garage in the winter to keep the toxic fumes away from your face, and vented out of a window...I wish I had a garage to try this in.