Velocity won't reduce drift anywhere nearly as much as going with a heavier bullet, even at relatively short range.
To test this theory,I plugged some numbers into a ballistic calculator and the results were.
140gr accubond at 3200fps(typical 7mmremmag)
wind drift with 10mph wind 400 yards-9.9" 500 yards 16.1"
160 gr accubond at 3000fps(typical 7mmremmag)
wind drift with 10mph wind 400 yards-9.8" 500 yards 15.8"
Going to a heavier bullet,both driven at maximum velocity improved wind drift by only .1" at 400 yards,and by only .3" at 500 yards.
Then we increase velocity by stepping up to the 7mmstw
140gr accubond at 3500fps
wind drift with 10mph wind 400 yards- 8.8" 500 yards 14.2"
The reduction in wind drift was 1.1" at 400 yards and 1.6" at 500 yards.The reduction in wind drift was much greater by increasing the velocity of the 140gr bullet than by simply using a heavier 160gr bullet of the same design.Theory proven incorrect at 400 yards and 500 yards!And you did post "even at relatively short range".
Now if you are talking 1000 yard target shooting,a heavier bullet does have significant advantages,but not for hunting out to 500 yards.
Once we're past 400 yards with ANY cartridge, wind and drop are MAJOR factors and much be compensated for much more with with skill than speed.
Bullet drop is easily compensated for if you have a laser rangefinder to measure the exact range.However,no instrument that you can easily carry,can measure wind direction and velocity all the way to the target.As such reducing wind drift is always an advantage.