You are most likely milking the gun. I posted the following on another forum earlier and it applies here. Hope it helps.
Grip the gun as hard as you can with both hands without causing tremors or degrading your ability to pull the trigger. For most people, this will be a lot firmer than they are used to or even think is correct. General rule of thumb; if you can hold full tension for more than 30 to 40 seconds you are probably not gripping firmly enough.
Accept that a certain amount of axial wobble is inevitable in an organic firing platform. However, realize that a A 2" axial wobble at the muzzle = a 2" group displacement on the target, which is very small. Nodal off axis or pivotal wobble is what we don't want, as small deflections at the muzzle = huge deflections down range.
While holding the gun in your steady-state hard grip and with sights aligned, prep the trigger to the wall of the sear or striker (if necessary, depending on the difficulty of the shot) and then rather than building pressure until the shot breaks, just commit to the shot and pull/jerk/yank it straight back through the wall. Note: you don't always have to prep the trigger. If the shot is easy/close you may be able to sweep the trigger through its entire range, breaking straight through the slack, the sear/striker wall and the over-travel, without deflecting your shots out of your intended zone. You mileage may vary so you will have to experiment to see what you can get away with at different distances. If your grip is correct you will be very surprised how far the target will be before you start missing.
Don't bother pinning the trigger back post-shot. Once the bullet has left the muzzle, no further follow-through is required or even productive, as you can no longer affect the bullets flight path. Just get your finger all the way off the trigger as quickly as you can and reset during the recoil phase. Basically, use the dead time during recoil to get the gun ready for the next shot. Also, don't bother trying to ride the sear or catch the link as that is a recipe for trigger freeze. Just get off it immediately and you will have plenty of time to be ready to shoot again the instant the gun recovers sufficiently to call your next shot.
Follow-through AS you make the shot, not after. In other words, see the sights settle on the spot you want to hit, hold them there and pull the trigger. What you see while the trigger is releasing is where the bullet will go. You may or may not see the sights begin to lift in recoil. If you are shooting multiple targets at warp speed, you will be driving your vision to the next target the instant the trigger is pulled. Seeing the sights begin to lift confirms the shot call, certainly, but it's a little slower than seeing the sights stop on the target, pulling through the wall, and just GO!
One of the major reasons people shoot low left or low right is grip inconsistency. The weak hand reflexively clenches as the trigger is pulled in a subconscious anticipation of recoil and we end up "milking" the gun down and away from our aiming point. If you are holding the gun as tight as you possibly can, the clenching cannot occur.
I know this will sound somewhat contrary to accepted technique doctrine, but it will work for you if give it a chance.