Check the erector screw on the bottom on the saddle.
There should be two on the bottom. One you can turn finger tight, the one that sticks out. The other one you will need a screwdriver to lightly tighten. If either one moves at least an 1/8th of a turn it should fix the problem.