Kudos to the op for noticing. Many would not.
Something tells be that you are smarter than the average bear and I doubt very much the problem is you.
The problem you noticed is quite normal and is caused by one of two things...
Either the close range zero is not stable and moves from one day to the next, or it is atmospherics of some sort.
You can test for short range POI shifts by shooting a group close like 100 yards and see if the offset is constant... If it is then look to the rifle... possibly bedding or action screw torque
If your 100 yard zero is correct, then look to atmospherics.
As stated earlier, lights up sights up... but understand that assumes the light source is above you... look where the sun is before you assume refraction is up... refraction is either toward or away from the sun... not simply up or down.
ON FLAT OPEN GROUND... A head wind causes rounds to hit low... a tail wind causes rounds to hit high.
A wind from the right causes rounds to hit high and left
A wind from the left causes rounds to hit low and right.
In hilly terrain you need to account for updrafts and down drafts based upon hills... or even tree lines on otherwise flat terrain.
Rounds go faster is warm thin air and slow down more quickly in cold thick air.
Temperature can also affect your muzzle velocity.
Also look into density altitude.
If you don't have a Garmin 701 or Kestrel with AB on it, you can download an app called BallisticsArc... you can pull weather data from a nearby airport and produce ballistics... compare your results to that and you may very well find the app anticipates the condition you noted.