Same here in the summer. Last week at -20c I had to set it to 4.
I've shot mine with surplus around 20 below as well. I never had to change to setting 4 however. I did notice that about 1 in every 30 rounds or so the bolt wouldn't hold open after the last round was fired.
Setting 4 would give me wicked trigger slap!
It probably goes without saying but watch the chamber for laquer build-up, and give it a good scrub on occasion.
I have always found on my VZ-58's and SKS's that ejection would suffer after a goodly amount of rounds,especially in the cold.