So the ultimate goal is to load it as hot as possible, for the best accuracy? But what about when it appears to be less accurate when its hotter, and yet, the primer face still looks like #01 in the pix?
No, the ultimate goal isn't necessarily to load it as hot as possible.
Like I said, my Savage likes it backed off a bit. Down from 2,800 or so fps to 2,650 fps means group sizes shrink.
But, if one is shooting at long distances, HOT can sometimes be the order of the day... for example, if one wants to keep the drops within scope-adjustment on some models (e.g. without a 20MOA rail, if one is cheaping out

), or if one wants a real barnburner so it's less sensitive to crosswind.
If it's less accurate when you sizzle it up, then don't sizzle it up - nobody ever got hurt from a moderately-loaded cartridge... and it didn't do the gun any harm, either.
If you're still looking like column #1, and you've got an accurate load which just gets worse when you juice it up any farther, then leave it where it is - and be thankful that your rig shoots best at the lower end of the pressure/velocity spectrum, because your barrel will last incrementally longer!
Where this becomes a concern (for me, anyway) is when I'm loading the long-range barnburners, and I've got 'em riiiiiiight to the max. And it's 15ºC outside. And I know that I'll end up shooting them at 30ºC at some point. Now, if it's already 'maxed' at 15ºC, and you shoot it at 30ºC, you're in "Danger, Will Robinson" territory... or at least potentially, anyway.
So, if I cook up a batch that's maxed at 30ºC, based on pressure signs, then I know it'll be good at 15ºC or anywhere below that. But I can't do the reverse - max 'em at 15ºC and let 'em eat at 30ºC - and be guaranteed of safety.
Finally, even if it's cold outside, if you've left your ammo in the sun (the 'car greenhouse' as I like to think of it), it'll often shoot at WAY higher pressure.
A directly observable bit of this happened to me shooting low-pressure .300 Fireball subsonics in my AR15 about a month ago. Had a few on the bench, and a few more in the car - in the shining sun. The load was 1020 fps (just subsonic - 1150 fps is a rough guideline for the speed of sound, for most Canadian summer temperatures) when 'cool' on the bench under cover, but when taken from the car the same (identical!) batch was shooting 1260 and had a sonic crack.
So, in my experience, the pressure signs give you a good idea of exactly where to stop - and then, backing off from there, I know I've got a good load that has some safety margin which is dependent on temperature.
-M