OP, there are quite a few people who never anneal their cartridge cases, and just throw them away after 3-4 reloads. A few even throw them away after the second reload.
Most people never anneal.
It's up to you and what efforts you're willing to put into the care of your now very expensive cases.
For instance, I don't worry about my cases not being bright and shiny. Yes, I wash my cases, with hot water and soap, if they need it, but tumbling is off the table, as is sonic cleaning, etc. If the carbon builds up too much on the case necks, then a bit of bore cleaner on some steel wool is used to clean it off. Carburetor cleaner works good as well for some stubborn stuff.
That was off topic, but an example of your query.
Some component manufacturers, as well as some commercial loaders "anneal" the case shoulders and necks, and some don't.
I don't know how they decide to anneal, but most of the "premium" brands of ammo I've seen have annealed shoulders and necks.
When I did a tour of a company in Brazil, which was reloading cartridges for the South American militaries, they fully annealed the cases to dead soft, after inspection and cleaning, then inspected them again, and reheated the shoulder/neck portions of the cases before dropping them into water. They did this because dead soft shoulders and necks lead to issues while mass reloading on their machines, and neck tensions were heavier and more uniform, for use in full auto firearms. There were other reasons for it as well.
You don't say what firearm you're using your cases in. It's likely unnecessary to anneal before the first loading, but if you're "careful with temperatures," it won't hurt anything, and most importantly, it may give you more confidence with your end product.
A few people swear by it, some swear at it. Your choice.
