If you want to get started, the Remington 5R in 223 or the Savage FTR rifle in 223 are both great starting points. I like the remongton because it is the basis for most customized guns. the actions are slick and every possible accessory under the sun exists. They shoot very accurately by all accounts.
The Savage FTR was designed from the ground-up to do exactly what you want. It is an FTR single-shot rifle and is accurate. Barrels are as easy to obtain for these as any other rifle, and you can start here and simply add a custom barrel later-on if you wanted to upgrade, but I'm inclined to believe this won't be necessary at all.
There is a point of diminishing return in japing up a cheap gun like an SPS or Stevens 200. yes, they have a resonable action to start with, but it is only a repeater action. You need a better stock and you need a barrel to be properly installed and the stock bedded.
You'll end up with a good-shooting rifle, but you will have spent more and it will be worth MUCH less. Savage made it possible to get into the game and be ABSOLUTELY competitive with a purpose-built single shot rifle. Team Savage in the US uses these and i have shot with all of them. These guns work.