I have both calibers. A CZ in .17 hornet and 2, .204's one custom and a factory Ruger Varmint.
The .204 is wonderful in the patch and does give you a bit more range (350+ vs 250+) then the .17 Hornet but it is nosier and I find I take the .17 out much more often then the .204's
Also do yourself a favor and get a single shot follower from James Calhoon in Montana.. I find the mag's a pain in the butt with either my .17 or my .22 Hornet and for the patch single feeding is the way to go.
I have shot yotes at ~250 yards and they were DRT if I did my part.
I reload and I load 25gr. Hornady bullets they work the best for me and my rifle. I found the 15.5 gr. (Factory) worked well to about 175 yds then the factory 20grs. had a bit of an edge. Save your brass! you will want to reload, and DO get some help the round is a bit of a bear and requires a bit of finesse to reload till you figure it out.
I have Sightron 4.5 x 14.5 X 50 blue sky (no longer in production) on both of the Hornets with the HHR reticule and Warne rings on one and a knock off on the other I never really notice much difference between the two sets but the Warne rings are much better polished / finished then the knock offs. I tend to leave both of my scopes at 10 power for most of my gopher patch work.
You did not make a mistake and you will soon fall in love with the little cartidge that can ! .
If you could only have one 204, would you get the varmint model or a shorter sporter model which is easier to walk around with?