My current load is 26.6 grains of H322, Hornady brass, CCI primers and 32 grain V-Max bullets. I have averaged 3996 fps with this combination, this from a 26" barrel.
This is about halfway between the maximum load (27.5 grains) and the recommended starting load (25.5 grains) for this bullet weight. I tested loads up to maximum (no signs of pressure) but accuracy was best at 26.6 grains so I chose that for my varminting choice. I only recently got a chrony so I don't know what sort of velocity the max loads would have yeilded.
Its below the factory claim if 4225 fps but as with most factory claims this is a bit optimistic. I only measured one factory round and it was 4095 fps but on a cold winter day, maybe summer would be closer to the claimed velocity.
I wouldn't get to concerned if you can't duplicate factory velocities. I just did a quick calculation and with a 200 yard zero at 300 yards bullet drop is 4.7" for my 3995 fps load vs 4.13 with factory 4225 fps. I will gladly give up a bit of velocity to get better accuracy.
Don't let this deter you from a .204, it is a great cartridge for gophers. Recoil is light enough to watch bullet strikes in scope, and you truly want to see the bullet hit

(see video in my signature)
Its also a surprisingly accurate round, despite being long and skinny (the opposite of the PPC and BR cases) it seems to show inherent accuracy.
Here is a link to powders used in the .204.
http://www.rugerhunting.com/204_data.php