If you just want "fun".... Something like a Rossi in a pistol caliber like .357 mag, .44 mag, .454 casul, etc. would be good. Factory ammo isn't too pricey and they sre down right FUN to shoot.
2nd I would say is a .243 win. I was never really a fan until all my buddies got one and then I had to get one too! Minimal recoil, great all around hunting gun (specifically coyotes is what we target and then save the fur). Ammo is available at pretty much any store with a wide range of bullets.
Lastly, what if you didnt buy a new gun, but spent the money on a reloading set up? Until I started shooting odd ball calibers I didnt see a point in reloading. Then I startes reloading for odd balls (like my .17 Fireball) and said, "I would never load for my deer gun, it only gets shot a couple times a year!" Now i load for every caliber I own and love it. Its almost therapeutic for me. $2,500 could get you all the tools you need, as well as a good selection of dies, powder and bullets.
Good luck!