That's a typical media misrepresentation where they give you a simple nugget that on the face of it, seems unreasonable.
There are 6,820 rifles being procured, to be precise. The contract is for $32.8 million. Yes, that's about $4800 per rifle, but the contract is for more than just the rifles themselves. There is also a premium being paid to establish a new, low-volume domestic production line for national security purposes.
Part of the value proposition in that contract is it employs 30 new high-paid skilled workers in Kitchener, who in turn will pay income taxes, property taxes, goods and services taxes, to the point most of their salaries will eventually flow back to the government.
What isn't often mentioned as well is that the capital cost or procurement will include at least a couple years (probably more) of initial provisioning, armorer training, all the special tools and test gear to equip the armorers, and an in-service support contract to procure more parts, training aids, manuals, accessories, etc. because the Rangers will be way harder on their government issue rifle (to which there is no emotional attachment) than you ever will be with your gear. Theirs will see continuous rough use in bad bush. They will need lots of spare parts, new mags, time on the smithing bench, etc.
A quick google on the RFP abstract also netted: "Ancillary items, such as a sling, cleaning kit, trigger lock, soft case and hard case for each rifle will be procured at the same time."
In defence procurements, typically the total cost of ownership for gear is about 1/3 acquisition of the gear and 2/3 for the long-term support of that gear. Source: US DOD Defence Acquisition University. I suspect that is what is driving the cost of the Contract.
Disclaimer: I have no insider knowledge of this contract, but google is my friend!