You don't say what kind of ammo you have been shooting. Assuming the ammo itself is very good, that it can produce very small groups at half the distance and still perform well at 100 yards, there are a number of factors that come into play with regard to group sizes, many of which have been referred to in a previous post. Below is a little more detail to the important points made above.
Using ammo with a consistent extreme spread (ES) is increasingly important as distance to target increases. As noted above, each 10 fps difference between one round and the next produces close to 0.25" of vertical spread. Very good match ammo can have an ES of 30 to 40 fps. That means that particular ammo can have one inch of vertical spread at 100 yards simply because of ammo velocity differences alone. Only a chronograph can confirm the ES of a particular batch of ammo.
Wind also plays an increasingly important role as distance increases. The following chart illustrates the effects of even relatively light wind at 50 yards. Note that a 2 mph cross wind at 50 yards moves the bullet .2". As noted above, without wind flags or indicators it can be challenging to account for the wind, which doesn't have to be significant or very noticeable to make itself felt.
And as alluded to above, it is worth considering how much bigger ten shot groups are than five shot groups. According to a report analyzing statistical groups sizes ten shot groups are statistically very close to 30% larger than five shot groups (see h t t p ://the-long-family.com/group_size_analysis.htm ).
And in general terms of statistics and the question of ammo, it is a good rule of thumb that when the distance is doubled, group size increases about three times. To illustrate, a five shot group that is .25" center-to-center at 50 yards will more than likely be closer to .75" at 100.