You don't have to wear a vest. I don't. I put some thought into my rig and it is simple, cheap and efficient. All you really need is a pistol holster and some kind of bag to hold 4 rifle mags. In fact, for my very first match all I had was the pistol holster. I just tucked all the mags into my waist band. It worked. Good thing we don't run very far.
I am right handed. Pistol holster on right side of my belt. On left side is the pistol mag holster, so I can grab pistol mag with left hand and load.
Front center is a 4 mag pouch. The 4 mags loaded with 5 rounds go here. mags go in down, with bullet pointing to my right. The mags have a dot painted on the bottom to tell me which way they are pointed. The 5th mag, loaded with 4 rounds, goes in my back right pocket. This is for the start of another match, and is not required in a hurry.
At the start of the match, I draw a rifle mag from the right side of the pouch, since that one is the most awkward to reach. After the first 4 shots, I reload with the second mag on the right. Lots of time for that reload, so I use the second most awkward mag to reach.
I have doubled my rifle mags, to reduce the need to go to the pouch so often. This speeded things up. Speed is important in the transitions so that I have enough time to get all my pistol shots off.
To double my rifle mags all I needed was a block of wood, some glue and some tape. My kit cost about $45 for two pistol mags and $25 for the rifle mag pouch.
I bought a vest, but when practicing reloads I found the fact that I had to reach to a different place for each rifle mag made me slower. I am old and don't learn new motor skills easily.
Here is my kit. Probably the minimum, unless you just use pockets.
Here is my son shooting the minimum rig:
http://www.youtube.com/watch?v=X5WWAPJZcTU