With a single action you are only dealing with one physical action in order for the hammer to fall. You are pulling the sear out of the hammer notch. With a double action you are first pulling on the trigger in order to compress the hammer spring (or striker) and after a certain point the hammer will fall.
With a single action (the cleaner the break the better) you have your sights on target, decide to shoot / pull the trigger, and the hammer falls. With a double action you have your sights on target, decide to shoot / pull the trigger and pull the trigger and pull the trigger, and then the hammer falls.
Not only is the trigger heavier but it also is longer. It take quite a bit of work to pull straight back without pulling the gun off target. A single action gun with a great light trigger provides you with an opportunity for a NOW shot. When your brain decides you sights are on target and it tells your trigger finger NOW and BANG goes the gun.
So now if you are shooting the first shot double action and the remainder single action, you have to learn both. Some guns have a double action pull for every shot. At least there is only one action to learn but unfortunately it is not the best for accuracy.
Some guns are both single and double action but can be cocked for a single action first shot. The CZ-75 is like this. When cocked for single action on the first shot you want (will be required) to have the safety on when in the holster.
But if you can learn to shoot a double action well you should also be great with a single action. Some guns with "safe actions" are a compromise between a single and a double action. The hammer/striker spring is partially compressed so you don't have to pull as hard as a regular double action.
But as raks said, get a gun you like and want to shoot. And no need to stop at one.
