Let's say the big guy presents himself, you have your .243, but he only gives quartering away/toward angles, no perfect side-on shots. Would be able to tell yourself, in that moment, to not take the shot?
With a .308 or .30-06 these 'in the moment' decisions become a little easier, because the terminal ballistics have been considered and optimized.
A .243 rifle is the same size as a .308, and .30-06 is only slightly longer. The only reason you would not use one is you do not have one; fixed with as little as $300 or asking someone who will borrow you one.
My 0.02, offered free of charge. Happy hunting.