If you didn't use wads, what would you fill the rest of the hull with?
Wads take up unused space and apply consistent compression to the powder. I'm just imagining loading slugs without a wad and there would be about 1/2" of air space inside the hull.
Lyman slugs with their hollow base may expand to seal the bore. At shotshell pressure they would need to be made from pure or near pure lead to be soft enough for it to work though. Lee slugs have the "drive band" in the base that prevents them from expanding. I've never tried Lyman slugs before but I know the pure lead Winchester Super-X 1oz slugs left loads of lead in the barrel of my 870 while no other brand, nor my handloads, do that.
Since slugs have far less bearing surface relative to their diameter I wonder about how well they would self centre in the bore without a platform under them applying relatively consistent pressure over the entire surface. If made from soft lead an angled initial acceleration could slam them into one side or the other harder and deform the slug. If made from harder alloy the base may not expand and they wont seal the bore. I suppose if they where oversized to start with and swaged in the forcing cone it could work but I'd worry about pressure spikes.
Personally I have tested a lot of different slug loads in both smooth bores and rifled barrels with Lee 1oz cast slugs and found the best accuracy was with some recommendations from the Cast Boolits forums. Put one or two 20ga .125" thick wads under the slug inside the wad with pedals attached. It is modifying load data from the manual since you are adding the extra 20ga wad in a 12ga shell but dozens of people reported increased accuracy with no other issues encountered and I had the same. It allows the wad to separate from the slug evenly as it leaves the barrel and doesn't throw the slug off randomly to one side. With plastic wads and Lee slugs the plastic can often be pushed into the base of the slug, locking onto the drive band, and not release cleanly when it leaves the barrel.
Load data for the Lyman slugs often already have 20ga or 16ga card wads under the slugs so you don't need to add them. It's just that load data for Lee slugs isn't nearly as common place and usually just involves a slug in a plastic wad, nothing else.