When I found my carb bodies/bore to be not perfectly round compared to the butterflies, I had a constant idle, but could not get the idle low enough. If your shafts are worn, and wobbling around, the idle would vary. If you hold the butterflies closed against the idle screw, does the idle still wonder a few hundred RPM or stay steady? -In order to see if the butterflies closed the whole way, I had to remove the carbs, turn out the idle screw and hold the carb up to the sun, looking through the bore to see how much light came around the butterfly (between the closed butterfly and carb bore). It doesn't take much light to have a high idle.
Another thing you might check is the spring that pulls the breaker plate (from the vacuum module) simply slides onto a cone shaped post on the breaker plate. If the end of that spring is stretched more round, and not fitting tightly against the post, the plate might randomly wiggle back and forth -which changes the timing a few degrees, and could effect the idle speed.
While you are checking out that spring, be sure the smaller spring on the mechanical advance weight under that plate always returns the advance the whole way -that spring needs to be under tension at rest. You can check it by gently rotating the rotor against the spring/weight force, and when you let go, it should snap back 'home'.