With that cam/carb combination, a certain amount of lower rpm misbehavior could easily happen.
Possible cures would be heavier dashpot oil, richening the jet adjustments, & changing the distributor centrifugal advance curve. ie. More initial spark advance, & a quicker lower rpm advance, while limiting the maximum mechanical advance.
Something like 15 crankshaft degrees initial at idle, 25 degrees at 2500 rpm, & 35 degrees maximum at 4000 rpm.
Also heavier carb springs might help. I think the three spring weights available for the 2" carbs are - red/green = 11.25 oz, - lt. blue/black = 4.5 oz, - lt. blue/red = 18 oz. I don't know which springs you have installed now but you could go up to the next heavier step.
D