There are lots of possible problems, but IMO the most common is wear in the steering box combined with improper adjustment. My old 3A was exactly as you describe, and everyone told me "They're all like that, convert to R&P"; until I finally bit the bullet and rebuilt the steering box myself. The result was nothing short of amazing! I did make some other changes at the same time, the rebuild was part of a complete overhaul of the steering and front suspension, plus I swapped the original Silentbloc bushings for some free-turning all metal joints and filled the box with full synthetic GL-5 gear oil. But I believe that rebuilding the box was the major source of improvement.
The worm in the box was noticeably worn, but not available new at the time (they are available now I believe), so I had to live with the old one. The main thing, I believe, was replacing the bushing in the bottom of the unit. Wear in that bushing allows the rocker shaft to tilt in the fore-aft direction when steering force is applied, which in turn causes the peg to tilt and bind in the groove in the cam.
But careful adjustment is important, too. I tried several methods of setting the end float, the only one I found acceptable was to measure the end float with a dial indicator. (This shot is of the solid column from my current TR3, but the same setup worked for the split column 3A)
I avoided doing this for many years, as it seemed like a lot of work and I wasn't convinced it would make that much difference. Countless people had told me "They're all like that". But the difference was like night and day, my overall impression was "Why didn't I do this sooner?!"
FWIW, on the current TR3, I still had heavy steering after rebuilding the box. What I eventually discovered there was that I hadn't gotten the new peg pressed into the arm just right, and it had worked loose. Since I had a good used arm on hand, I replaced the arm & peg, which solved the problem.
Also FWIW, on the split column car, I was able to get the steering box out (and back in) without removing the front apron. In retrospect, I'm not sure it was any less work than pulling the apron, but it did mean less risk to the paint and trim beads. I had to take the engine loose from it's mounts and jack it up on one side to get room to maneuver the column out of the mount to the spring tower. Not sure offhand, but ISTR I removed the crosstube as well. And the horn of course. Lots of work with long extensions and swivel sockets with magnetic inserts.
PS, I should mention that IMO the box adjustments are only to compensate for manufacturing tolerances. They cannot compensate for wear without causing other symptoms, like heavy steering and even faster wear. The worm in the 3A looked terrible, with lots of damage to the groove, which I believe was caused at least partially by my misadventures in trying to adjust "for wear". So my advice is to set the box once, on the bench with proper tools and methods, and then LEAVE THAT SCREW ALONE.