There is supposed to be a small amount of play, everywhere except straight ahead. When properly adjusted, the steering box should hit zero clearance only in the straight-ahead position.
Although there are adjustments on the steering box, IMO they are primarily to compensate for manufacturing tolerances, not wear. If there is wear, you need to rebuild the box and replace the worn components.
IMO, wear in the bushing in the bottom of the steering box is one of the primary causes of stiff steering in these cars. The peg that rides in the worm hangs way out on the end of the shaft through that bushing, so any steering force tries to twist the shaft off-axis. The peg is the same V shape as the groove it rides in, so when the shaft twists, it causes the peg to bind in the worm.
Anyway, years ago when I finally broke down and did the box on my former TR3A, the result was like night and day. I could parallel-park with one hand, or change lanes on the freeway with one finger. (I'm not saying you should do those things, just trying to describe how much easier the steering got.) Of course I learned to drive with manual steering (on cars, tractors and trucks), so I'm more familiar with the little tricks like letting the car move a little while you turn the steering wheel.
I did do some other things at the same time, which probably helped too. Instead of the rubber "Silentbloc" bushings that have to deform to turn, I installed some free-turning pivots with a SS pin in a brass bushing. (If I were doing it today, I'd use one of Joe A.'s Delrin setups instead of the brass/SS from BFE.)
Also started using full synthetic GL5 in the steering box (same as I use in the differential).
And I spent some time trying to get everything lined up just right, making sure the vertical links didn't bind in the trunnions and so on. The brass/SS pivots are much less tolerant of any misalignment, so it's important to get the idler arm lined up with the Pitman (aka drop) arm.
Tires and how they are inflated makes a big difference too. I run about 32/36 instead of the 24/32 given in the book (for radials).
When I put my current TR3 on the road, I replaced the pin, bushing and seal as a matter of course. Unfortunately, it appears I didn't get the pin pushed into place just right, as it worked loose and made the steering really stiff (again by twisting sideways in the slot). Just something to check while you've got the box apart.
To adjust the box, back off the screw on top (or just remove the top cover) and remove all play from the steering shaft by changing shims under the end cover. What worked best for me was to add a shim, measure the float with a dial indicator, then subtract shims equal or slightly larger than the measured end float.
Then install and tighten the screw on top until you can just barely feel the box bind as it moves through center. You should still be able to turn the bare shaft with your fingers, but there should be a spot with noticeable drag (which should be the straight-ahead position). Note that the drop arm makes a slight angle to the steering column at the center point.
The worm in my 3A was worn badly enough that there were two tight spots, either side of center. No worms available at that time, so I just lived with the play in the steering.
Oh yeah, on the current TR3, I couldn't get the replacement felt (for the top of the column) to fit the way I wanted. So I turned a Delrin bushing to fit instead. Might have made some difference, maybe not.