I would go ahead and do the box, at least replace the bushing & peg. It made a world of difference on my TR3A, like night and day. And I don't believe it is practical to change the seal without removing the box anyway. There is also a good chance that the seal is leaking because the bushing is bad.
As Moses says, it can be removed without pulling the front apron. But having done it that way, I am not totally convinced that it is less work or at least not much less. The cross tube has to be taken loose on both ends (which isn't easy with the apron in place), the horn removed and the motor mounts loosened so you can jack the motor up for clearance. Then you have to fiddle with turning the shaft to position the Pitman arm, as you lever the lower steering column forward past the mounting bracket and then lift it out. Lots of finicky work in cramped, hard to see places while leaning over the fender; compared to sitting comfortably while you loosen the dozen or so bolts that hold the apron to the wings.
Reinstalling the cross tube can be a challenge. You may have to jack the car up by the center of the frame, to get the towers to spread enough to get the tube into place.
When reinstalling the seal, you may find it advantageous to either leave it sticking out a bit, or drive it in deeper than usual, so the seal rides on the unworn portion of the shaft. Or just replace the shaft, as I see TRF now has them listed.