Hi JPeter,
First, let me say something about your TR4. There are differing suspensions on those, too, depending upon when they were built. The earliest ones used a front suspension almost identical to most TR3. The upper control arms, ball joints and vertical links are interchangeable, as far as I can tell.
However, if your TR4 is later than approx. CT6000 or CT9000 (I forget which), it will have -3 degree castor. The early TR4 and all TR3 have 0 degree caster.
The change in caster was accomplished by changing the vertical link and upper control arms.
I'm pretty sure the lower control arms remained the same throughout.
With approx. CT20000 the TR4 steering rack was changed, now mounted on rubber. That also meant the tie-rods were changed. But, most other parts remained the same.
TR4A use a different mounting for the lower control arms, to allow for adjustment of camber by shimming the inboard ends of the lower control arms, and a different arrangement for the horizontal trunnion.
Your TR3 upper control arms should all four be identical, front and rear, right and left. They are pretty square shaped, with a sharp turn at the end, and join at a single hole in the center through which the ball joint bolts with a single, large bolt.
The -3 degree TR4 uses a more rounded style of upper control arms. These do not meet at the outer ends. Instead, the ball joint is sandwiched between them and held in place with two 3/8 bolts that pass through the control arm/ball joint/control arm. The rearward upper control arm is much shorter than the forward one. They are interchangeable side to side, just not front to back.
The mount for the upper control arms that sits on top of the spring tower *might* be the same on all cars. I don't have a TR3 to reference, but I think on the 0 degree caster cars, the mount was placed with the bowed part facing inward, while on the -3 degree cars, it was reversed and the bow faced outward. (I'm going by memory here, which sometimes is faulty these days.)
Also look at the vertical links. With the wheel and hub removed, the vertical link should appear to tip toward the back of the car, if it's a -3 degree version. It should be perfectly vertical if it's the 0 degree type. This difference is clearly visible if you have the part out and are looking at it, or if you take a couple steps back and eyeball it on the car.
I wouldn't think it a big deal to change a car from 0 degree to -3 degree, so long as you have the right combination of parts and you aren't concerned about originality. But, -3 degree probably makes more sense with a rack and pinion steering setup like the TR4 uses, than the TR3 type steering box. -3 degree caster will likely give heavier steering feel, which is compensated for by the rack & pinion. 0 degree caster would be a little "squirrely" with rack & pinion, but probably feels fine with a worm gear type steering box.
I hope this helps!
Alan