How old is your TR3? If it has front disc brakes, then the TR4 wheels are a direct bolt-on affair; just shorten the studs or use spacers under the adapters. I would NOT try to swap the rear hubs (which is a major task), although you could swap the front hubs if you want.
I believe you can also install the adapters onto the earlier "Lockheed" axle (cars with front drum brakes), but there may be some fitting involved. In that case though, the rear hubs will definitely not swap (although the fronts might).
Hmm, I looked it up; I don't understand your comment about the front hubs being a different part number, unless you are looking at the early (Lockheed) front hubs. After TS13046 (introduction of front disc brakes), the TR3 used 114283/114284 hubs (the only difference is the length of the studs), which are the same numbers shown in my TR4 parts book.