Personal preference? I have never had a 250, but owned a 4A with IRS when it was almost new and currently have a TR6. I like the smoothness of the 6 cyl engine, but sometimes I think that a live axle TR4A would offer the best of all worlds.
I had a live axle 4A for about 15 years, restored it and drove and enjoyed it all over the place. When I am on a rough road a like the IRS, when I was restoring my TR250 I missed the simplicity of the live rear axle, where you usually only have to worry about seals and shocks, not rusty bracket mountings and hubs, and splines and cracked diff mounts.
I would tend to agree with Randle a stock live axle TR is probably a better handler than a stock IRS car. A stock TR4 or 4A four cylinder probably pulls as well or even a little better than a stock US spec TR250/6 six cylinder. I never felt my TR4A engine was rough (at least after I rebuilt everything, it was a bit of a mess when I got it) although maybe not as smooth as the six, it really pulled strong and smooth between about 2-4000 rpm.
However, if you want a fast street car however, I think the later cars have a little more easily tapped potential, it is awfully easy to get about 20 more horsepower out of six with a shaved head and a mild cam, and, unless you have a slower stock car to compare it to, you would never know it is a breathed on motor. My car was a total dog as far as handling when I first got it back on the road, but when I put a matched set of sway bars on it, it now corners flat and neutral and still rides much better than a solid axle TR, and will put the power down out of a tight corner much better than my (admittedly stock) live axle TR4A ever did.
If you get a chance to drive some examples that is the best way to see what you like best. The early cars have more of a vintage ambiance about them as well, with more chrome in the cockpit, where the TR250 is somewhere in the middle, but after getting used to my TR250 was surprised how much the alike yet different 4A cockpit looked to be from a different are just because of the chrome vs. matte black accents on the instrument bezels and such. The look and feel of the TR6 is of course more modern still (to the point where I don't think wire wheels look good on a 6 at all).
There are differences, I like them all, somewhat a matter of taste, and also if you don't have a lot of these cars available locally what decent model comes up for sale.