Some say, all else being equal, cylinder size difference won't affect the pull as the fluid pressure is equal;
That much is clearly wrong. The force exerted by a hydraulic cylinder is the pressure of the fluid times the area of the piston. The area of the piston is 3.1415... times the radius squared (or if you prefer, .785398... times the diameter squared). So, for example, if the pressure is 100 psi and the cylinder is 1" diameter, the force will be 78 lbf. If the pressure is 100 psi and the cylinder is 1.25", the force will be 123 lbf. Quite a difference.
And 50% of the latter say pull to the left,
The theory there is that it takes time for the fluid to flow into the cylinder, so it would take longer to fill the larger cylinder. There is something to be said for that, but I think the return springs will more than compensate. The pressure does not rise instantly, it takes time for the pedal and master cylinder to move (and the fluid to move through the common part of the lines). So, again for example, if the springs take 10 lbf to overcome, the pressure in the 1" cylinder has to reach 12 psi before the piston even starts to move; while the 1.25" cylinder will start to move at only 8 psi.
Of course, there can be lots of other reasons for them to pull. The front drum braked TR3 I owned for awhile always seemed to pull one way or the other and it had the correct cylinders all round (I checked). The amazing part was that it rarely pulled the same way twice! Sometimes it pulled left and sometimes it pulled right. Never did work that one out. Instead I robbed the OD (which was the only reason I bought it in the first place) and sold the car.
Note that a size difference front/rear may be intentional, to account for the weight distribution of the car and possibly any construction differences in the brakes.
If it still pulls, check out the front suspension mounts to the frame. I had a TR3A that would pull to the left, and eventually discovered that the inner lower pivot for the RF wheel had torn loose from the frame. The pin was still attached at the other end, so it would spring back straight when there was no force on it, but the pin would bend under heavy braking and make the wheel turn to the left. Three different shops failed to spot the problem. That was around 1975, so it wasn't even that old a car at the time and not particularly rusty.
The pull wasn't too bad under normal braking; but got progressively worse as I braked harder. I finally decided that I had to stop driving it that way after someone pulled out in front of me, and I was unable to keep the car from jumping the median into what would have been oncoming traffic if anyone was there.