Healey--actually, Laycock De Normaville--overdrives are electro-hydraulic. When disengaged, 8 (IIRC) very strong springs hold a wet clutch engaged with a cone housing so that the drive is essentially one piece. When you throw the switch, the electrics open a valve through which highly pressurized oil is directed to two pistons that simultaneously disengage the cone clutch and lock a planetary gearset's sun wheel, causing the planetary gears to rotate about the sun wheel and drive an outer 'annular' gear which overruns the transmission output, hence 'overdrive.' As mentioned, very high hydraulic pressure, usually 400psi or higher, is achieved by a piston pump driven by an eccentric cam on the gearbox output shaft.
First, I wrote this:
My best guess, since this happens when the engine slows--and shifting from 4th to 3th will not interrupt the electrical flow which operates the solenoid that switches the HP oil circuit and prevents engagement in 1st, 2nd or reverse--is that the pressure from the pump is marginal. Theoretically, since the gearbox and OD are bathed in relatively clean oil the pump should last forever (I just went through my BJ8's OD and the pump was in good shape after over 200K miles). You can try a thicker oil--20W-50 works well--but the best diagnostic is to put a pressure gauge on the OD to make sure you're getting adequate pressure.
Then, after thinking about it, I wrote:
This may be normal. Since the drivetrain is slowing, your pump may not be able to maintain the necessary pressure below a certain RPM (even if it's in top nick). I can't say what this RPM is, because I don't think I've ever intentionally operated my ODs this way; when slowing, I throw the OD switch then blip the throttle to disengage the secondary circuit--which is there to prevent the overdrive from disengaging; and applying potentially damaging reverse torque to the OD--and either run in straight 3rd or 4th or downshift, then re-engage the OD when accelerating. Straight fourth and 3rd OD are just a fraction of a ratio apart, I can't really think of a good reason to do this. There is a roller clutch in the OD that I believe is there to prevent reverse torque from damaging the unit, but shifting from 4th/OD to 3rd/OD is basically dropping two gears.
Then, I thought even more:
I think this is normal. Sometimes, when I coast in gear/OD to a stop, then accelerate, the OD will engage when I hit 3rd gear at a certain speed. I believe this is because there is a minimum RPM required to actuate the hydraulics; if I had to guess I'd say it's around 1,500RPM. Don't think I've ever seen a spec.