Hi Wangdango,
Yes, you can remove the heater blower without taking off the fender and I have found the most difficult part to be reinstallng the ducts once the blower is installed. I performed this effort early in the 1970s when I depended on my Healey as our family car and felt the defogger/defroster blower was just to weak to have any effect. Back then, you could roam salvage yards looking for parts and I found a wrecked Olds Starfire Conv. and pulled its blower motor and squeral cage. Since my blower housing was also cracked quite badly after being dropped, I epoxied it back together and installed the motor after mounting my original squeral cage onto its shaft. When activated, the blower produced an air flow with much more force and could now be used to heat the cabin AND deliver sufficient air to defrost the windshield simultaneously.
If you are going to go through the trouble of addressing your heater blower, I would suggest adding power to improve its performance.
Just my thoughts,
Ray(64BJ8P1)