Yep, if it's a Facet, cube-shape, solid-state pump, it can be very noisy. And all the time too.
But also simple, cheap and reliable.
I have one in my race car, where noise is not an issue. Very reliable.
If my street MGB needed a new pump, I'd consider one, just because I've been impressed by the Facet in my racer.....despite the noise.
For street cars, you can re-mount it on rubber mounts. This can help a lot.
Or chose one of the other type of pumps such as a Carter or even original SU pump.
If you replace it, be sure to use a low-pressure pump (less than 5 psi). Or use a higher pressure pump with a pressure regulator.