Likely what's spinning is not the bulb itself, but rather the little steel collar that provides some reinforcement of the capillary tube where it enters the bulb.
Most likely, you're going to have to remove the housing from the engine, so you can push on the bulb from the inside. The good news is that the bulb is already ruined, so no need to worry if you mangle it further getting it out.
I doubt they really need the bulb anyway, you could just show them the photo I posted. But you'll need the hole open to install the new gauge anyway, so might as well get it out now.
BTW, if all else fails, new housings are available (as well as good used ones).