No, it is good to get it out.
Don't worry that we will think the worst of your husband. Everyone is a jerk sometimes. But they're also good guys most of the time. That's why it has to be for 'better or worse': there's always SOME worse.
That said: grrr. FU indeed! You are a paragon for NOT saying it.