Even if it was a patriarchal power play, it is essentially harmless in the physical sense. The price we pay for living in a fairly free and open society is that people will say dumb sh*t to us and about us.
But as Daniel Suelo once said: "What other people think of you is none of your business."