Are all dowagers women?

By definition, a dowager is a widow or an elderly woman who holds stately position. Dowagers typically have gained status in the form of title or possessions from their deceased spouses.

From Charlotte Brontë's Jane Eyre:

But the three most distinguished — partly, perhaps, because the tallest figures of the band — were the Dowager Lady Ingram and her daughters, Blanche and Mary. They were all three of the loftiest stature of women.

Henry James writes in The American:

It was muffled in sociability and urbanity, as an old dowager in her laces and strings of pearls.

And in Lord Byron's poem Don Juan:

Remember, or (if you can not) imagine,
  Ye, who have kept your chastity when young,
While some more desperate dowager has been waging
  Love with you, and been in the dog-days stung