What do you call a man who is dating a married woman?

Adulterer (or adulteress) can be used for anyone who has a sexual relationship with a person who is married to another, but again does not always imply the specific social situational that mistress does.