Do you have herpes? If so, it would be hard to prove slander or defamation. He is telling family members and your nearly and or adult children who have long ago formed their opinion of you, his opinion of you. If you have no bones hiding in your closet, then laugh it hff and tell your family to consider the source. If you don't have herpes, then it is a different story, but the cost of a lawsuit and collecting would be very expensive.