Seller refuses to return earnest money

What is the name of your state? Georgia

We put an offer on a house contingent on getting approval from the Home Owners Association to build a detached garage on the property. The HOA wrote a letter stating they would approve a 4 car garage as long as it looked like the house, BUT at the bottom of the letter it said "We cannot officially approve or deny anything until the above party becomes members of the HOA". Well, we went ahead and submitted the plans for the garage to the Architectural Control committee - and they said they couldn't tell us anything until we were members. My husband then told them our offer was contingent on getting an approval first. They didn't know that (The listing realtor never told them). They scrambled to review the plans - telling us that they didn't approved them. We redesigned it - resumbitted plans and the day of Closing they said they still had an issue with the size of the garage. (We own several classic cars and want them to be protected - we needed a 4 car garage). So we backed out of the deal. It clearly stated on the offer that if the contingencies had not been met - the buyer had the option to terminate the contract. We signed the termination of contract form and the wife of the seller signed it BUT HE WOULD NOT. He does not want to return our earnest money.

My questions: Does he have a case? How long can he drag this out?

Any advice is greatly appreciated. Our realtor won't give us concrete answers - she says she has her broker checking on things.

Thank you!!!!

Get yourself to a real estate attorney in the county where the property is located ... and pronto!