This is actually more of a family law question than a tax question.
It is true that never married parents have to have an IRS form in order to claim a child of whom they do not have custody, however, if a court orders the custodial parent to sign it, it has to be signed.
Sounds like it's time for you to hire an attorney.
This post does not create an agreement to represent you before the IRS, nor does it invoke confidentiality regulations. Postings are based only on the information provided and you should consult a tax professional in your area before relying on information contained in this post.