"Rumors" of molestation aren't going to get you anywhere. Also, even if they wanted nothing to do with the children before, they apparently want them now. The law recognizes that they are the childrens' fathers, and they have a right to their children.
I'm going to post the New Mexico statute regarding grandparents' right to visitation, but even though NM allows for grandparents' visitation in certain circumstances, this is by no means a slam dunk. First of all, the court will make its decision based on the best interests of the children. Secondly, this might not even apply to either of the children who no longer reside in New Mexico.
I am not an attorney. If you wish to pursue this, you need to consult with an attorney.
I'm going to edit this and remove the parts that don't pertain to you (visitation in the case of divorce, etc.) so if you want to read the entire statute go to
http://nxt.ella.net/NXT/gateway.dll?f=templates$fn=default.htm$vid=nm:all
40-9-2. Children; visitation by grandparent ; petition; mediation. (1999)
B. If one or both parents of a minor child are deceased, any grandparent of the minor child may petition the district court for visitation privileges with respect to the minor. The district court may order temporary visitation privileges until a final order regarding visitation privileges is issued by the court.
C. If a minor child resided with a grandparent for a period of at least three months and the child was less than six years of age at the beginning of the three-month period and the child was subsequently removed from the grandparent's home by the child's parent or any other person, the grandparent may petition the district court for visitation privileges with respect to the child, if the child's home state is New Mexico, as provided in the Child Custody Jurisdiction Act [40-10-1 NMSA 1978].
G. When considering a grandparent's petition for visitation privileges with a child, the district court shall assess:
(1) any factors relevant to the best interests of the child;
(2) the prior interaction between the grandparent and the child;
(3) the prior interaction between the grandparent and each parent of the child;
(4) the present relationship between the grandparent and each parent of the child;
(5) time-sharing or visitation arrangements that were in place prior to filing of the petition;
(6) the effect the visitation with the grandparent will have on the child;
(7) if the grandparent has any prior convictions for physical, emotional or sexual abuse or neglect; and
(8) if the grandparent has previously been a full-time caretaker for the child for a significant period.
H. The district court may order mediation and evaluation in any matter when a grandparent's visitation privileges with respect to a minor child are at issue. When a judicial district has established a domestic relations mediation program pursuant to the provisions of the Domestic Relations Mediation Act [40-12-1 NMSA 1978], the mediation shall conform with the provisions of that act. Upon motion and hearing, the district court shall act promptly on the recommendations set forth in a mediation report and consider assessment of mediation and evaluation to the parties. The district court may order temporary visitation privileges until a final order regarding visitation privileges is issued by the court.
I. When the district court decides that visitation is not in the best interest of the child, the court may issue an order requiring other reasonable contact between the grandparent and the child, including regular communication by telephone, mail or any other reasonable means.
J. The provisions of the Child Custody Jurisdiction Act and Section 30-4-4 NMSA 1978, regarding custodial interference, are applicable to the provisions of the Grandparent's Visitation Privileges Act.
