Ok. Go TOMORROW and file for a restraining order against both him and his mother, on an emergency basis.
Then, make sure anyone who has your child (day care, whatever) is aware of the restraining order.
Sit tight. Until he actually files for visitation in YOUR county, he's not entitled to it. And no, he's not going to get emergency custody. And there's no such thing as emergency visitation.
STOP taking phone calls/communication from him NOW.
In fact, call the local PD/Sheriff and ask them to send an officer to make a report right now.