want custody of two sons

My ex and I have joint custody of our two sons. He being primary custodian. We did use to see the boys equally. He never took very much time with them though. They stayed with his grandmother,who he lived with most of the time. We never had any visitation arrangements and he would never speak to me.
Now he has been with his new wife for about a year and he just served papers on me saying he wants sole custody. Since he met his new wife he has started whittling away my time with the boys. I now only see them every satuday and sunday. Before he never helped financially.He lives in an extremely small trailor that his parents bought us while we were married and is on their property which they are in the process of selling.He was fired from his job a few months ago.

Now,I have moved to a town 45 minutes away to live with my fiance.Which means if I get custody the children will change schools. We live in an apartment where there are many children and lots of love in our home. We are looking to buy a house in a few months

I worry constantly about the boys and I wonder if they are being cared for properly. I am so scared of losing them . Any advice?

What was the reason he got physical custody of the children ?

We were seperated for two years. I had the children during this time. He visited occasionally. I wanted to try to work the marriage out.He did not. We went together to sign the papers and I did not understand anything in the papers.Being stupid I trusted him when he said everything will be like it has been. I did not have a lawyer and being so young(21)I was also vulnerable.
I never even knew what the term "physical custody" meant until I looked over the papers a year later and began to study them.
I consutled a lawyer about this and she would not help me because I worked in an exotic entertainment club. So I decided to go to school full time now. My fiance supports me and my children when they are with me. If I could only erase the stupid mistake I made signing those papers**************..