Discriminating against own child?

My son's father has four older children from a previous marriage. We were not married- he threatened to kill himself if I didn't abort or give-up the baby. I kept him, and left his father when he was 8 mos. old. He started to pay support soon after, he was afraid to go to court, and so agreed to pay. His ex-wife wouldn't allow their children to be with me, when we were still together, so I stayed away every other wk. end so he could be with them. They were still married when we became involved. We agreed it would be difficult for them to have to be with me, and that it wasn't right for him to not see them to spend the time with me.
Then he said because neither of us had enough money, he should move in with a woman with money, that it would only be 'a business situation', he would still feel the same for me, we could continue our relationship. I refused, and left him. Turned out he already had the woman with money on the side (of course), and is now living with her. So, I 'got mine', but it's not fair for my son for suffer for it.
My son is no longer allowed to visit his paternal grandparents- whom he adores, because his ex-wife causes him problems with seeing his other kids, if my son visits his family. His father won't disagree with her, even though legally he could. He keeps my son his 'dirty little secret'. Pictures of my son are not allowed to be displayed at the homes of his family, so his other children won't see him. His father has pictures of the other children at his house, none of my son.
I told him my son is not allowed to be with his girlfriend. He honored this for his other children, and didn't sacrifice time with them. I feel that he did it for his girls, they should not have had to be subjected to me, and my son should not have to be subjected to her. His other children are taken care of financially and emotionally. They have their father's last name, he refused to let my son have his. They don't have the stigma of being illigimate like my son. He said he will only spend one hour a week with him, since the girlfriend can't be with him. He said if she could, he would take him two days a month, because he has his girls 9 days a month, and he needs time for himself too.
At Christmas and birthdays, he goes all out for his girls, and spends $10-20 on my son.
He says it's different because they are his kids, he was married to their mother. Society looks down on illegitimate children-it kills me that his own father does it too.
The things he did to me when we were together are unimaginable. Sometimes I would like revenge, but most of all I want my son to have a good life. I want him to have self-esteem, and that's not going to be easy, being labeled illegitimate. It's going to be so much harder when he's old enough to realise he's not as good as Daddy's 'legitimate' children either.
What recourse do I have? I know he'd treat my son better if I'd have agreed to 'continue our relationship', but how low can I stoop. Not that low. The only thing that really matters to him is money. Can I sue him for discriminating against my son? Can I sue her for interferring in our 'relationship', and 'stealing' his father, comdenmimg him to being illegitimate? One of the best parts is, he says that all he has to do to enjoy the lifestyle she gave him is to be good to her kids, because their father isn't!
He's good to them, too, for money, but not to his own son, because it's not convenient to him!
I appreciate any advice.

sounds like you have your hands full. I am not an expert by any means but you can't force someone to see their child as much as we feel it is necessary and 'healthier for a child". I believe now I don't know where you are ex what state. But I would take him to court for child support insurance and 1/2 of daycare. I'd also have him waive all visitaion rights to him so that you and your son can move on without having that in your life But as he gets older your right he will be subject to Mental abuse by his bio father