Well, you could give him joint custody, but if you do it would still be giving him the right to object to you moving the child away. You could name him the standby guardian in your will should something happen to you, but again, even though the biological father is not in the picture, he still has rights and both he and your entire family could challenge your husband being the standby guardian.
All in all, you either let him adopt (if you can get the biological father's agreement) in which case he would be equal to you, or you don't let him adopt and he has no legal rights. If there were no father named on the birth certificate he could assert legal right because he was married to you when the child was born. However, since there is another man on the actual BC, he couldn't remove him without a paternity test.