Telling your employer that you would take care of something "after church" or that you had family committments on Sundays or wished to attend church with them does not put your employer on notice that you needed a religious accommodation. It only puts them on notice that it's inconvenient for you to be available at certain times.
It appears to me that you were terminated due to your unavailability at certain times rather than due to your religous beliefs/a requirement to attend religious services. Since you occasionally worked on Sundays, you can't claim that you are prohibited from working on Sundays due to your religious beliefs.
I don't see that you have much of a position to pursue a claim of religious discrimination but you may wish to speak to an employment law attorney to get an expert legal opinion. A job not fitting in with your preferred life style, no matter how unreasonable the boss is, is not prohibited discrimination.
A person, who is nice to you, but rude to a waiter, is not a nice person. (This is very important. Pay attention. It never fails.)