Responsibilites taken away

I work in property maintenance. I have a new supervisor and he is taken some jobs away from me that I have been doing for last 8 years. I have been doing a good job but he wants me to do extra work at another property. That's ok with me because I want more responsibilites.But now he has given me an evaluation stating I need need improvement in "accepting responsibilites".

Is it legal to take away responsibilites and "claim" I don't accept them?

I work in Virginia which is an "at will" state and I'm worried he is trying to get me fired. What can I do?