If the employee quit in the midst of a termination procedure, that would cloud the issue a little bit. You might have to go into the reasons you were going to terminate. But if he quit when he was not in the office being terminated, it's considered a voluntary quit of his own choice. Is it worth it to fight this? Probably. The unemployment system is set up so that it is not terribly difficult to protest a person's receiving benefits. And when someone does receive benefits, it hurts the experience rating of the company, eventually its bottom line.
What will happen is that you will be contacted by the unemployment office, asked what the circumstances were that led to this person's being unemployed. You will need to very honestly answer that the person voluntarily quit, giving you two weeks notice, and then later came back in and tried to rescind the notice and get their job back. You opted not to give the job back. You do not have to mention any reason for this decision. It was the personal choice of this person to resign. They would have to prove, to be approved for benefits, that they had a good, job related reason to resign. A really serious reason, not just "I was being disciplined" or "I got mad at my supervisor." A reason like being subjected to sexual harrassment, or having a paycheck that wouldn't clear the bank, or being assaulted on the job by a belligerant co- worker. A really good job related reason.
Unemployment is not designed to pay people who just decided they couldn't take it, or they were having stress at home, or they didn't like their job, they got their feelings hurt somehow, so they quit. It is for people who are out of work through no fault of their own. If the company did not have any more work for them, or they were fired unjustly with no wrongdoing then you are determined eligible for unemployment. But this person quit, and then changed his mind about quitting. At this point you were under no obligation to give the job back.
This is what you should tell them. This does not take an attorney, a consulting agency, or even very much of your time. It's worth it, give the unemployment office the information they need. At some later point, you may receive notice that the person has appealed a denial and that you are requested to appear at a hearing testing the decision for the second time. This can be done over the phone. In any case, if you won the first one, you'd certainly present your material at the second hearing, making sure the person does not receive benefits because he quit your company by his own choice.