If the person is already receiving benefits and you tried to call him in, and he gave a personal reason such as "no baby sitter" for his refusal, I hope you notifed the Dept of Labor (unemployment office) in your state at once. This should result in their stopping his benefits until they can investigate a "refusal of work." If in fact, he was drawing benefits and you told him if he did not come back he was discharged, then you should also have immediately notified the unemployment office. Then you participate actively in any appeals process he makes, keeping him from receiving benefits in the future.
A person who is seasonally laid off from an employer, but is considered "job attached" in other words, will likely go back to that employer when work picks up is not obligated to do a work search while receiving unemployment benefits. However, if his employer asks him to come back for a temporary job and he refuses without a very good reason such as mortal illness, his claim should be stopped. Since he probably won't mention this to the unemployment office, refusal of recall should be reported at once by the employer.
WHen they are notified, his benefits would stop and he could not receive any more until a fact finding investigation and decision was made. If he had not previously filed, only filed when you actually terminated him, were you not contacted to ask for the reason for his discharge? Was there not an opportunity to present your evidence of misconduct? Or has that not happened yet?
Do you have a company policy in place about refusing to go on a job...say so many times? Had this person ever received any write ups or even verbal warning about refusing a work assignment before? If so, you should have a very good opportunity to get his benefits denied. This, of course, will protect your tax rates. But you have to be participating in the adjudication process done by the unemployment office.
The reduced work load in the winter, even though the employee is aware it will happen each year, does mean he is eligible for unemployment if his wages in a week do no exceed the amount he would make drawing weekly benefits. It is illegal for an employer to discourage the employee in any way from signing up for this benefit, even if he is being provided a vehicle, or receiving any other non-wage incentives. All employers must have posters up about unemployment insurance and must cooperate with allowing their seasonal employees to receive benefits.
That said, each time a laid off employee certifies for a week of unemployment, he states that he has not quit work refused work or been fired from a job during that week. If your former employee was offered work by you and refused it, you need to notify the unemployment office immediately, and that would stop his unemployment. Even if he isn't being honest, and would prefer to draw and work under the table, you can do much about stopping his doing so if you pursue the protest of his claim appropriately.
If you can find out where he is supposed to be working, even though it is for cash "under the table", call the unemployment fraud hotline in your state and report it, and it will be investigated. (This, of course, is separate from the appeal process involved with your company.) I hope that when you received notice his claim has been approved, you have gone through the appeal process after reporting his refusal of recall. But it does no good to be in the hearing about your termination of him and maintain that you know the guy is working under the table. This is a separate issue that will be dealt with between him and the fraud unit.
If I were you, I would not so much stress how good we were to this guy giving him a company vehicle and all, but the fact that he knew the importance of being able to answer a call, that he knew he was subject to temporary recall, and he had refused you before and you had warned him
I hope you have given him such warnings, and that he was fully aware that such refusal could lead to dismissal. If there is any kind of progressive trail of discipline having gone on here, this can very possibly be called misconduct and he will not be able to draw off you any longer.