So, when you filed for benefits, apparently you failed to mention much about the reason you were discharged/quit by not calling in...whatever. If you filed your claim saying you were laid off due to lack of work, and then the company protested saying you had quit -- or had been determined to have quit because you failed to show up for work and failed to call in, then it sounds as though they were telling the truth.
Yes, keep certifying for weeks, though your claim has been stopped. Talk to the person from the unemployment office who will doubtless be contacting you. Send back in the information related to this separation that they are asking you about. The decision that they will make at this time will be an "initial decision." If after talking to this person, you receive a letter saying that your claim has been denied, then yes, you'll owe them those two checks back. It will take a while for them to set up an overpayment, because they are going to wait until you have had time to appeal the decision, and have a hearing, and then get another decision. That will probably take you five to six more weeks.
But from what you are telling us, frankly, the company probably has some sort of policy that if you are absent and you do not call in before shift, there is the assumption that you are quitting. Why, for goodness sake, did you not call in if you did not want to be fired for this absence? That's what they will ask you about. And unless you were in a car accident on the way to work and were in a coma, there's not a very good excuse for an unplanned absence without calling in.
So whether you are determined to have quit by doing this, or were fired by the company for doing this, you may not have a very good chance of winning your case and being approved for benefits. The only thing that might save the day is if it was common for you (and others who worked there) to fail to show up, and fail to call in, and you did not have any idea that this thing you had done before might result in your termination or might be interpreted as a termination. But it'd be very hard to run a company this way.
If you do not pay the money back after you have had the second decision and have been determined to be overpaid and have been notified of this (as I said, this will be some time out in the future) there will very likely be no lawyers and no cops breaking into your house in the middle of the night. But they will try to get the money, no doubt. You will definitely have to pay it before you can draw any more unemployment benefits ever. If you have a state income tax refund coming, or if you try to obtain a business license or do any kind of business with the state in the future, this will come up.