I am not a lawyer, but I am an engineer and have worked on a number of Superfund cases.
If you refuse and the EPA wants the samples bad enough they do have the power to require you to take them at your expense. But this is rarely, if ever, done with homeowners. If you don't grant access the odds are they just won't sample on your property, if they get half or so of the homeowners to let them on to sample that will probably be enough for their purposes anyway. And if you don't want them on your property I would not out and out refuse, just don't respond.
Making a decision like this is complex. If you allow access and they find contamination they will provide the results to you and if you ever go to sell your home you will have to disclose those results. On the other hand if your soil is clean you will have evidence of no contamination. As racer says if they find contamination, and if the responsible party is viable it would help support any claim you might make. And even if the responsible party is not viable the EPA might clean up your property, but that usually takes years, sometimes many years. To be honest in most cases if I were the homeowner I wouldn't let the on, but I do know more about the risks and implications than most folks and I certainly can't tell you that would be the best decision in your case.
Of course if you really do have some kind of toxins in your yard that could be causing risk to you or your family you'd want to know. But more often with soil the levels EPA considers problematic are below what would cause risk to a normal residential user. For example when EPA does a risk assessment at a residential property they assume you have kids who eat the contaminated soil, a worst case assumption. And most families don't have dirt eating kids, but a few do.
I would call the EPA people working the site, they always have public outreach folks to talk to. They can give you some idea as to what they are looking for and what this might mean. Most big Superfund sites have an active Community Action Group (CAG) made up of interested locals, EPA supports these groups and often provides funding so they can hire independent technical consultants. Ask if there is a CAG and who the contact for that would be. Also find out who the responsible parties are, who is paying for this. If its a viable company that would be different than if it was EPA using Superfund money but with no one to reimburse them.
Good luck.