It depends.
If the neighbor's use was permissive, then their possession of the access route was not "adverse" and they have no rights.
If your seller told them not to go across his property and they did anyway, their use was non-permissive and they now have an easement *if* your state allows easements to be created by adverse possession.
Talk to the seller and get the facts in writing. A signed affidavit is best. Require the seller to either warrant that there is no easement for water access OR to discount the price of the lot. OTOH, if you love the house, don't lose the deal just because there is a small path along the edge of the property.
You should talk to a real estate attorney who is familiar with water access rights. Don't rely on the brokers' reassurances.
Good luck.
------------------
This is not legal advice and you are not my client. Double check everything with your own attorney and your state's laws.