It depends whether the address on his driver's license is the same as your address, because (I believe) typically anyone in the same household is required to be listed as a driver on your policy to be covered, because such a driver with access to the keys or vehicle (with or without your permission) is a higher risk.
But I am not a legal expert, so I can only suggest that you read your policy.