So if I used tinyurl.com to create a link to an illegal mp3 site, and placed that tinyurl.com link on my site, then, according to your rational, I could also be held liable. I doubt that would hold up. Plus, for websites with dedicated IP addresses, a domain name really is just another "link" to the site and isn't even required to access the page.
I'm just playing devil's advocate because no one seems to be able to point to any case studies that resolve this question. It makes for a fascinating discussion, though.