I agree that its not an easy defense in a lot of cases. The question of how much change to the original is needed before its considered transformed into something new is often a tough one because in those close cases that get litigated, you could get a panel of judges or experts together and come out with a variety of views as to just which side of the line the work falls. It demonstrates, as you've noted in a number of threads, why getting permission from the originator of the work (if possible) is pretty much always the best approach.
I have often referred posters to the Jeffrey Koons cases because, as an “appropriation artist” who relies on the works of others to create his own work, Koons has been sued frequently for infringement. The Koons’ infringement cases generally center on the differences between transformative works (fair use) and derivative works (not fair use).
Sometimes the new work compared to the original work is clearly a derivative of the original, with no new expression or meaning added. The new work is just a copy of the original but in a different medium. And making derivatives is the exclusive right of the rights-holder.
Other times the new work is clearly transformative, with the old work perhaps only used as a jumping-off point.
Most times, though, the works require a side-by-side, careful and in depth analysis of the works, which of course adds to the cost of litigation.
All in all, getting permission is the cheapest and easiest way to avoid a lawsuit.