Ok...let's break it down....right now the housing market is in the toilet and therefore most houses are not fairly valued...and MANY people are upside down who did not take out risky mortgages.
So, someone with a 200k home could have a 100k deficit. Under your theory, the person not keeping the house should have to cough up 50k towards the deficit. However, 5 years from now the market rebounds, housing goes back up to its proper value, and the 100k is recouped via appreciation, not via housing payments. Is your theory still fair or did the person who kept the house simply get a free 50k windfall?
Lets try another....a married couple each puts 5k towards a downpayment on a house. Six months later the marriage falls apart and because they just purchased the house, it doesn't actually have any equity, its upside down by 2k. So the person not keeping the house not only loses their 5k down payment, but also has to pay another 1k? Do you think that is fair?