If you’ve been in the market for real estate lately — or even just idly browsing your old neighborhood from curiosity — you’ve almost certainly seen the “Zestimate” splashed across a property’s Zillow listing. But now, multiple lawsuits are claiming those numbers are so far from realistic that they’re actually harmful, and Zillow itself is launching a contest, hoping to crowdsource a better Zestimate.
The estimates are based on a proprietary formula, Zillow says — publicly available data goes in, magic number comes out. And what happens in between has some homeowners and developers completely confused.
Now, Zillow is under fire for the results that formula gives them. It began with a lawsuit filed by an Illinois real-estate lawyer in early May.
Her suit says that although Zillow cautions users not to take the “Zestimate” as an appraisal, it nonetheless has the same effect: It’s a number “promoted as a tool for potential buyers to use in assessing [the] market value of a given property.” That, the suit claims, meets the Illinois state definition of “appraisal.”
The Zestimate given for her house undervalued it badly, she claims, making it much harder for her to sell. The most recent Zestimate for her property values it at $64,000 less than she paid for it in 2009, despite other similar homes in the neighborhood selling for significantly more.
The difference between Zillow’s estimate and the seller’s actual sale price can, indeed, be pretty wide. For example, on this randomly-picked house outside of Washington, D.C., the gap is nearly $600,000.
A few weeks later, the lawyer dropped her claim but filed a second suit — this time, on behalf of several Chicago-area builders. This one seeks class action status that could “consist of millions of homeowners,” and seeks an injunction preventing Zillow from publishing Zestimates for the time being.
Zillow itself concedes its automated math isn’t perfect. Zestimates are within 5% of the actual sale price about 54% of the time, within 10% of the sale price about 76% of the time, and within 20% about 90% of the time, Zillow told the Washington Post.
But even 5% is still a large amount of money when you’re talking about hundreds of thousands of dollars. And that’s just for the most accurate range. By Zillow’s own figures, 1-in-10 Zestimates are off by more than 20%. That’s like valuing a $200,000 house at less than $160,000 or more than $240,000 — an enormous difference, for buyers and sellers.
The accuracy of estimates also varies hugely by region. MarketWatch reports that the numbers are within 6% of the actual sale price barely 44% of the time. In Washington, the Seattle Times ran the math and figured out that Zillow’s median error rate, applied to Seattle’s median home price, will be off by about $40,000 in either direction.
“We believe the claims in this case are without merit. We always say that the Zestimate is a starting point to determine a home’s value, and isn’t an official appraisal. It’s a computer-automated estimate of your home’s value,” Zillow’s spokesperson said in a statement about the lawsuits.
While Zillow defends the Zestimate, it has nonetheless launched a competition encouraging data scientists to come improve its algorithm.
The Zillow Prize competition takes place in two rounds. From this week until Jan. 2018, teams can enter the public qualifying round by developing a model “to improve the Zestimate residual error.”
The teams with the prototypes that most narrow the gap then advance to the second round, which runs from Feb. 2018 until Jan. 2019. Those 100 teams have to build an actual algorithm to do what the Zestimate does. The winner from that round gets $1 million for their trouble.
“We still spend enormous resources on improving the Zestimate, and are proud that with advancements in machine learning and cloud computing, we’ve brought the error rate down to 5 percent nationwide,” Stan Humphries, Zillow’s chief analytics officer (and Zestimate inventor) said in a statement.
“While that error rate is incredibly low, we know the next round of innovation will come from imaginative solutions involving everything from deep learning to hyperlocal data sets — the type of work perfect for crowdsourcing within a competitive environment.”
And although the timing may seem a little convenient in proximity with the lawsuits, Zillow says it’s not. Humphries tells the Chicago Tribune that the company has been planning the contest for a year and it’s unrelated to the recent lawsuits.
by Kate Cox via Consumerist