News for hackers. Stuff that matters.
Re:Zillow just gave us a look at machine learning's future by cs702 on Sunday November 07, 2021 @10:15AM (#29140104)
I agree with the OP that "it's put up or shut up time for Data Science teams. When the business depends on our work for revenue growth, success is expected. Failure is fatal." Zillow's CEO blamed the Data Science team for the failure of a mission-critical business unit and fired 25% of all employees. The stock lost a third of its market cap in a few days. When revenue growth depends on an ML/AI system, failure is indeed fatal.

However, I disagree with the OP's diagnosis.

The initiative failed because to a close approximation the only homeowners who would routinely accept Zillow's offers were those whose homes could not fetch anywhere near as much via traditional sales channels. Everyone else would rather sell their home via conventional listing. Zillow gave homeowners a free put option: Whenever Zillow's offer was above market, the homeowner would take it; otherwise the homeowner could freely ignore it. The incentives were strongly stacked against Zillow from day one.

Zillow unwittingly launched what I would describe as a seller's market for lemons. Naturally, Zillow ended up buying... a lot of lemons.

Re:Zillow just gave us a look at machine learning's future by mattmcknight on Sunday November 07, 2021 @10:33AM (#29140263)
I made the same point on twitter a few days ago. https://twitter.com/mattmcknight/status/1456096267768147971

Even CarMax and Carvana with commodity products in the traditional lemon market have to work around this problem by adding an inspection to confirm the data- and they have a large spread between their buying price and their selling price.

Re:Zillow just gave us a look at machine learning's future by ethbr0 on Sunday November 07, 2021 @10:51AM (#29140431)
> large spread between their buying price and their selling price

This is the market I was amazed Zillow didn't get into: the "WE BUY UR HOUSE 4 CASH" market.

I'm guessing the volume is too low (few people sell their house for stupid low offers) and advertising costs are too high (those people probably aren't obsessively checking Zillow for their home's value every day).

But still, it seems like easy money, that someone far less knowledgeable and talented than them is currently scooping up, that would better insulate them against market turns.

Re:Zillow just gave us a look at machine learning's future by JumpCrisscross on Sunday November 07, 2021 @12:17PM (#29141335)
> the market I was amazed Zillow didn't get into: the "WE BUY UR HOUSE 4 CASH" market

They could have bootstrapped by offering a product to those cash4home operations.

Re:Zillow just gave us a look at machine learning's future by benjaminjackman on Sunday November 07, 2021 @12:17PM (#29141341)
And don't forget flipping a car is a lot faster and easier than flipping a house at much lower capital. This makes each cycle of trial and error a lot cheaper and a lot more frequent.

Also there is something to be said for a market where the customer brings the product to your location for inspection, and whose product is considerable faster to inspect more rigorously than a house.

Also the products within that market are much more uniform and share several unifying characteristic across even different makes and model so its easier to learn a good and accurate pricing function.

That gives them a lot more cycles to improve their algorithms (both human and machine based).

Plus when CarMax were starting out they likely did so with a bevy of folks already experienced and profitable at buying and selling used cars at a profit who had the systematic knowledge of how to do so in their heads.

It then was simply a matter of building up customer goodwill by focusing on another metric customers cared about, taking a perhaps slight loss to avoid the risk of being considerably ripped off by a savvy local dealer and saving oneself the anxiety induced by high pressure sales tactics.

Having said that I think Zillow could still be quite successful in this endeavor however from my experience they didn't deploy their trading algorithm in a prudent way, they massively over-traded an untested by the real world model, they trusted their paper trading (backtesting) way to much, and they didnt focus on the nuts and bolts of building a successful market making system: making actual money over and over at small size and ramping up size as your empirically demonstrated pnl grows.

Also putting trading aside they didn't have to be a flipper, they would probably naturally be more fit as an exchange+ or intermediary that provides exchange type backing and reduces the friction in buyers finding sellers, taking on risk by only making an offer when they have a bona fide buyer for fixed or % fee, but glossing over all the hassle of two people who dont know each other making a large trade. Offering services like if you dont like the house you just bought through us we will help your sell it at a cost proportional to renting no questions asked.

Re:Zillow just gave us a look at machine learning's future by treis on Sunday November 07, 2021 @02:47PM (#29142796)
Carmax and Carvana also provide a service. They bring a bunch of cars to one place that you can peruse and buy. That's a better experience than trying to connect with a bunch of randos on craigslist.

Zillow hasn't gotten to the point where they provide that, but theoretically they could. You could drive around with a Zillow agent and see their 50 homes for sale a lot quicker than coordinating with 50 random sellers.

Re:Zillow just gave us a look at machine learning's future by ethbr0 on Sunday November 07, 2021 @10:39AM (#29140312)
The broader point is... Zillow's alpha should not be valuation, for all the reasons you listed.

It should be geospatial trends.

There's a fixed margin on any given home, caused by the inefficiency of real estate markets. But many actors (the seller, other agents) have an inkling of what that margin and the numbers on each side of the sale are.

What they do not have as good of a bead on, and Zillow is best placed to know, is high vs low neighborhood boundaries, and how they're moving, across time.

It seems like there would be more (and more reliable) margin in betting on the boundary movement, that the seller isn't aware of yet, than the absolute value of a particular home.

And yes, they are presumably capturing this in their price difference, but it seems like it should be a core foundation.

Re:Zillow just gave us a look at machine learning's future by BryantD on Sunday November 07, 2021 @10:23AM (#29140170)
I think your basic point is good -- a free put option is always going to be a problem.

However, can you unpack "Everyone else would rather sell their home..."? At the core there are two factors at play in this decision. One is financial; anyone with the luxury of time will compare a realtor's comps with an ibuyer's offer. The second is certainty; an ibuyer offer is guaranteed, and can be executed without having to show a house over the course of 1 to X months.

It seems to me that certainty has value to some sellers in some situations. Right now, the certainty value of the traditional route is pretty good because the market is on fire. That is not always the case everywhere.

Re:Zillow just gave us a look at machine learning's future by cs702 on Sunday November 07, 2021 @10:29AM (#29140224)
It depends mainly on how much more money the homeowners believe they can fetch by listing the home versus selling it right away on the spot.

For example, if you're the homeowner and you think you can get, say, a few extra $100K's by listing your home instead of selling it right away on the spot, you would probably take your changes and list it. Even if you're under pressure to sell quickly, you might still try listing the home for a couple of months to see what happens. Only if you think your home is a lemon will you consider selling to Zillow.

That's why Zillow ended up buying all those lemons.

Re:Zillow just gave us a look at machine learning's future by imajoredinecon on Sunday November 07, 2021 @11:09AM (#29140615)
> Only if you think your home is a lemon will you consider selling to Zillow.

What if Zillow just makes an offer that’s high enough that you think it’s unlikely you’d do better if you went through the hassle of looking for other buyers?

Re:Zillow just gave us a look at machine learning's future by leohart on Sunday November 07, 2021 @01:38PM (#29142144)
Even if Zillow does, they still take a 12% fee. When I was selling my house a year ago in Texas, Zillow's offer is both lower than comp in the area AND has a hidden 12% fee. Did you know they also charge YOU the repair fee?

I had a chance to talk to an inspector who looked at my place who has previously worked with Zillow to inspect houses that they were buying. The gist is that Zillow are getting the worst houses (the ones with trash everywhere, burnt carpet, feces on the wall). For them to get a "normal" house is very rare. Nobody with a normal house (and better) under normal condition (and better) will sell to Zillow. You can get a 1%+3% fee full service (pro pix, pro staging, pro cleaning, ...) any day of the week and still sell your house in mere weeks. I was not even in Austin. I guess if you don't care about making tens of thousands in a month (or hundreds of thousands in nicer cities), then you have bigger things to worry about.

Zillow's offer was already higher than OfferPad and other iBuyer's services. After my experience on the sell side with Zillow, I am incredibly wary of ALL iBuyers on the buy side.

Re:Zillow just gave us a look at machine learning's future by JoshCole on Sunday November 07, 2021 @12:05PM (#29141212)
Lets say P(h) is the price that a home fetches after being listed, O(h) is the price that an owner expects to get from selling the home, and Z(h) is the price that Zillow expects to get if they sell the home. An owner will sell to Zillow when O(h) < Z(h). They will not sell when O(h) > Z(h). Zillow's outcome would then be (P(h) - Z(h)) * (Z(h) >= O(h)).

The extremely important thing to note is that this is an asymmetric information game. Therefore O(h) < Z(h) is going to be self-selected such that O(h) < Z(h) because P(h) is closer to O(h) then Z(h) is to P(h). If Z(h) was actually able to be a reliable and better estimate of O(h) then it would mean that P(h) wasn't determined by information available to O(h) that was not also available to P(h). The market would already offer this value to O(h). So O(h) value is going to be reflective of Z(h). Its only if the non-public information changes the value that O(h) != Z(h).

Re:Zillow just gave us a look at machine learning's future by acjohnson55 on Sunday November 07, 2021 @06:26PM (#29144595)
Like the previous posts said, the advantages of selling to Zillow are price certainty and immediacy, so your model isn't quite right. But I agree with the adverse selection problem you highlighted.
Re:Zillow just gave us a look at machine learning's future by essdas on Sunday November 07, 2021 @10:32AM (#29140252)
Exactly. And to add to what you say, adversarial problems in data science are much harder than the regular ones which data scientists deal with and the hubris that I often see with data scientists won't cut it here. You need a different mindset to tackle these problems and they must be handled incrementally with the real world in the loop.
Re:Zillow just gave us a look at machine learning's future by curiousgal on Sunday November 07, 2021 @10:22AM (#29140166)
Re:Zillow just gave us a look at machine learning's future by PKop on Sunday November 07, 2021 @11:36AM (#29140898)
>those whose homes could not fetch anywhere near as much via traditional sales channels

Isn't this just another way of saying the models overstated the value of the homes?

Re:Zillow just gave us a look at machine learning's future by kshacker on Sunday November 07, 2021 @11:56AM (#29141122)
On an average they may not have. Three separate million dollar houses get offered 900k, 900k and 1.2M. The point is that the guy offered 1.2m knows he will not get the 1.2 anywhere else and takes the offer.

The individual house is overvalued but you don't find out until after, let's say, 7000 of those sales.

Re:Zillow just gave us a look at machine learning's future by imajoredinecon on Sunday November 07, 2021 @11:06AM (#29140561)
> Zillow gave homeowners a free put option: Whenever Zillow's offer was above market, the homeowner would take it; otherwise the homeowner could freely ignore it.

Sorry for the possibly dumb question, but doesn’t this just describe buying things in any financial market? Your offer will only be accepted in cases where nobody else is willing to pay more.

Re:Zillow just gave us a look at machine learning's future by 88913527 on Sunday November 07, 2021 @11:22AM (#29140748)
In a financial market (e.g: stocks) 1 share owned an individual investor is just as good as 1 share owned by an institutional investor: they have identical representation. The financialization of housing glosses over the fact the some houses will have foundational issues, need new roofs in a year, etc but you can't see that when you treat all houses as if they were shares, which seems like Zillow did, or by the nature of the asset class, sellers with low-quality product dumped for high prices.

Maybe Zillow went wrong because they deployed too much capital too quickly. With the opportunity to be selective, they could have found homes with 'good bones' that were under-priced. But if the executives mandated X billions of purchases in Y months, you move downstream to progressively less good deals, due to your own insatiable demand, when housing stock is already extremely low. That's not the AI/ML team's fault. That's you simply taking what you can get.

Re:Zillow just gave us a look at machine learning's future by DwnVoteHoneyPot on Sunday November 07, 2021 @11:23AM (#29140764)
Yeah, it’s not correct to call it a put option. A put option has a time component where you are guaranteed a contract price while the market price fluctuates.

Zillow is an example just bad bidding (due to bad valuation model)

Re:Zillow just gave us a look at machine learning's future by bradleyjg on Sunday November 07, 2021 @11:24AM (#29140769)
In a commodity market (all goods are fungible) pricing is easy to come by. For idiosyncratic goods it’s harder to get someone to make you an offer. The traditional way of doing price discovery in those cases was an auction. Zillow was willing to give a price upfront with no commitment, even in the sense of face time with a charismatic salesperson. That’s valuable for the seller and dangerous for the buyer.
Re:Zillow just gave us a look at machine learning's future by dragonwriter on Sunday November 07, 2021 @11:18AM (#29140708)
> Sorry for the possibly dumb question, but doesn’t this just describe buying things in any financial market? Your offer will only be accepted in cases where nobody else is willing to pay more.

This isn't a financial (securities and derivatives) market, nor is that behavior specific to financial markets. OTOH, in financial markets, you are dealing with masses of interchangeable items (e.g., every share of IBM is the same); whereas with real estate every individual item is different. In financial markets, you typically have (some) other participants order data for the same asset as a pricing signal.

It's a lot easier, when you are willing to give an offer on any item in the market, to be way off in real estate than financial markets without knowing it till you try to flip the assset.

Re:Zillow just gave us a look at machine learning's future by encoderer on Sunday November 07, 2021 @12:48PM (#29141670)
It wasn’t a free put, the problem here is that it was a put but it was not priced correctly.