Online property valuation models: How accurate are they?

As might have been anticipated, my recent article providing a guide to the current portfolio of providers of online property valuations models triggered the inevitable question – "just how accurate are they?"

So I thought I would do some desk research. However, before I unleash a barrage of criticism stating that there are heaps of examples where the automated valuation models (AVMs) are so wide of the mark to make them laughable, let me simply say this. There are more than 1.5 million AVMs or potential AVMs for New Zealand properties – there will always be outliers and extremes. I do not have the time or patience to review thousands of properties or even hundreds of properties. I chose to select just 12 properties.

The method I used was to track the latest published auction results as the auction year re-started after Christmas. I simply took the first 12 I saw, which comprised eight properties in Auckland and four in Tauranga. So again I acknowledge that my sample is hardly representative nor truly random. It is made up of auction sales only. The sales are only for those two areas of the country and represented a very quiet period of the year.

With these 12 property sales results I went to each of the five providers:

• Homes.co.nz
• Trade Me Property Insights
• Realestate.co.nz Property Profile
• MyValocity (you do need to create an account to use the AVM)
• QV – mobile app for iOS and Android

I knew none of these providers had updated their valuations to take account of any of these actual 12 sales. Neither would the sale records have been picked up through local council sales or agent reporting, so there was no bias of an AVM being influenced by these recent sales.

Another point to note is the analysis compared the sale price at auction to the mid-point of the price range of the AVM.

So here is the table of results. The colour code used is blue where the AVM equalled the sale price exactly, red signifies an AVM below the sale price with green where the AVM is above the sale price. Finally, grey indicates that the provider had no AVM for the property.

As you can see, the visual skew towards red indicates that, based on this sample set, most AVM’s were below sale price. (The original version of this article I used an average variance measure, after receiving valuable feedback I have now used the calculation of gross median error.)

All providers achieved a gross median error of less than 10%, with Realestate.co.nz achieving less than 5%, which is impressive. I would deduce that a factor in its accuracy, is it benefits from the very latest REINZ data each month of unconditional sales, while all other providers rely largely on settled sales, which come through at least a month to two months later.

Another perspective I was keen to examine in respect of the accuracy of AVMs was the indicative range they provide to reflect the level of confidence. For each provider, for each property I assessed the range as a percentage of the midpoint price.

This analysis is very illuminating. The provider with the tightest range (in theory indicating confidence factor) is MyValocity, closely followed by Homes, both just under 10%. This effectively means that their AVM range is 5% below the midpoint to 5% above which I would judge as fairly acceptable given this is a computer-based estimation with no detailed knowledge of the specifics of the property.

Of interest in this analysis is the wide margin in the range from Trade Me Property at close on 30% with its tightest range being for a single property at just 19%. Similarly, Realestate.co.nz seem to apply a standard circa 21% to all AVM’s.

Alistair Helm is the former chief executive of Realestate.co.nz and former head of product at Trade Me Property. He has just restarted his independent Properazzi blog after a three-year break.