Forecasting Folly

The combined resources of Statistics New Zealand, DairyNZ, Fonterra, and the Ministry of Primary Industries (MPI) have failed to predict or even acknowledge two huge market shifts in two years.

I recently downloaded 2012 to 2015 MPI Milk price forecasts from their annual ‘Situation and Outlook for Primary Industries Report’.

Each year these government agencies report three years of historic prices, estimate the current year and forecast four years ahead. Their sources include Statistics New Zealand, DairyNZ, Fonterra Co-operative Group and other government data.

In a table they look like this:

(Click to zoom)

If you look at these forecasts the current “crisis” in dairy prices shouldn’t be surprising.

In 2013 they predicted the price for milk solids would be $5.90. But in 2014 they revised that figure to $8.54. It ended up being $8.35 per kg. This was a 41% difference between forecast and actual over two years.

In 2014 the forecast was $7.20. However, in reality, we know the price will be much lower. The revised number earlier this year was $4.65 – 35% lower than the forecast.

If the 2013 forecast prices were 41% out by 2014, then why was the market so shocked in 2015? We had a 12 month warning that our forecasts were woefully unreliable.

In 2014 we celebrated a boom – and while we assumed the market shift would ease we thought it would maintain relatively high prices through the forecast period. Suddenly, the market shifted again and we panicked. It now seems observers have overestimated the recent fall in prices as they are revised upwards.

Forecasting and continually re-forecasting might make us feel in control our future, but it is not going to help.

The truth is that the market forces are complex and beyond our direct control (and apparently beyond the capability of our linear forecasting models). As we already know, these forces create a market that will at times be unpredictable, volatile and uncertain.

The solution to managing this market complexity isn’t more data or better models. Companies in the production, processing, marketing, logistics and services sectors are vulnerable to overreliance on forecasts for business planning. We need to get much better at operating in conditions of uncertainty.

Agility is a term often touted by organisations. But in reality the practice often relies on simply wrapping old ways of working with new terms or technology. What we need instead is a different way of working – a better mode of operation rather than better planning or analysis.

There are significant advantages to being agile in uncertain markets. But agility means we have to change the way we operate. We need to stop relying on analysis-based prediction and get better at weak signal detection. We need to move from fail-safe rigidity to safe-to-fail resilience.

This means formalising the things that we already do in our day-to-day practice. We should actually pay attention to anecdotes and weak signals of change. If our frontline staff or people at the industry coalface are hearing themes, then we should put those people in touch with decision-makers.

Too often we ignore this real-time information, either waiting for it to come up the decision chain, or waiting for periodic data releases (which we then use to forecast).

Our best forecasts are happening right now with the people working for us. We shouldn’t abandon our data collection, but rather enrich and augment it with the messages that these individuals are hearing.

We should break down organisational barriers to put decision-makers in touch with this market information. This will provide us with early warning, market context, and better forecasts than any retrospective data analysis.  

Steve McCrone is executive director of Cornwall Strategic