Weighing the value of carbon price predictions

Power generators and major industrial emitters rely on carbon price forecasts to understand whether it is worth pursuing investments that could decarbonise their businesses.

If carbon forecasts prove to be too high then investors may pay too high a price, bringing forward investments when other opportunities could have made better returns on capital. On the flipside if carbon forecasts prove to be too low then investments may be delayed, and obligated emitters may find themselves paying a high price to meet future compliance needs.

Carbon price forecasts also influence the actions of individual investors. Bullish predictions may encourage investors to buy carbon futures in anticipation of higher prices. For example, in late 2021 a number of analysts were making very bullish calls as to how high carbon prices could go.

That begs the question: How accurate are carbon price forecasts?

Asset management group DWS sought to provide an answer by comparing analyst forecasts made at the start of each calendar year between 2010 and 2021 and comparing their predictions with the outturn at end of the year. According to DWS carbon price predictions looking out 12 months typically have an average absolute error rate of 35.5%.

DWS then compared carbon analyst prediction performance against gold, copper, corn, Brent crude and US natural gas forecasts. No surprise that high price volatility tends to increase the likelihood that the forecasting error rate is also high. That being said, carbon prices exhibit similar levels of volatility to US natural gas prices, yet carbon’s forecast error rate is almost double that of US natural gas.

In 2017 I published my second book Crude Forecasts: Predictions, Pundits & Profits in the Commodity Casino. After reviewing forecasts between mid-2007 and 2016 I found that the average 6-month consensus forecast for WTI crude oil had an error rate of 27%. Oil price forecasts looking twelve months out were only slightly worse, off by an average of 30%.¹

Show me the incentives and I’ll show you the outcome²

Incentive structures, operating at both the firm and the individual analyst level are often dismissed and discounted by ‘consumers’ of forecasts. In my book I argue that incentives are the true driver of forecasting performance. As Warren Buffet says, “Forecasts usually tell us more of the forecaster than the future.”

Incentive #1: Safety in numbers

As commodity prices soared late in the first decade of the 21st Century the investment plans of major commodity companies were increasingly based on the assumption that high prices would be sustained indefinitely. When prices have been high and rising for some time, it becomes an entrenched assumption that these high prices will persist for the foreseeable future.

In market conditions such as these there is an institutional inertia among forecasters. Analyst’s update their view of the world slowly and iteratively, not wanting to appear too far from the pack or consensus.

The exception to this appears to be when markets reach a peak or a trough. Then investment banks and commentators, etc, all want to come up with an even more extreme prediction of where prices could go - they seek safety in bullish or bearish sentiment. For example, towards the peak in the early 2000’s commodity super-cycle, forecasters came up with ever more bullish projections of how high prices could go. This was mirrored in early 2016 as forecasters sought safety in ever more bearish projections for crude oil prices.

Diversity of opinion often breaks down when beliefs (often also reflected in commodity prices) become too stretched. Seasoned investors might take a back seat while novices push prices to more extreme levels. When this happens, there is no countervailing force to cancel out the irrationality of one individual or group.

Incentive #2: Anti-herding

Researchers at the European University Viadrina Frankfurt (EUVF) analysed over 20,000 forecasts of nine different metal prices over different forecasting horizons, during the fifteen years between 1995 and 2011. Instead of finding the institutional inertia and forecasting herding that we might expect, they found strong evidence of “anti-herding”.

So why might some forecasters want to stray from the herd? It all comes down to incentives; and the incentive to herd or stay away depend upon the mix of clients, both existing and prospective. Think about who buys commodity forecasts. There are two groups of buyers. The first are those that buy forecasts regularly, perhaps as part of a subscription to a company’s analysis for example. In contrast to the regular buyer, there are also onetime or irregular consumers of commodity price predictions.

Examples of frequent buyers of commodity forecasts might include an oil company or a manufacturing company that regularly buys a certain small range of commodities. Given they are long-time consumers, they may have based their decision on how accurate a forecaster was over several forecasting periods.

In contrast, the second group of buyers is more likely to be swayed by the commodity forecaster that was most accurate in the past year or so or has been the most vocal about his or her success. This is rational from the irregular buyer’s point of view in that perhaps movements in the price of copper or another commodity only have a minor impact on their business, or maybe they only need to buy or take a view on a commodity infrequently. Either way, the cost/benefit of monitoring whether the commodity forecast they are buying has been accurate in the longer term is much higher than in the first group of buyers.

If the second group of buyers dominates (the infrequent consumer), forecasters have a strong incentive to differentiate their forecasts from the predictions of others by making extreme (or non-consensus) predictions. Even though an extreme forecast may have a small probability of being accurate, the expected payoff of such a forecast can be high, since the number of other forecasters making the same extreme prediction is likely to be small. Should they be successful in their prediction, then the forecaster can capture the attention and the budget of the infrequent consumer of forecasts.

In contrast, if a forecaster publishes a less extreme forecast, one close to the consensus forecast, then there is a high probability that other forecasters will make similar forecasts. If this is the case, then even if a forecaster’s price prediction is spot on the impact on his income and reputation will be minimal. The infrequent buyer will ask, “why pay for a forecast from an average forecaster?”

While a single or a string of successful predictions will bolster a forecaster’s reputation, this may result in future forecasts being much less extreme to protect their reputation. When a person or a company has their name on a forecast that may also alter the incentives; for example, a commodity research firm with a relatively low profile would be rational to make a wild forecast, drawing big attention from the media. In contrast, firms with a strong reputation are likely to make much more conservative forecasts, not wanting to stray too far away from the consensus.

Incentive #3: Career risk and analyst incentives

Herding behaviour isn’t specific to explaining how the forecasting firm appears to the outside world, it can also affect internal incentives. You might think that there is the temptation for an analyst to produce a bold prediction. If the analyst makes an “outlier” forecast that turns out to be spot on, it is likely to capture a lot of attention in the financial media, raising the prospect of the analyst being recruited by a rival firm, touting a bigger salary and an even bigger bonus. However, set against this is the risk of being fired (or at least having a few rungs taken from under the career progression of a young analyst) for a bad call.

To examine what the age and experience of the forecaster has on the degree of herding, research published in the RAND Journal of Economics examined over 8,000 forecasts by equity analysts between 1983 and 1996. Equity analysts should produce reliable forecasts of future earnings of the companies that they monitor, which are then used to produce recommendations on what their clients should buy. Equity analysts face their own quandary, having to balance the interests of the buy-side (i.e., their clients who prefer accurate forecasts) and those on the sell-side (other parts of the same bank they work for that might value trading commissions and large initial public offerings more than the accuracy of their analysts’ forecasts).

The researchers found that younger analysts tend to herd more than their more experienced colleagues do. Less experienced analysts, meanwhile, are more heavily punished for getting their forecasts wrong and so they have every incentive to stick with the herd. In contrast, older analysts, who have presumably built up their reputations, face less risk of termination. The researchers also found that, contrary to expectations, making bold and accurate predictions does not significantly improve a young analyst’s career prospects.

Incentive #4: Forecasts as marketing

Investment banks, other financial institutions and consultancies frequently publish their views on the future path of various commodity markets. There are two main issues that investors need to consider when verifying the objectivity behind these forecasts:

First, there’s a potential conflict of interest, i.e., the providers may benefit from those same clients acting upon their predictions. For example, a bullish commodity market outlook will benefit other parts of the bank with an interest in investors “buying in” to a growth story. This could involve a miner looking to an investment bank to secure more funding from the capital markets.

Second, ‘left-field’ forecasts are often used to garner media attention. Not necessarily for the accuracy of the prediction, but instead to market other services or products. As we’ve seen earlier, non-consensus forecasts, especially ones that are proved correct often mean that the analyst, and the firm they work for can dine out on that reputation for years, no matter the accuracy of their previous predictions.

Incentive #5: Skin-in-the-game and wishful thinking

Miners, commodity trading houses and other firms involved with the physical buying and selling of commodities also opine on the outlook for commodity markets. In the case of a mining company, these forecasts might be released around the same time as annual reports detailing the company’s activities are published, or when they are trying to raise funding for new investments. These forecasts can be said to have “skin-in-the-game”, with many commodity investors looking for clues as to how underlying physical demand and supply are likely to evolve. On the flip side, it’s difficult to argue that they are an unbiased prediction of commodity prices.

It’s not unreasonable to assume that a buyer (perhaps a large airline wanting to buy aluminium) or a seller (a miner/smelter of industrial metals) of commodities must have a much stronger incentive to have an accurate view of where a commodity market is heading. However, that’s not what the evidence finds. Wishful thinking is a powerful factor in adversely affecting forecasting ability.

The economist Guy Mayraz conducted a simple experiment at Oxford University’s Centre for Experimental Social Science to test how wishful thinking affected the accuracy of predictions. Mayraz ran sessions in which the participants were shown ninety days of historical wheat price data and were then asked to predict the price of wheat on the one-hundredth day. Besides being paid a bonus for accurate forecasts, half the experimental subjects were told that they were “bakers” who would profit if the price of wheat fell, and the other half were told they were “farmers” who would make money if the price of wheat rose.

Logically, in the study a “farmer” should make the same forecast as a “baker” since the forecast does not change the outcome, and both are paid for accuracy. However, that’s not what Mayraz found. Instead, nearly two-thirds of “farmers” predicted higher-than-average prices and nearly two-thirds of “bakers” predicted lower-than-average prices. Even when the scale of the bonus was increased, Mayraz found no significant increase in forecast accuracy. It seems that wishful thinking can get you into a lot of trouble.

Looking to the curve

What are the alternatives to model based approaches to predicting carbon prices? Well, many analysts (especially, but not exclusively those employed by governments) point to the forward curve to frame where prices could be in the future.

The forward curve shows the price at which it is possible to buy or sell contracts for a date in the future at a price agreed on today. Forecasters may view a rising futures curve (a market in contango) as a sign that the market expects higher prices, and vice versa for a downward sloping futures curve (backwardation). This is based on the perception that futures prices should incorporate all the available information to market participants, and that they also act as signals of what the “market” expects to happen.

In addition to examining analysts forecasting record, at the start of each year DWS also recorded the December contract price for the same calendar year. Across all commodities the forward curve produced a more accurate prediction than the average analyst forecast. In the case of carbon the average forecast error by analysts was 35.5% while the average carbon forward curve error was 32.9%.

Remember, there are several factors that affect the futures curve, not just market expectations of where the price of a commodity will be. First, the physical characteristic of the commodity – whether it is easy to store and whether there are ample inventories, etc. Second, longer dated contracts are illiquid, raising doubts of whether they are an effective aggregator of information. Third, the futures curve fails to account for real interest rates.

The fourth and final factor is that futures prices will always be discounted to the market’s expected future spot price in order to give investors a “risk premium” to take on risk from producer hedgers (known as the “normal backwardation” theory). This means that even if the spot price is estimated correctly, the traded futures price will tend to understate the market’s current real price expectations.

Menzie Chinn and Olivier Coibion, writing in the Journal of Futures Markets, looked at the overall efficiency and predictive ability of commodity futures markets. Using data from 1990, the researchers found that overall, energy and agricultural futures prices have much better predictive capability than precious and base metals.

Within the energy sector, Chinn and Coibion’s research found that crude futures markets do not predict subsequent price changes as well as other energy futures markets, such as natural gas and gasoline. There is a similar situation in agricultural markets, with corn and soybean futures markets found to have a much better predictive power than wheat.

As a whole, however, the overall predictive ability of commodity futures curves appears to have declined since the mid-2000s. Chinn and Coibion believe this may have something to do with the increased financialisation of commodity markets, which has increased the degree of co-movement in futures prices.

Despite it’s obvious flaws, one reason it is favoured by governments is that the forward curve is a simple and transparent market-based measure, and so it is often preferable to more opaque model-based forecasts. Given the forward curve is nearly as bad as analysts forecasts you can hardly blame them.

What is the downside risk?

If forecasts are awry as much as 30% over only twelve months, how can investors have much confidence in making decisions based on much longer term forecasts? Afterall, it may be several years between the decision to invest in upgrading existing or new infrastructure and it actually coming onstream.

Rather than try to predict the absolute price level, what investors really want to know is the downside risk, i.e., what is the likelihood that carbon prices drop to very low levels, or move higher and stay at extremely elevated levels?

It’s only by understanding the payoffs priced into the market that investors and other market participants can gauge the degree to which they are exposed. In contrast, commodity price forecasts - including those of the carbon market - only deliver a faux degree of certainty.

And so as we approach that time of year when analysts begin to publish their traditional “What to expect in next year” reports, bear in mind that carbon price forecasts are just as flawed as any other.

If you are interested in finding out more about forecasting commodity prices then please check out my book, Crude Forecasts: Predictions, Pundits & Profits in the Commodity Casino.

1. I reviewed the accuracy of WTI crude oil price forecasts made both six and 12 months prior to June and December each year using data published by The Wall Street Journal (WSJ).

2. Charlie Munger is oft quoted as saying, “Show me the incentives and I’ll show you the outcome.”