On the Use of Forecasts
Address to the Australian Business Economists
Sydney - 24 November 2011
Thank you for the invitation to return to this platform.
This being a forecasting conference, you will have spent much of your time contemplating the outlook for 2012. The Reserve Bank set out its views on the outlook only a few weeks ago, and I will not canvass any changes to them on this occasion. Instead, I propose to return to a theme I have covered on some previous occasions, namely, the nature and use of forecasts for policy purposes. A couple of important points that have been illustrated over the past two or three years are worth drawing out.
To begin, I would like to draw some observations from another kind of forecast. I do a bit of aviation in my spare time. Hence I am a serious user of weather forecasts and there are very detailed forecasts prepared on a frequent basis for the aviation community, in order to make flying more predictable and safer.
I want to give an example. A couple of months ago, a pilot I know had planned to fly in a light aircraft from Bankstown airport in Sydney's west to Armidale, about 90 minutes flight time to the north, pick up some people and return to Bankstown.
On the relevant day (25 September), pertinent excerpts from the forecasts for the two aerodromes were as follows:
15015G25KT 9999 LIGHT SHOWERS OF RAIN FEW 012 SCT020 BKN030
TEMPO 2500/2506 3000 SHOWERS OF RAIN BKN008
16012 9999 LIGHT SHOWERS OF RAIN SCT030
INTER 2501/2508 4000 SHOWERS OF RAIN BKN010
PROB30 INTER VRB20G30KT 3000 THUNDERSTORMS WITH RAIN SCT 045CB
It was windy and wet in Sydney that day, one of those days when strong south easterly winds bring in moisture from the Tasman Sea and dump it on the Sydney basin. The Bankstown forecast indicated that for much of the day, there were expected to be periods of reduced visibility and heavy, low cloud. Conditions could thus be quite marginal for a landing on the return flight, and it was possible they would be below the legal minimum for a landing off an instrument approach during those periods of time. This meant a requirement to have extra fuel on board in case of the need to hold prior to landing, waiting for the weather to improve. The Armidale forecast showed some similar periods of weather, also requiring extra fuel. More significantly, the Armidale forecast indicated that there was a probability (assessed as 30 per cent) of thunderstorms in the area, which could persist for up to half an hour at a time. Thunderstorms at the airport amount to very dangerous conditions in which to attempt a landing because of the potential for very strong winds and windshear near the ground, not to mention heavy rain or hail. Even large aircraft avoid landing in such circumstances. Again, this meant a requirement to have additional fuel in case of the need to hold prior to landing, waiting for the storm to pass.
Apart from that, the general conditions between the two airports were forecast to include isolated thunderstorms, rain and areas of low cloud, all produced by the south easterly airstream operating across much of NSW. The route goes more or less along the Great Divide, which means that terrain effects on weather conditions are an issue to keep in mind.
These conditions did not necessarily preclude the flight, which could have been legally commenced, provided the requisite additional fuel was carried. It is very likely that it could have been safely completed. For professional pilots, who fly every day in a multi person crew environment in high performance aircraft, dealing with these conditions would be seen as reasonably routine, if a bit tedious. The main question was whether it was prudent for an amateur single pilot flying a light aircraft to conduct the flight on this occasion. It was observable that at the intended time of departure from Bankstown, conditions there were at least as bad as forecast, while a phone call from Armidale indicated that there were in fact storms present at that time. It did not look like a day on which the forecasters had been too pessimistic. The pilot in question decided to stay on the ground.
The point of this little diversion into aviation is to make a few observations about the nature and use of forecasts, which I think have some relevance in the economic sphere.
The first is that the weather forecasters had understood what I would call the ‘big forces’. In this case, there was a high pressure system over Tasmania, and a low pressure system off the north-east coast of NSW. This combination fed very moist air over the south-east of the continent, resulting in cloud and rain.
In fact, meteorologists know a lot about how weather works. They have pretty long time series of observations and increasingly frequent real time observations of conditions. They have highly developed models. The combination of real time data, understanding of how the dynamics of weather occur and their experience, enabled the forecasters to get the big picture right, and give a very useful forecast for those planning on venturing into the skies that day.
The second point of note is that some elements of the forecast were probabilistic in nature. This was explicit in the use of the term ‘PROB30’. Weather forecasters know they are dealing with a very complex, non linear system, and are careful to present their forecasts accordingly. They are able to observe the unstable atmospheric conditions that are conducive to storms – mainly heat and moisture, with a role played by terrain as well. They cannot say for sure that there will be storms over a particular location, but they know enough about likely conditions in an area to assess a probability. On the day in question, there would almost certainly have been some storms to avoid somewhere along the route.
The third observation is that forecasts are used in particular ways. In aviation, lives can depend on the way a forecast is used. Professional pilots in large, well equipped aircraft that fly above most of the weather still carefully study forecasts and make the requisite amendments to their plans. They carry additional fuel, have a plan B for an alternate airport and so on as needed, in response to the forecast conditions. Many of us have had the experience of fog induced delays in Canberra in winter, for example, where the runway is not fully visible at the legal minimum on approach and so the aircraft cannot land and must go somewhere else. International flights into Sydney very occasionally end up in Brisbane or Melbourne because of fog, having been required by the forecast to carry the necessary additional fuel. They may have carried it all the way across the Pacific, at non-trivial cost.
Despite the criticism aimed at weather forecasters, the forecasts I have seen in use for aviation are generally pretty good. And the saying that economic forecasters are there to make weather forecasters look good has something going for it. Of course one big difference in economics is that some decisions based on forecasts may alter the outcomes – as in the case of economic policy decisions, or spending decisions by businesses and households – whereas our response to a weather forecast will not actually alter the weather. That factor makes economic forecasting more difficult than weather forecasting. Still, some aspects of the process of weather forecasting are valuable in the economic sphere.
One is that the most useful economic forecasts, like weather forecasts, are those that are based on a good sense of the ‘big forces’, as well as on an understanding of the dynamics of how economies typically behave. In addition, we should admit that economic forecasts have a margin of error – they are a point in a distribution of possible outcomes.
On the latter point, often much is made about small changes to forecasts, or small differences in two forecasters' numbers. But when consideration is given to the real margin for error around central forecasts, such differences are often, for practical purposes, insignificant. For example, in the case of a year ended forecast for growth of real GDP four quarters ahead, experience over the past couple of decades is that the probability of a point forecast being accurate to within half a percentage point is about one in five. For year average forecasts the accuracy is better, but still the margins for error are non trivial. So any point forecast will very likely not be right. The likelihood of some outcome other than the central forecast is actually quite high. When comparing forecasts, if we are not talking about differences of at least half a percentage point, the argument is not worth having.
In any event, the question is not really whether the forecasts will turn out to be exactly right. The question is whether they form a reasonable basis for sensible analysis or decisions at the time. When the forecast turns out to be not exactly correct, as is very likely, that is actually not much of a basis on which to criticise the decision makers who used the forecast (or, for that matter, the forecaster).
For monetary policy operating a medium term inflation target, we are naturally interested in our ability to forecast inflation. Experience over the inflation targeting era (since 1993) suggests that the probability of the CPI outcome being within half a percentage point of the central forecast is roughly two in five at either a one year or a two year horizon. For underlying inflation, the probability of the forecast being within half a percentage point is about two in three at one year and just over one in two at two years. The smaller forecasting errors for underlying inflation reflect the inherently more stable properties of the underlying measure, which of course is by design.
Hence, if the central forecast for CPI inflation at a two year horizon was 2½ per cent, the chances of the outcome being between 2 and 3 per cent, based on this historical experience, would be about two in five. The chances of being between 1½ and 3½ per cent would be three in five. I note in passing that, if this is a reasonable description of forecast accuracy, it suggests that the configuration of the inflation target is a pretty good one (though I hasten to add that, when it was first set out, we did not really have a great deal of confidence in the accuracy of inflation forecasts).
It would, in my judgement, be vastly preferable for discussions of forecasts to be couched in more probabilistic language than tends to be the case in practice, and for there to be more explicit recognition that the particular numbers quoted are conditional on various assumptions. Careful observers will have noted that the latest forecasts published by the Bank actually have a range for growth and inflation at the horizon. Moreover, there is more extensive discussion these days of the ways in which things could turn out differently from the central forecast. This goes at least some way to recognising the inherent uncertainties in the forecasting process, and is also important in relating the forecast to the policy decision.
Taking account of the accuracy statistics I have quoted above, we can characterise the RBA's latest published outlook as suggesting that, absent large shocks to oil prices or the Australian dollar, or further extreme weather events, or the world economy taking a serious turn for the worse (say because of events in Europe), Australia's inflation rate in 2012 has a pretty good chance of being between 2 and 3 per cent. The chances of a similar outcome in 2013 are also reasonable, though with slightly greater probability that inflation would end up above 3 per cent in that year than seems the case for 2012. That is, the point forecast is a little higher in the second year. Even so, a margin of uncertainty is inevitable. A big change in any of the variables subject to assumptions would quite easily push outcomes away, and maybe a long way away, from the forecast.
This degree of uncertainty can of course be quite disconcerting. It is only natural to desire certainty. Everyone wants to know what will happen. We all want to believe that someone, somewhere, does know and can tell us what to expect. But the truth is that the best we can do when talking about the future is to speak about likelihoods and possible alternative outcomes.
This is not a counsel of despair. It is not as though we can say absolutely nothing about likely performance. We know something about average rates of growth through time, and we know something about the long run forces that work to produce them (productivity and population growth). We know that there have been, and will be again, periods of recession and recovery, though our ability to forecast the timing of those episodes is limited. We know from experience some things about the nature of inflation, including its characteristic persistence, and the things that can push it up or down. We know some of the ‘big forces’ at work on the global and local outlooks – a once in a century terms of trade event, for example, and a once in a century deleveraging event in major countries. We know that our country is exposed to both forces – the expansionary effects of the rise in the terms of trade, and the dampening effects of a mild degree of deleveraging in our household sector (and indirect effects of the more intense deleveraging in some other countries). We also know that the terms of trade change is a large shift in relative prices, which will bring about changes to economic structure.
So there is a good deal we can say about the things that are relevant to our future, and economists' understanding about these forces will be helpful in making sense of what occurs over time. We simply have to recognise the limits on our capacity to predict their net impact with any precision.
This in turn has implications for the way policymakers use numerical forecasts. In the case of monetary policy, forming a forecast is unavoidably part of the process, simply because the evidence suggests that monetary policy changes take time to have their full effect. So we have to use forecasts – but not unquestioningly. We have to form a view about the big forces at work, but also operate with due recognition of the limitations of numerical forecasts. The extent to which policy should respond to forecasts will therefore always have some element of judgement.
The conduct of policy over the past few years has exhibited these features. The Bank's assessment of the very broad major forces at work has been central. Policy was tight in a period in which the economy was very fully employed, confidence was high, the terms of trade were rising and inflation was picking up. The very large and rapid easing of monetary policy late in 2008 and early 2009 was a response to a major change in the outlook, which occurred because the ‘big forces’ changed direction very quickly, due to the financial events at that time. Among other things, this saw strong growth in Asia go into sharp retreat, appetite for risk and willingness to lend sharply curtailed and confidence slump. The changes to monetary policy beginning in the latter months of 2009, designed to restore ‘normal settings’, occurred when it had become clear that the risk of a major economic contraction in Australia had passed. In fact the ‘big forces’ in the expansionary direction had reasserted themselves, after an unexpectedly short absence: resurgent Asian growth helped to push the terms of trade to new highs. In that world, leaving interest rates at 50-year lows would have been imprudent.
Over the past 18 months or so, policy changes have been much less frequent, but the process of decision making has nonetheless not been dull. A year ago, the then current data on inflation showed nothing particularly alarming. The analysis of the Bank's staff suggested, however, that the fall in inflation we had been seeing since 2008 would probably not continue, but instead inflation would probably begin to rise, albeit quite gradually. The Board took the view that, on the basis of that outlook and in the circumstances prevailing, it would be prudent for policy to exert some mild restraint, and so it decided late last year to raise the cash rate by 25 basis points.
The inflation data for the first half of 2011 do indeed show some increase in underlying inflation (though after a sequence of revisions, this is not quite as large as it looked a few months ago). As of May this year, the central forecast was for inflation to pick up further over the ensuing couple of years, eventually rising to be clearly in excess of 3 per cent. This carried a simple message. As the Bank said in the May Statement on Monetary Policy:
‘The central outlook sketched above suggests that further tightening of monetary policy is likely to be required at some point for inflation to remain consistent with the 2–3 per cent medium-term target.'
But there was still a matter of judging how to respond to that message. The Board did not tighten policy at that time, nor did it do so three months later when the forecasts for inflation still looked similar to those in May. It certainly considered whether that course of action would be appropriate, but elected to sit still, watching unfolding events. Eventually, last month, far from tightening, the Board actually eased policy slightly – though by then, of course, the forecasts had changed materially from those of six months earlier.
This was not a repudiation of the forecasts, nor a sign that forecasts are not useful. The process of forming forecasts remains key to the forward-looking conduct of policy. In electing to take time in considering their response to signs of an increase in inflation, and central forecasts of further increases, the policymakers were simply recognising the inherent uncertainties of the situation and the difficulties the forecasters face, and giving those factors due consideration.
It is, in my view, entirely appropriate that there be this degree of limited discretion for the policymakers in their response to changes in numerical central estimates. It is not that forecasts should be ignored. But neither should the decision be rigidly and mechanically linked to forecasts. Were that to be so, the policymakers would in effect have delegated the policy decision to the forecasters, which is not what policymakers are supposed to do.
So the relationship between the formal forecasts and the policy decision can sometimes be a subtle one. Ultimately, the policymakers have to make a judgement call, based partly on what the central forecast says but conditioned also by the degree of confidence they have in it. At the same time it must be emphasised that policymakers can be in a position to make the sorts of judgements I have just been describing only if they have generally acted in a timely, forward-looking way in earlier decisions.
In my view, these judgements over the past year were the right calls. But in truth we will not know for a while – such are the lags in monetary policy. As always, more data will help the process of evaluation, though they might also provide evidence of new shocks (that is, things the forecasters could not predict). Such is the nature of the forecasting game.
In conclusion, to those of you here who do not have to make forecasts, I hope you realise how fortunate you are! To those who do, I offer my sympathy – and best wishes for clear vision over the year ahead.