Michael Baker - Thesis - Problems in Longterm Forecasting and Planning
About | Contents
Previous: 6. Criticisms of Forecasting | This: 7. Revised View of Forecasting | Next: 8. Planning
In this chapter I shall review some of the lessons which I have learnt from the case studies. The conventional view about forecasting [1] is that it is an "objective" process. However there are so many subjective inputs to the forecasting process that there is no way in which it or its results can be "objective" I shall talk about this in sections 7.2 and 7.3 below on input data and models respectively. Then I shall give some fundamental reasons why "objective" forecasts cannot be made, in section 7.4.
Referring to the description of the modelling process used in forecasting given in Chapter 5, there are three types of inputs to forecasting models. These are assumptions about relationships, calibration data and projected values for exogenous inputs. There are three sources for the projected exogenous inputs, which are extrapolation of past trends, targets and other forecasts. However as I explained in Chapter 5 only the first two are independent inputs to the totality of the forecasting process. They are dealt with below. I shall deal with the construction of forecasting models in the next section.
At some point in the forecast modelling process a stage is reached at which no further causal explanations for the variables in the model are looked for. This can be for one of several reasons including the judgement that the underlying causes will not change over the forecast period or complete ignorance of the underlying causes and the assumption that they will not change.
In either of these cases the judgement of the forecaster is involved. Of its nature any such judgement must be subjective. There are two possible ways of making a trend projection. In the first some characteristic of the variable is assumed to remain constant such as its level or its first derivative (linear growth) or its growth rate (exponential growth). The assumption of constancy on the part of the forecaster involves his value judgement. The second way of making a trend projection is for the forecaster to use his judgement that such a characteristic will change over time. In this case his judgement is more directly evident. In either case the judgement of the forecaster is involved.
The judgement that the underlying causes will not change over the forecast period often goes with the belief that the underlying trend is desirable. If it were not believed to be desirable it is likely that the forecaster would find some mechanism whereby it might change.
By their nature targets are subjective. they are not "what is likely to happen" but what the forecaster or forecaster's client wishes to happen. Targets can either be judged to be realistic, that is that they will be attained, or they are part of a conditional forecast. Conditional forecasts are answers to one or other of the questions: "what happens if the target is met?" or "what is necessary for the target to be met?"
As shown in Chapter 5 the only independent inputs of exogenous variables to the forecasting process are trend projections and targets. As explained above both of these inputs are subjective. In consequence the output of the forecasting model will also be subjective. It is no more than the consequence of the subjective inputs. However I shall also show that the models are themselves also the result of subjective judgement of the forecaster.
As explained in Chapter 5 a forecasting model should cover all "relevant" variables. However the choice of which variables are relevant is made by the model builder. His choice of which variables to include (what is to be within the system under study) and what is to be excluded is arbitrary.
For non-causal models the choice of relationship form is made on the judgement of the model builder. Of more importance is his judgement, or even act of faith, that the relationship found in the historic data will continue in the future. This judgement requires the assumption that the underlying causes of the relationship will remain in the future or that if there are several causes for the relationship that a change in direction of one cause will be offset by a change in the opposite direction by others.
In causal models there is a similar judgement that the causal relationships modelled will remain constant over the time period of the forecast. In summary the models used in forecasting are arbitrary and subjective and consequently so are their outputs.
The historic data upon which forecasting models are based (which is used for calibration) is poor. The outputs of these models will be of no greater reliability and in consequence will also be poor.
At one level criticisms can be made of individual forecasts and forecasting models as was done in Chapter 6. It is at this level that the acknowledgement can be made that there are very great problems with forecasts which is often then followed by the statement "but we have to do something." At this rather superficial level it will never be possible to ensure that all avenues of enquiry into how forecasts might be improved, have been explored.
There is however a deeper level on which it can be shown that there are fundamental problems which can not be overcome by improved technique and which make objective forecasts impossible. These problems can be summarised as:
The reason models are used in forecasts is that it is believed that the past will not be like the future so the use of "simple forecasts" (ie future value of variable will equal current value) cannot be used. The models depend upon the Ceteris Paribus condition (that everything else will remain equal). However it is just because it is believed that things will change, that the models are used.
Figure 7.1 illustrates a system which is under study.
Figure 7.1 A system which is under study
If a forecast is to be made of a component of the system, the system boundary will probably have been drawn so that there is a minimum of interaction between the component of the system and the environment. However I shall redefine the system boundary to be conterminous with that of the model. Consequently everything which is modelled is within the system and everything which is not modelled is outside the system.
Both open and closed systems [2] can be modelled. However when open systems are modelled all interactions with the systems environment must either be known or assumed to remain constant. It is the later of these two which is known as the Ceteris Paribus condition (everything else remaining constant).
To me it is paradoxical that the two assumptions:
are made whenever a forecasting model is used. The justification for doing this is that the forecaster is of the opinion that he has included all relevant things which will change in his model and that anything which changes outside his model will have no significant effect upon it. However this is dependent solely upon the model builders judgement and can not be based upon any objective criteria.
The way in which hypotheses are used is that they are deemed to be true until evidence emerges that they are not true. The process of searching for such evidence can be considered to be testing a hypothesis. However such a search can not be guaranteed to be complete until such time as evidence for the falsity of the hypothesis is found. Ultimately the most that can be said of a hypothesis (or theory) is that "it is not known to be untrue", which is very different from the statement "it is known to be true".
For a hypothesis to be falsifiable the conditions under which it holds must be repeatable. In general it is only in closed systems that conditions can be repeated since, by their nature, open systems can not be isolated from everything else which is constantly changing. The consequence of this is that hypotheses about open systems are in general unfalsifiable.
The result of this is that when facts emerge about an open system which do not agree with a hypothesis it is possible to find an explanation. This can be something which was not considered, either within the system, or within its environment.
The effect of this is that any examination of the success or failure of past forecasting exercises, "reasons" can always be found for the failures.
One of the elements of any system being forecast within the policy formulation field is people. Unlike inanimate objects important properties of people change in unpredictable ways. These properties which change can be called "values".
People are involved in the system being forecast and they act according to their value systems. However they can change their value systems in ways which can not be modelled. Consequently there are important aspects of any system being forecast which cannot be modelled. Basically because people are involved there are relationships within any forecasting model whose future can not be predicted.
In this chapter I have outlined my revised view of forecasting which is basically that objective forecasts are not possible and that there are severe problems in the whole of the forecasting process.
Even if forecasting models were objective representations of reality they would be no more than opinion transformation devices. The outputs of forecasting models can be no more than the consequences of the input assumptions. Given that the inputs to forecasting models are the forecaster's subjective opinion of what will, or might, happen in the future the output of the models will merely be a transformation of these subjective opinions. There is no way in which a subjective input could be converted to an objective output and there is also no way in which the inputs to forecasting models in the policy field can be objective.
The fallacious view that forecasts are objective is maintained by the use of highly complex mathematics through which many observers of the forecasts output can not see to the underlying inputs. However just as in computing, the expression:
equally applies in forecasting.
[1] That is the view of those who make and use forecasts within the policy process.
[2] In this discussion I shall not consider closed systems because none (or very few) occur in the forecasting / planning field in which I am interested.
About | Contents
Previous: 6. Criticisms of Forecasting | This: 7. Revised View of Forecasting | Next: 8. Planning
Copyright © Michael Baker 1981,2005. All Rights Reserved.