4. Review of Lessons learnt

In this very brief chapter I shall review the lessons I have learnt about forecasting.

4.1 More detail

By including more detail in the analysis of past data and in modelling of a system it is possible to gain a greater understanding of what has happened in the past. However this is usually at the expense of increased complexity when it comes to using such a model for making forecasts. The problem arises because each explanatory variable which is added to the model requires its own projection to make a forecast. Effectively this means that more detail in an analysis or model requires mire forecasts if it is to be used for forecasting purposes.

4.2 Data problems

There are at least four major problems with data>

It is possible to construct models of reality for which no data exists. An example of this is a multi-region input-output model for which the data is not available in the UK. The model can be build in theory but in practice it can not be calibrated. As well as data not being available, that which does exist is often incomplete as I found in the case of inland waterways (in the TIO study) and in gathering data on physical imports (in the freight transport study).
Data is not reliable due to the way it is collected.
Different sources of data on the same thing often give different estimates. For example in my work on the TIO study my estimates of output varied from those of another source used by the Central Statistical Office by a significant amount.
Data is difficult to obtain. When data does exist it is not always possible to get access to it. There are many reasons for this. One of the many ones being the conditions under which the data was collected. For example surveys of business conducted by the government are done under rules of confidentiality. One of these rules is that figures relating to less than 6 respondents may not be released. Another is that where a figure from one response represents 75% or more of a total, that total may not be released. These rules led to potential problems in the TIO study where the detail I required for my analysis required data below these confidentiality limits. It was only on giving an undertaking that the data would be aggregated before publication that I was given access to it.
In another instance in the TIO study I was given an unpublished report from which I could only quote total figures. In a draft report I included tables derived from this report to show how I had arrived at some figures, but I was asked to remove them.

In summary data is often incomplete, inaccurate, inconsistent and difficult to obtain.

4.3 Scenarios

Although the use of scenarios avoids some of the problems of using forecasts, such as being unable to say which of the possible sets of futures will be "the" future, scenarios have problems of their own. For example it is very easy for those using a scenario to mistake it for a forecast. It is also very easy to call a forecast a scenario but treat it as a forecast in all but name. I think that this is what happened with the ETSU scenario in Energy Paper 39 (Department of Energy 1979).

Another way in which problems can arise in using scenarios is when they are used for comparative purposes. As I found in the VRI study it may not be possible to construct particularly relevant and comparable scenarios.

4.4 Modelling of Behaviour

There appear to be no adequate models of behaviour in terms of forecasting behavioural patterns. One way of avoiding this is to assume that behaviour will remain constant. However this is unsatisfactory as I found in the VRI study.

4.5 Conclusion

These lessons led me to look in more detail at forecasting, what is it? and why is it done? In the second part of my thesis I will look at forecasting problems, why forecasting is required in the planning/policy making process, and the implications of the problems for planning.