determining aggregated expectations in economics

The paper describes a package of new research techniques, which have been developed to allow investigation of long-term global trends. It is based on the theory that expectations (of managers) intimately effect the macro-outcomes of aggregated individual actions. A knowledge of these expectations may be used as a guide to their future actions, and hence to future outcomes overall. To allow this knowledge to be obtained, the qualitative techniques - a combination of focus groups with scenario forecasting - were developed as part of a programme of research lasting more than half a decade; and tested against 17 groups containing managers from 140 organisations. Subsequent quantitative work - using semantic differentials to map the importance of the 162 dimensions discovered at the qualitative stage - has initially been tested by a survey with respondents from more than 150 organisations.

The author is a senior lecturer in the Centre for Strategy and Policy, at the Open University Business School, of which he was the first Head. In addition to being the author of a number of books, including the MBA text-book 'Marketing' (in its second edition in Europe, and a newly-launched first edition in the US launched), he is in charge of the university’s Millennium Project- which is also the British focus of the UNUAC project - and has been advisor on long-range planning to a number of organisations and governments.

In the context of the theory described elsewhere in this issue, this article describes a viable set of techniques which now allows such managers to make more meaningful measurements of the relevant trends, even in an otherwise uncertain macro-environment, and hence offers, once more, a real possibility of intervention. The techniques were developed as part of the Open University’s (OU) ‘Millennium Project’, working with more than a thousand large organisations and government departments. The ‘Millennium Project’[1], overall, attempts to improve the accuracy of long-range forecasting using new research techniques [1,2,3] derived from a combination of scenario-planning and focus-groups[2].

The origin of this sophisticated package of techniques for larger organisations (including governments) lies, paradoxically, in our work in developing simpler techniques for smaller organisations. Thus, much of the formal work on ‘mapping’ the long-term future has now become so complex that in recent years it has been effectively restricted to the specialist staff groups in larger think-tanks, and a very few multinationals; using, for example, Delphi techniques[3] [4] or cross-impact matrices [5]. Even then, the results they have obtained (from experts) have often been challenged as being too inaccurate for the purpose of serious long-range planning [6] ; and have usually been unsuitable for tracking changes in their macro-environment. In any case, in the context of this earlier aspect of our development work, such techniques were not practical for smaller organisations or departments within larger ones. The work most widely reported in the media, on the other hand, has been that of individual futurists, usually reporting their own subjective viewpoints. Perhaps most (politically) influential work in this context has been that of the Tofflers [7].

None of these approaches provided the practical support for the long-range planning undertaken by most (smaller) organisations. Accordingly, as part of our development of the teaching material involved in the MBA, we were forced to develop new techniques - such as 'simpler scenarios' [8, 9] - as integral elements of the strategic planning process.

As part of this overall development work, we have found an increasing need to develop more specific research techniques for identifying the key forces for change emerging in the external environment; techniques which are - accordingly - of interest to the larger organisations. As one result, we have been attempting to develop new, formal research techniques which may be used for this purpose [10]. This paper reports our progress to date - with some new techniques which appear to successfully produce both qualitative and quantitative results in terms of identifying macro-environmental (global) trends.

The (global) research described in this paper is the latest stage of a programme aimed at improving our understanding of long-term global trend. It has so far lasted more than half a decade. The previous stages have been, in chronological order:

1. Depth Interviews - the first element - intended to sketch out how knowledge of the wider environment, over the longer term, was incorporated into strategy - comprised depth interviews with 50 senior executives from multinationals and government departments. This form of research is commonly used as the basis, sometimes the only basis, for published work on the future. We took it as our starting point.

2. Computer Conferences - based on the views of these executives, a series of computer conferences - under the general umbrella of the 'Millennium Project - were run; in which 400 managers, members of The Strategic Planning Society and Demos as well as OUBS students, debated both the techniques involved in long-range planning and especially the management issues which were going to be important over the longer term. Although the scale of this research was much larger[11], it was still exploratory in nature; though it did indicate some interesting deviations from accepted ‘facts’, which stimulated further research.[4]

3. Industry Scenarios - to obtain a better picture of these issues, 200 OUBS students completed detailed scenarios of the future of their own industry. These were then content analysed as the basis for statistical analysis. This was originally intended to represent our major research effort; offering definitive sector predictions: which could then be aggregated into a global view. In reality, the resulting ‘industry view’ proved to be limited - with ‘short-termism’ dominating the picture, and considerable emphasis on regulatory considerations. This picture, of organisations in effect sub-contracting their long-term future to government, was interesting in its own right[5] but it did not offer the global perspective we were seeking [12].

These various pieces of research were intended to provide results in their own right. On the other hand, their main benefit - not least in terms of demonstrating their shortcomings as global tools - proved to be as input to our progressive development of research techniques for investigating longer term global futures. This paper reports the techniques we now use to successfully address these.

Although the specific techniques involved 'emerged' from previous stages, the key underlying assumptions behind the latest stages of the research are derived from the hypothesis of ‘Aggregated Expectations’, described elsewhere in this issue. Crucially, in the context of this paper, the most important implication of this theory is to be seen in its emphasis on observation and measurement of these expectations:

“observation/measurement of existing expectations - thus, the starting point must be an understanding of what the population's existing expectations are; and, hence, what will happen if no intervention occurs. Without knowing where you are, and in what direction you are currently going, it is impossible to steer a course to the destination you want.”

Overall, then, the research techniques, described here, are intended to observe these expectations and, hence, to more accurately predict global futures.

One key research element, reported here as the final stage of our qualitative research on global futures, was the framework derived from the techniques we have developed for simpler scenario forecasting. This originated with the work of Shell Oil [14 ,15, 16], and indeed the initial stages of our own development work were conducted in conjunction with management from that organisation. In the context of our 'simpler scenarios', as initially developed for use by smaller organisations [17, 18], this process comprises five main steps - all to be followed sequentially by the managers wishing to investigate the future of their organisations:

In the new form of qualitative global research, specifically reported here, only the first two steps are undertaken exclusively by our participants and the last two are undertaken exclusively by ourselves. In the case of step 3 there is an overlap. Following this framework:

The first, crucial stage of our usual (simpler) scenario forecasting process is to examine the results of the prior environmental analysis - such as that described by Aguilar [19] - to determine which are the most important factors that will decide the nature of the future environment within which the organisation operates. In the case of the global work described here, however, there is no prior analysis, since all the members of the groups come fresh to the work. On the other hand, they all will have been exposed to a wide range of analytical inputs from the media; and this has proved, in our tests, to be quite sufficient for them to productively engage in the debate - and for them to produce meaningful results.

In any case, perhaps the most difficult aspect of any scenario planning is freeing the participants from the preconceptions they take into the process with them. Due to the disparate backgrounds of the members of our global futures groups, we have found that this is not a problem. Equally, the requirement that they look 30 years into the future creates few objections - and poses no significant problems in practice.

The specific technique we have developed for implementing the first stage of scenario forecasting, in general, is based upon:

Once more, the simple technique we use for the first stage of our scenario work is one we have previously developed for general usage by all our focus groups, not just those which are involved in our (global) scenario work. It is based on the now almost universal availability of Post-It Notes and is a very simple technique; since it requires only a conference room with a bare wall and copious supplies of 3M Post-It-Notes.

In this way, the six to eight managers/professionals[6] taking part in each group - in much the same way as in a normal focus group [20]) - meet in a conference room, isolated from outside interruptions. At the start of the session, in order to avoid undue bias, the participants are deliberately given a very short briefing, with the bare minimum of information. Even this takes place whilst they are standing - the first major difference from traditional focus groups - in front of the bare wall they are going to use; chairs are deliberately not provided. In essence, apart from introducing the basic principles of using Post-It-Notes as the vehicle for discussion (and record), only the basic question - which is the starting point, is introduced. In the case of our global scenario work, with 17 such groups (containing 140 participants, in total, from a wide range of organisations) we said little more than "We want you to describe the future of the world in thirty years time." In practice - despite the initial surprise of the participants - this proved quite sufficient for them to then engage in free-form debate. Most groups are producing a flow of relevant ideas within five minutes and - in our experience - none take more than ten minutes to become fully involved in the debate.

The issues they then identify, the 'drivers' for global change in the case of our own scenario groups, are written, with a thick magic marker so they can be read from a distance, on separate Post-It Notes. These Post-It Notes are, at least in theory, randomly placed on the wall. In practice, we have found that even at this early stage participants want to cluster them in groups - another difference - which seem to make sense. The only requirement - which is why Post-It Notes are ideal for this approach - is that there is no bar to taking them off again and moving them to a new cluster. Thus, the first major difference from conventional focus groups is that this new approach forces a degree of structure on the participants. On the other hand, this structure is determined by them - and usually proves to offer a valuable insight into the underlying relationships.

As in any form of ‘brainstorming’[22], to which the process is also indirectly related, the initial ideas typically stimulate others. It is, indeed, important that everyone is positively encouraged to add their own Post-It Notes to those on the wall. This is a second difference, in that the quietest members of the group can, and do, participate without intervention by the facilitator/moderator [23]. The role of the facilitator is, indeed, much less problematic than is the case for conventional focus groups; and the skills needed by them are also less critical. The participants only have to add a note to the wall to make their point, which most people seem to find easier than entering into a conversation!

The result is a very powerful form of creative interaction within the group, which is applicable to a wide range of situations (but is especially powerful in the context of these global scenario groups). In view of the short time available for such groups to work together, just one or two hours, it also offers a very obvious context for those who are coming to the process for the first time. Since the workings are largely self-evident, participants very quickly come to understand exactly what is involved.

As the initial flow of ideas slows down, participants can easily be introduced to the next stage, with rather more ease than in a traditional focus group [24], though many of the groups may well move on of their own volition. The key fact is that the facilitator can progress the group through the stages of the research especially easily; and without undue intervention - again requiring less skill. In addition, as the record of the previous discussions is retained - on the wall - the participants rarely need to be reminded of where they have been.

One aspect which is much more difficult to achieve with conventional focus groups, is to ask the participants to develop the relationships between the various factors - something which may be particularly important, say, for work leading to cluster analysis or (as in our futures research) to establishing the dimensions to be explored in subsequent quantitative work. Thus, in the case of our scenario groups, they were asked to try and arrange the drivers, which had emerged from the first stage, into groups which seemed to make sense to them, where the final aim of this stage was to lead into the creation of the 6 - 8 larger groupings; 'mini-scenarios'. This is where the Post-It Notes are almost essential - they will continue to stick no matter how many times they are moved around. While this clustering process is taking place the participants can continue to add new topics - and more Post-It Notes are added to the wall. In the opposite direction, the few unimportant ones can just as easily be removed by them (to be grouped, if required, as an 'audit trail' on another wall).

As the clusters - in our case the 'mini-scenarios' - start to emerge, the associated Post-It Notes can be stuck to each other rather than individually to the wall; which made it easier to move whole clusters around. One great benefit of using Post-It Notes is that there is no bar to changing your mind. If you want to rearrange the groups - or simply to go back (iterate) to an earlier stage - then you strip them off and put them in their new position.

Needless to say, the highly visual nature of the patterns produced by the various arrangements of the Post-It-Notes is especially effective in stimulating investigation into the relationships between them.

Reverting to the framework which specifically relates to global scenarios, and applying the techniques of self-documenting focus groups:

At this stage, therefore, our participants are asked to try and arrange the global drivers, which emerge from the first stage, into groups which seem to make sense to them. This is usually the most (conceptually) difficult step. It is where managers' 'intuition' - their ability to make sense of complex patterns of 'soft' data which more rigorous analysis would be unable to handle - play an important role. In practice, there are initially many such small groups of drivers created; but these are then gradually refined down to a smaller number

This is the stage which is completed by all of the groups - and is the main basis for our subsequent analysis.

Indeed, the outcome, in the case of our ‘simpler scenarios’, is usually between seven and nine logical groupings of drivers. In practice, due to the scope of the research (covering all aspects of society) our global groups tend to produce rather more clusters; typically around 10-12 in number.

This stage is also completed by all of the groups, but is only used as a check on the main work which is undertaken by ourselves.

A final, major advantage of the new, self-documenting focus groups, approach is that the main ‘documentation’ of the results is progressively accumulated as a natural part of the process - in the form of the Post-It-Notes adhering to the wall - by the participants themselves. It is they who decide, in this way, what they want to say; and this requires no interpretation to be useful [25]. The process may be tape-recorded, or a video made, but the prime record is the notes on the wall - which may then be most easily photographed to form a permanent record. Following the same line of argument, the analysis, at least at the first level focus group work, has already been started by the participants - in terms of their own clusters and patterns - reducing the need for content analysis[26]. In the case of global scenarios, however, it should be noted that this is only a ‘cross-check’ on the central analysis - which starts again with the individual drivers for change.

In our scenario planning theory for smaller organisations, the main action, undertaken by all the participants as the next stage, is to reduce the seven to nine mini-scenarios/groupings detected at the previous stage to just two or three larger scenarios [27]. We have found, however, that the time limitations prevent our global scenario groups from fully completing the later stages of the process in the one session available; which typically cannot last longer than two hours. Thus, in the earlier (global) groups, the rearrangement of the drivers to produce fully-fledged scenarios was incomplete; though, even so, the results were enlightening and useful in suggesting what the final scenarios might have been. In the later groups we achieved the same effect by specifically asking the groups to derive 'scenario titles' separate from the drivers - though clearly influenced by the work they had done on them - and this has now become our standard (global scenarios) approach.

In this 'global' research, therefore, the transformation into the final scenarios (and even the production of the overall 'mini-scenarios') is now undertaken by ourselves rather than the participants. The prime input to this process is all the drivers identified - for instance, by the 17 research groups (covering some 140 organisations) in the original research. Once more using the standard (Post-It-Notes) scenario forecasting technique we have developed [26] , all these drivers (from all the groups involved) are progressively clustered - by central staff - into aggregated 'mini-scenarios'; regardless of how they have originally been clustered by the groups. This overall clustering is, however, then compared (cross-checked) with the separate work of the groups - to ensure that it is still meaningful in broad terms. This has generally proved to be the case. It is our experience that there is, in any event, some considerable convergence between the results of most of the groups.

These combined mini-scenarios are then transformed into the final scenarios. In view of the very broad scope of the investigation, covering the whole of society, it has proved necessary to utilise four scenarios - rather than the two we would normally recommend for our more traditional work with smaller organisations. These are again compared with those suggested by the groups, and once more we found that there were no major disagreements between them in our original research.

In the normal scenario forecasting process, having grouped the factors into two scenarios, the next step is to examine them for consistency. In our global work, in general, this is supplemented - as mentioned above - by the comparison with the groups’ own scenarios. In the case of the original research, however, this test also took the form of a comparison with the published work of a wide range of other futurists. Here too there was a surprising degree of agreement. Though the nature of future developments is often seen to be a controversial issue, one result from the research was therefore that, at least amongst those studying the long-term future, there seems to be a degree of unanimity; a workable consensus. Indeed, one major aspect of our results turned out to be a surprising degree of convergence between the outcomes reported by almost all the various groups - ranging from uninformed generalists to informed futurists - in contradiction to the uncertainty generally reported by the media. It is this convergence, indeed, which we believe goes a long way to underwriting the viability of the techniques.

So far, most of the technique(s) described have been assumed to be those applying to just one group of eight participants. It should have been obvious, however, that in the last two steps we aggregate the drivers produced by a number of such groups. The requirement for significant numbers of participants, even - unusually - at this qualitative stage of the overall research, is generated by the need to ensure that all the significant drivers (the dimensions to be quantified in subsequent stages) have been identified. As there may be more than a hundred such drivers, if the research is genuinely to be ‘global’, it may be necessary to run five or more groups, so that multiple occurrences of each of these significant drivers may be observed. Otherwise, researchers cannot be confident that the single occurrences (emerging from a smaller number of groups) may not just represent one individual’s idiosyncrasies. The problem may be resolved at the quantitative stage, but it will waste respondent’s time and - worse - annoy them, by the questionnaire’s apparent triviality, to such an extent that this distorts the results.

Above all, though, the more participants recruited to the process the more confident that researchers, and the ‘customers’ for their work, will feel that all the significant drivers have been identified.

One group of events which have traditionally caused problems for long-range forecasters [27] are those ‘wild cards’ which surprise everyone. According to strict theory, these ‘wild cards’ must be unpredictable (or at least of low predictability) as well as of high impact; and, hence, these might cause problems even for those using the techniques described here.

There is, though, one important sub-category of wild cards, those involving slow hidden processes, which are present over long periods but suddenly emerge (as ‘creeping catastrophes’[28]). This category - which accounts for a large proportion of wild cards overall - may be detected by the technique(s) described here; as long as sufficient numbers of participants are involved. Essentially, researchers should look for two or more occurrences which surprise them. They need two such occurrences because it is almost certain that otherwise they will reject the issue as unbelievable!

Clearly, the more participants the greater the chance of two such occurrences emerging. Indeed, it is unlikely that they will emerge from less than ten or twenty groups (comparable with the number we, ourselves, completed). As a result, this is an approach more suited to larger organisations, with their larger budgets.

As these wild card issues are - almost be definition - unbelievable for most respondents, it is that much more difficult to incorporate these in the quantification stage. On the other hand, once they have been identified, these ‘creeping catastrophes’ typically become certain factors; not needing further research.

One specific, remaining limitation of the techniques described is a failure to highlight structural changes occurring in the macro-environment. This is partly because the techniques typically tend to be most effective in aggregating large numbers of ‘small’ expectations. But it is mainly because Western managers are ‘culturally blind’ to many of the structural changes which occur across society as a whole; one group of Third World participants who had a Marxist background, for instance, did detect such structural factors.

The result is that the central staff, aggregating the drivers into the final scenarios, must themselves detect where the issues reported are symptoms of underlying structures. In practice this proves less problematic, than might be expected; where the academic researchers, involved in our work to date, are motivated to find such structural trends.

Once more, it is difficult to quantify these, but - again - these structural changes (once detected) also typically prove to be certain.

Conventionally the resulting scenarios are then 'written up' in the form most suitable for the target audience; and, indeed, a summary of our initial work was published conventionally [29]; and summaries of the future developments are already scheduled for publication in the same way. In the case of the original material, due to the very wide scope of the investigations, the final ‘report’ ran to more than 80,000 words. Accordingly, the publication of the more detailed material has been on the Internet 'World Wide Web' (WWW).

Again, the new results - and our revised interpretations - will be published, as they become available, on the Web.

The output from such qualitative focus group research is often used to determine the questions for subsequent quantitative research [30]. In particular, it is used to determine the dimensions to be investigated in market segmentation and product positioning[31]. As the requirement for quantifying our futures research was very similar, we chose this approach - in preference to Repertory (Kelly) Grids [32] - because of the large number of dimensions involved. Thus, the results of this qualitative research have been used to identify the 162 dimensions, the most likely factors (drivers) for change, to be explored in the subsequent quantitative research.

Using rather more traditional marketing research techniques, in an initial mail-survey conducted with 300 OUBS students, each of the 162 factors (questions) is now presented in terms of two semantic differentials [33]. The first of these asks for a date (from AD 2000 to 2050) when this factor will become a reality; or, if thought less likely, the percentage chance (10-50% or <10% chance). The second asks for rating of its importance to the manager’s organisation (on a scale from 1, unimportant, to 7, very important). Although the result may look like an early stage of a conventional Delphi study [34], and might even be used as such, the dimensions are already derived from the wider population rather than being in the process of being sought from expert opinion.

Despite the complexity of the questionnaire, 160 responses (53%) were received from the initial survey - in line with our normal response rate. Less than ten percent of these were completed incorrectly. In addition to simple tabulation, these detailed results will be factor/cluster analysed, to establish the relationships between them.

The overall project involved the development of two new sets of research techniques. The first of these, ‘simpler scenarios’ [35, 36], allows this approach to be used as a framework for more general research. The second, ‘self-documenting focus groups’ (reported here in full for the first time), enables a large number of dimensions to be generated; as input to the subsequent quantitative research which uses well-tried techniques (based on semantic differentials). As initially evidenced by the observed convergence of results, the combination allows the expectations of a large population to be investigated - and, thus, future outcomes to be more accurately predicted.

3. Mercer, D., A New Qualitative Technique for Exploring the Future, Marketing Education Group (MEG), Warwick (August, 1996)

5. Kane, J., A Primer for a New Cross Impact Language - KSIM, Technological Forecasting and Social Change, 4 , 129-42 (1972)

6. Makridakis, Spyros, Metaforecasting: Ways of Improving Forecasting Accuracy and Usefulness, International Journal of Forecasting, (4), 467-491 (1988)

7. Toffler, A. and Toffler H., Creating a New Civilization, Turner Publishing, Atlanta, 1994

8. Mercer, D., Large Scale Conferencing for Inexpert Users, Proceedings of the TeleTeaching Conference - Trondheim, Elsevier, Oxford, 1993

9. Mercer, D., The Foreseeable Future - 'Millennium Project' Qualitative Results, OR38 (Operational Research Society Conference), (September, 1996)

10. Acemoglu, D. and Scott, Andrew, Consumer Confidence and Rational Expectations: Are Agent’s Beliefs Consistent with the Theory, The Economic Journal, 104 (January), 1-19 (1994)

11. Wack, Pierre, Scenarios: Uncharted Waters Ahead, Harvard Business Review, 1985(Sep/Oct), 139-150 (1985)

12. Wack, Pierre, Scenarios: Shooting the Rapids”, Harvard Business Review, 1985(Nov/Dec), 139-150 (1985)

14. Aguilar, F. J., Scanning the Business Environment, Macmillan, London, 1967

15. Goldman, A.E. and McDonald, Susan, The Group Depth Interview: Principles and Practice, Prentice-Hall, Englewood Cliifs N. J., 1987

16. Shaw, G. K., Rational Expectations, Bulletin of Economic Research, 39(3), 187-209 (1987)

17. Holt, K., The Role of User in Product Innovation, Technovation, 7, 249-258 (1988)

18. Bellenger, D. N, Bernhardt, K.L. and Goldstucker, J.L., Qualitative Research In Marketing, American Marketing Association, Chicago, 1976

19. Wells, W. D., Group Interviewing, in Handbook of Marketing Research. Robert Ferber, McGraw-Hill, London, 1974

20. Krueger, R. A., Focus Groups: A Practical Guide for Applied Research, Sage, London, 1988

21. Holsti, O. R., Content Analysis for the Social Sciences and Humanities, Addison-Wesley, Wokingham England, 1969

22. Kahane, Adam, Scenarios for Energy: Sustainable World vs Global Mercantilism, Long Range Planning, 25(4), 38-46 (1992)

23. Rockfellow, John D., Wild Cards: Preparing for the ‘Big One’, The Futurist, 1994(Jan-Feb), 14-19 (1994)

24. Steinmuller, K., The Future as Wild Card, 5. Internationale Sommerkademie des Sekretariats für Zukunftsforschung - Gelsenkirchen, (September, 1996)

25. Mercer, D., The Foreseeable Future, Management Decision, 34(3), 55-61 (1996)

26. Johnson, R. M., Market Segmentation: A Strategic Management Tool, Journal of Marketing Research, 8(February), 13-18 (1971)

27. Sampson, P., Qualitative Research and Motivation Research, in Consumer Market Research Handbook, 3rd. ed. R. Worcester and J. Downham, McGraw-Hill, Maidenhead England, 1986

28. Osgood, E.C, Succi, G. J. and Tannenbaum, P.H., The Measurement of Meaning, Urbana, New York, 1957

[1] Which is now also part of the United Nations University American Committee’s (UNUAC) Millennium Project.

[2] The courses at the United Kingdom’s Open University Business School (OUBS) - especially those at MBA level - typically stress longer-range strategy. The key MBA course, which was the vehicle for developing and testing the techniques described in this paper, is 'The Challenge of the Environment' - which has broken new ground in addressing the issues in the wider environment which are critical in terms of long-term (10+ years ahead) survival.

[3] Which is, indeed, the technique chosen by UNUAC for its own work, with experts from around the world (including ourselves from the OU) feeding in their judgements.

[4] For instance, one debate, contributed to by managers from a number of multinationals, provided the surprising insight that these organisations do not recognise common interests with other multinationals in general, but only with those in their own (or closely related) industries!

[5] In particular, this research (to be published in Industrial Management and Data Systems) quantified the level of optimism which participants felt about the future; where it has been claimed that confidence levels are useful indicators of future consumption [13]

[6] We have used senior managers, and professionals, in our own work because we - unusually - have relatively easy access to them, and we assume that - although they represent a large population overall (much larger, for example, than the small group of actors typically described by the rational expectations hypothesis [21]; in terms of the macro-environment they have greater leverage than ordinary members of the wider population.