Previous PageTable Of ContentsNext Page

THE MANAGEMENT OF MODELS: RISKS AND RESPONSIBILITY

P.G. Cox

CRC for Weed Management Systems, NSW Agriculture, Orange Agricultural Institute
(Present address: Cambodia-IRRI-Australia Project, PO Box 1, Phnom Penh, Cambodia)

Abstract

Models are characterised as purposeful representations: they are tools intended to help practitioners work through a specific issue in order to improve a situation in the real world. Clearly, since issues are diverse, and the nature of what constitutes an improved outcome is often contentious, several models will be used, in various ways, to support debate and ultimately to change behaviour. Any attempt to impose a single or rigid model structure is restrictive and, in the end, unproductive. Unless a model generates incremental insights into what can be done, and the implications of different courses of action, and changes the probability that a particular course of action will be realised, nothing useful is achieved. Often, responsible professional modellers will not impose any model structure, but facilitate development and increasing sophistication in the representations of others. Risks and responsibility in modelling go together: risks are contained by a responsible pluralism.

Key words: Management, models, risk, responsibility, pluralism.

When it was proclaimed that the library contained all books, the first impression was one of extravagant happiness. All men felt themselves to be masters of an intact and secret treasure. There was no personal or world problem whose eloquent solution did not exist in some hexagon.' This is how Borges (4) described the Library of Babel. So it has been in recent years with models. Thus, in an article in Scientific American, Karl et al. (13) state that, 'In the future, to reduce the uncertainty regarding anthropogenic climate change, especially on the small scales, it will be necessary to improve our computer modelling capabilities, while continuing to make detailed climatic observations.' It is assumed that the solution to our problems will be through the discovery of improved representations.

Modelling is used in many scientific areas to stimulate debate, to support a particular line of argument (including justification for increased research funding), to structure our understanding of the way a world works, and as a contribution to improved (or at least changed) management practices. In this complex dance, different scientific representations engage each other, and with other kinds of representation in the context of agricultural research for development, in an attempt to bring about (and to demonstrate) better outcomes: improved understanding, novel insights and opportunities realised (eg. 11, 14, 15).

This paper is not concerned with definitions of sustainability and environmental stewardship, although I believe that the process of continual re-negotiation of models is part of what sustainability is all about (7). Instead, emphasis is given to our increasing reliance on models (or, rather, on the representations of professional modellers), and the associated risks and responsibilities for those who fabricate and use these representations.

The management of models

There are several dangers for modellers: coming to believe that their model of a world is the only possible one (or the best, or even the best available) because other models do not exist, or are impossible to parameterise, or are inferior in some other way; that their model is right and the world is wrong; that models will solve problems (cf. people using models resolve problems!); that solutions derived without the aid of particular models are invalid in some way; that modelling is (or even can be) objective; that modelling is costless; that other people need specific models to make decisions about issues they face; or that espousal of these beliefs does not pose a significant threat to professional modelling activities. The risk for the rest of us is that we will accept such implicit assumptions unthinkingly and continue to support modelling activities long after their termination date is due. Unless a model generates incremental insights into what can be done, and the implications of different courses of action, and changes the probability that a particular course of action will be realised, nothing useful is achieved. The opportunity cost of doing nothing, of letting models run their course, is high: models cost. Risks and responsibility in the management of models go together.

Risks

A model may be the wrong model: in relation to an agreed purpose; in relation to the required resolution needed to precipitate a decision about a preferred course of action; and in relation to the costs and benefits of model development and use (eg. 9). This can arise because: the issue is mis-specified; the structure of the model is not isomorphic with the problem; the initial conditions and other assumptions are mis-specified; or because the model is not evaluated in comparison with alternative models. No single model formulation will ever be a complete representation of how worlds work: all representations are partial, but partial in different ways.

There are ways out of this dilemma: by checking that the resolution of the model is adequate (but not excessive) compared with alternative model formulations (6); use of several models simultaneously to address an issue; ensuring that the sources of error are recognised and incorporated into the analysis (8, 10); that the scenarios considered for analysis are relevant situations (or, at least, not extreme cases construed as representative); that the model is accessible and understood (there are no black boxes which have to be taken by others on trust); and that investment in generic models is focused on improving the modelling capabilities of others, not used to constrain the development of those capabilities by imposing a 'one size fits all' straight-jacket. The risks associated with models are largely ones of failing to appreciate, or explain to others: their structure; the assumptions on which they are based; the reliability of their formulation and data; their partiality; their articulation with other representations; and the associated costs and benefits of model construction and application in relation to existing (or other possible) model formulations.

Responsibility

This poses a special responsibility for professional modellers: to seek alternative model formulations; to justify model choice; and to evaluate models as effective tools in relation to a particular and specific purpose. The key question is: what kind of representation would help progress the resolution of a specific issue? Thus, the kind of model we develop is determined by the nature of the issue we face. In practice, this is rather hard to do – a world is only poorly separable from our model of it, and is conditioned by sets of values of which we may only be dimly aware. How we see (model) a world depends on where we sit and who we are.

Romm (18, p. 185), in her discussion of the issue of researcher responsibility, suggests that we can define "failure" to exercise responsibility as the propensity to defend one's stance as if it constituted the only rational way to proceed. Responsibility, conversely, implies that modellers investigate alternatives and use this investigation as the basis for how one thinks and acts. Such a modeller would acknowledge that a world of multiple knowledge frames is neither value-free nor valueless. 'It is a world of possibility and responsibility. It is a world in which we refuse to use "objectivity" [defined as value-freedom] as a reason for avoiding personal involvement in our knowledge productions.' (12). Romm proposes that responsible action requires: admitting that there are choices to be made; and being able to defend choices in the light of a serious consideration of what other approaches may offer. Many of these risks (and the need for professional responsibility in managing them) have been the subject of various attempts to define a professional code of practice for researchers in the conduct of Operational Research (OR, ie. one kind of professional modelling) eg. 1, 20.

White and Taket (21) indicate that their approach to working with organisations is to provide a semi-structured framework within which the organisation members are enabled to: identify problems or issues of concern to them; explore the nature of the problem/issues; generate and explore feasible opportunities for change; and build a plan of future action. They argue that 'the project of a modernist methodology for OR in this context is flawed, and that the concomitant modern expert is dead'. This is not to say that the person "doing OR" has no role in such a situation, only that the role is different from that usually ascribed to the modern expert. The construction and manipulation of models can contribute to such debates, but are not a substitute for the debate, and the privileged access of one contributor to particular classes of representation does not confer on her the status of expert.

Ethical issues thus arise: in constraining the way in which alternative representations are sold (they cannot provide an answer, and are only a patterning tool); the kinds of representation that are proposed/developed by the OR practitioner (generally simpler, more transparent representations rather than a blatant argument from authority - the death of the expert); how models are developed and the output interpreted (perhaps a move towards community OR eg. 16); the need for continual exploration of the boundary assumptions of particular model formulations (under what circumstances is the particular model formulation applicable?); and much greater concern with the negotiation process by which different representations of a problem situation are articulated (5).

White and Taket (21) offer several guidelines to help shape OR (modelling) practice: a recognition of the co-responsible nature of the OR encounter; the aim of achieving skill transfer (both ways); the importance of recognising difference and working with it; and flexibility - the readiness to work in different ways at different times. Their aim is to develop a more critically reflective OR practice. Often, responsible professional modellers will not impose any model structure, but act to facilitate development and increasing sophistication in the representations of others.

Decision support systems (ie. OR models that have been transplanted to a routine operational context) have been proposed as a way of transferring information from researchers to farmers. I have questioned (6) whether this is justified as a general strategy because routine decisions are often clear, and difficult decisions are often only difficult because they are marginal - it does not matter a jot which way you jump, either because there are no differences between the outcomes associated with alternative decisions or the background is so noisy that these cannot be distinguished. The set of decisions for which decision support is of any value is thus much reduced. It is reduced further because one way out is not to provide a comparison of options, but merely to point out that there are options which have not been considered (the choice between them being clear or marginal as before). Within an agreed set of options, adequate resolution to distinguish major differences can often be achieved using lexicographic models such as production rules, or holistic insight (intuition) about how the alternatives fit with established practice. Thus, the set of issues for which a more structured screening may be justified is further reduced. Add to this the uncertainty surrounding the precise formulation of the scientific model, and the situation gets even more problematic. Beeson (3) argues that meaning is always too much to specify.

Example

Pascual et al. (17), using the management of the Serengeti wildebeest as an example, fitted several age-structured and unstructured models to a common data set and compared model predictions of wildebeest viability and harvest. Models that depicted demographic data in very different ways fitted the data equally well. Traditional model selection methods, based on the principle of parsimony, were thus not practical for judging the value of alternative representations. However, the models produced contrasting assessments of the consequences of harvesting wildebeest (ranging from enhanced extinction risk to reduced extinction risk). They conclude: 'The major lesson to be learned from our analyses is that biologists working with population management should avoid pursuing some "best model" as a holy grail and should instead be willing to consider many alternative portraits of the system under study.' The experience of the Agricultural Production Systems Research Unit in Australia was, in many ways, similar: different model formulations fit the data equally well (badly) and the choice of model must be based on other criteria. These are determined by the use to which the model will be put (and by whom) and the net value of being able to discriminate between the implications of alternative decisions.

Conclusion

Models have often been built in relative isolation of specific issues they are used to address. It may be, when we look closely, that there are no issues requiring the kind of models we have built. It is also likely that many issues can be addressed adequately using off-the-shelf models: the responsible strategy then is one of gaining familiarity with a range of model categories that are already available rather than constructing, or refining, or refurbishing new ones. In some situations, it is the process of model construction and application that is more important than the provision of pre-configured tools. This requires continual questioning of model-related activities. The opportunity costs of building the wrong models, or excessively elaborate models (but essentially meaningless ones because of their limited contribution to good communication practice), are high.

But we cannot avoid models - or, as Baudrillart says, 'simulations' (2). Human discourse is based on the sharing of negotiated representations. At best, what we know as "modelling" within agricultural R&D is no more than that. Thus, the management of models is grounded in communication practice. The abdication of model development and application to single model structures, not tailored to the requirements of specific issues, and unreflective about own-practice and the particular contribution of specific model formulations (even worse, to a single group of actors which has privileged access to idiosyncratic knowledge - in other words, to "experts"), risks the marginalisation of that set of model-constructing activities. In a world in which resources for the scientific project are increasingly scarce, the failure to use models in a responsible way risks too much. Opening modelling up to wider audiences, sharing experiences, valuing diversity in model structure, and encouraging a multiplicity of approaches to the resolution of specific issues – in short, a responsible pragmatic pluralism (19) - all help to contain this risk.

Acknowledgements

The manuscript benefited a lot from the comments and suggestions of David Freebairn, Dick Medd, Kevin Parton and Peter Ridge.

References

1. Ackoff, R.L. 1974. Oper. Res. Quart. 25, 361-371.

2. Baudrillart, J. 1983. "Simulations". Semiotext(e), New York).

3. Beeson, I. 1997. Meaning - too much to specify. Paper presented at the Third Forum, Centre for Systems Studies, University of Hull. Centre for Systems Studies, University of Hull: Hull, UK.

4. Borges, J.L. 1970. In "Labyrinths". Penguin Books, London, UK.

5. Cox, P., Parton, K., Shulman, A. and Ridge, P. 1996. In "Applications of Systems Approaches at the Farm and Regional Levels", Volume 1, Proc. 2nd. Int. Symp. Systems Approaches for Agric. Devel.,  Edited by P.S. Teng, M.J. Kropff, H.F.M. ten Berge, J.B. Dent, F.P. Lansigan, and H.H. van Laar). IRRI, Los Baños, Philippines. pp. 213-228.

6. Cox, P.G. 1996. Agric. Systems 52 (2/3), 355-381.

7. Cox, P.G., MacLeod, N.D. and Shulman, A.D. 1997. In "Systems for Sustainability: People, Organizations and Environments." Edited by F.A. Stowell, R.L. Ison, R. Armson, J. Holloway, S. Jackson, and S. McRobb. Plenum Press, London, UK. Ch. 6, pp. 33-38.

8. Finlay, P.N. and Wilson, J.M. 1997. Systems Res. and Behav. Sci. 14 (3), 169-182.

9. French, S. 1995. J. Oper. Res. Soc. 46, 70-79.

10. Gaunt, J.L., Riley, J. and Penning de Vries, F.W.T. 1997. Agric. Systems 54 (2), 153-168.

11. Hook, R., Williams, J. and Gascoigne, H. (Eds.) 1997. "Farming Action Catchment Reaction." CSIRO Publishing, Melbourne, Victoria.

12. Jacques, R. 1989. In "Operational Research and the Social Sciences." Edited by M.C. Jackson, P. Keys and S.A. Cropper. Plenum Press, New York, NY. pp. 703-708.

13. Karl, T.R., Nicholls, N. and Gregory, J. 1997. Sci. Amer. 276 (5), 54-59.

14. Lane, D.C. 1995. J. Oper. Res. Soc., 46, 604-625.

15. Oriade, C.A. and Dillon, C.R. 1997. Agric. Econ. 17 (1), 45-74.

16. Ormerod, R. 1995. J. Oper. Res. Soc. 46 (3), 277-293.

17. Pascual, M.A., Kareiva, P. and Hilborn, R. 1997. Conserv. Biol. 11 (4), 966-976.

18. Romm, N.R.A. 1996. In "Critical Systems Thinking. Current Research and Practice " Edited by  R.L. Flood and N.R.A. Romm. Plenum Press, New York, NY. Ch. 10, pp. 179-194.

19. Taket, A. and White, L. 1996. Systems Practice 9 (6), 571-586.

20. Wallace, W.A. (Ed.) 1994. "Ethics in Modeling." Pergamon, Oxford, UK.

21. White, L. and Taket, A. 1994. J. Oper. Res. Soc. 45, 733-748.

Previous PageTop Of PageNext Page

...

Search Site