EVALUATION OF PARTICIPATIVE APPROACHES TO RD&E: A CASE STUDY OF FARMSCAPE

J.A. Coutts¹, Z. Hochman², M.A. Foale², R.L. McCown, ², and P.S. Carberry²

¹Rural Extension Centre, University of Queensland, Gatton, Qld, 4350 ²CSIRO/APSRU, Toowoomba, Qld, 4350

Abstract

This paper describes the evaluation framework and process developed and used for the FARMSCAPE project. It considers the impact of the evaluation on the RD&E process and its value/limitations in making meaningful statements/conclusions about the effectiveness of the participative RD&E project. Indicative results emerging from the evaluation about the FARMSCAPE project are presented. The paper concludes that the evaluation approach used was useful in providing benchmarks and teasing out the impact of the project on different participants. It also provided some evidence that the FARMSCAPE approach to developing and using decision support tools was useful in complementing farmer experience in dryland farming systems.

Key words: Process evaluation, participative RD&E, farming systems research and extension.

Effective evaluation of Research Development & Extension (RD&E) projects and programs has been a difficulty faced by funders and project managers for many years. Adoption and cost-benefit studies have been plagued with the problems of measuring and then attributing changes to specific activities, and capturing impacts in extremely variable climatic and marketing conditions (1). Participative RD&E adds an extra level of complication: outcomes are often emergent rather than pre-determined; RD&E is enacted through interaction between a number of players; and impacts on all stakeholders are also critical rather than change in a single target group.

There is also a stronger need to provide ongoing input into project management in participative RD&E rather than a reliance on post project analysis. (2). The FARMSCAPE project, a participative RD & E project 'Farmer-Adviser-Researcher Monitoring, Simulation and Performance Evaluation for best dry land cropping practices', has these features and was the subject of this study.

Methodology

The evaluation sought to monitor and interpret the project through the eyes of the key participant groups: researchers; farmers; private consultants; and extension officers (both within and outside of the project) over the life of the project. At intervals throughout the project, interviews were undertaken with (approximately 30) representatives of these groups to capture learning and practice change as it happened and within the context of project activities and seasonal conditions. Interview data was collated and summarised from each participant group and returned to interviewees for checking that the information was correct. The project team received the summaries of all groups to provide an overview and stimulus for change in the project direction and activities. Insights gained were included in a project newsletter which was widely circulated.

Results and discussion

The iterative interviews and their analysis yielded strong, corroborated data about the impact of the project and its process on the key participant groups. Information gathered from each participant groups supported emerging conclusions. For example, follow-ing the mid-term (two years into the project) analysis of interviews with commercial advisers, an external evaluator, Van Beek (3), concluded that "... the inter-viewees in this group confirmed all the effects claimed by farmers: ... farmers have begun to think differently: more three dimensional: taking soil, water and nutrient balances down to 1.8 metres into account more long term; more strategically towards maintaining and improving resources; and taking economic aspects more into account." The iterative nature of the interviews also proved valuable in providing benchmarks of attitudes and practice. As Van Beek (3) observed...comparing last year's and this years notes... there is a substantially more positive attitude to FARMSCAPE products than a year ago. The evaluation process was flexible to fit in with key decision-making periods in the farming cycle, and to pursue data from new sources as their importance was established. For example, the mid-term interviews were broadened to include cotton growers and research managers. Some of the impact value of the evaluation was lost, however, because the data collection and its analysis took place within a sub-team rather than involving the whole team.

The analysis of interview data (developing and linking emergent categories and relationships - and using participants' direct quotes to minimise bias) indicated that those farmers directly involved in the project increased their use of soil testing; and explored crop simulations for: confirmation of current practices; use of alternative crops; and considering 'what-if' scenarios for the most efficient use of soil water. They did not embrace simulation outputs as expert knowledge to be adopted, but rather as an inquiry framework to test against their own experience. Commercial advisors used new soil monitoring techniques and simulation model outputs to enhance their advisory value to leading farmers. Model complexity and organisational changes limited the ability of commercial companies to directly and independently use the simulations - they relied on researcher support and input. On-going evaluation will monitor whether these farmers and advisors continue to use the tools and framework post the intensive phase of the project. Some extension officers claimed that they benefited from the project through an increased understanding about soil, water and crop management. Concern by others centred around the limitation of the approach to benefit the wider group of farmers and advisors, and the need to maximise the educational value of simulation, rather than focusing on its role in making recommendations. As a result of the evaluation process, researchers modified the model parameters and user interface, and changed the emphasis of the project from providing solutions to dryland farmers to providing a framework for farmer decision makers to test alternatives and complement their own experience.

Conclusion

The use of iterative interviews with different participant groups as an evaluation approach has proven to be very effective and robust in terms of capturing perceptions, learning and practice change in the stakeholder groups closely associated with the project. It has permitted a 'teasing out' of the value and impact of different project activities on participants, and provided a deeper understanding of the context and complexities operating within the project environment. The process could be strengthened by including the total project team in the analysis of collected data

The evaluation process provided some evidence that the project was having a positive impact on: learning within each participant group; attitudes, decision-making and practice. It highlighted the complexities in the management of dryland crops and the limitations of simulation aided decision making in providing expert recommendations. However, the evaluation has shown that simulation, adequately contextualised, was valued by participating farmers and advisers in: (a) gaining insights into production system function; and, (b) augmenting their farming experience in making judgements required in tactical responses and the evolution of improved production strategies.

References

1. Coutts, J. 1994 Process, Paper Policy and Practice: A case study of the introduction of a formal extension policy in Queensland Australia 1987-1994. Wageningen Agric University, The Netherlands.

2. Dart, J., Petheran, R.J. and Straw, W. 1997. Proc 2nd Australasia Pacific Extension Conf. Albury. p 408.

3. Van Beek, P. 1997. The FARMSCAPE Project: External Mid-Term Benchmarking. APSRU, Toowoomba, Queensland.

Abstract

Methodology

Results and discussion

Conclusion

References

...

Search Site