Previous PageTable Of ContentsNext Page

Evaluation for NAVIGATOR

Patricia Murray1 and Katherine Boon2

1 University of Adelaide.
Email:
patricia.murray@adelaide.edu.au
2
Primary Industries and Resources South Australia.
Email:
boon.katherine@saugov.sa.gov.au

Navigator is an innovative program designed to enable producer groups to determine their own agenda, set their own goals and design projects that will be useful to them. The evaluation of Navigator has also been innovative in many ways. Navigator is currently being piloted with five regionally diverse groups of wine grape producers. The groups have been running from between six and twelve months and will soon be augmented by industry run groups. The bulk of the evaluation therefore is incomplete. The evaluation can be divided into four parts, evaluation for project development, process evaluation, third, evaluation looking for cultural change and finally participant evaluation of their projects.

First as part of project development we have undertaken extensive evaluation of each aspect of the Navigator processes. This has involved close collaboration between project developers and evaluator, and has resulted in an iterative project development process. Facilitators, project developers and evaluator have developed a set of indicators related to objectives, these have been refined over time. For example, indicators of cultural changes in the direction of development of self-reliance include: group members undertaking to find information for the group, changes in content of comments in group discussion.

Changes in behaviour of the group or particular members are noted as means of identifying and systematically recording evidence of group development. Thus the conduct of discussion has changed in some groups to include more banter and greater openness. Similarly the levels of participation in discussion can change over time, indicating that participants are more or less comfortable in the group. Other changes have included the nature, focus and content of discussion, with groups becoming more focused over time and content indicating group members have increasing mastery of topics. In addition material such as outputs of group discussions, any material generated in such discussions, minutes are recorded and analysed. These are correlated with facilitators’ indicators.

The central point though is that though ‘data’ about group activity and progress are gathered, this is not done at random but with the long term intention of tracking the progress of the process (and set backs, explaining failure). Thus while facilitators can be flexible about the amount and type of information collected, they must also be aware of the ultimate aims of the project and the group.

Throughout, facilitators look out for ‘counter instances’. If the facilitator at the conclusion of a session feels the session has gone well, they try to develop evidence for this. People were enthusiastic: body language, active discussion, lots of points on butcher’s paper, etc. However facilitator’s also look for ‘counter instances’, though most people were engaged in the discussion, were there some who were not? Were some left out, unable or unwilling to get into the activity? Post session feedback from participants is helpful here, but questions are constructed to the precise purpose of developing and understanding the operation of the processes employed in the session, rather than did participants enjoy the session.

This from of evaluation has fed into the second part of the evaluation process. This is evaluation to identify how precisely the process promotes cultural change. This part of the evaluation, together with the evaluation of the change achieved, is in progress. Where objectives include empowerment or cultural change, indicators include: group ‘ownership’ of group activities, group determining activities, group or members taking responsibility for organising activities, changes in power relations in group. It is anticipated that individual participants will adopt or adapt ideas originating in the group or identified by the group. Further evidence would be in the confidence with which group member approach problems, and the ways they seek, evaluate and use information. Other indicators are anticipated to emerge when the process is further along.

Last the Navigator process itself builds in evaluation processes for participants. Because of the nature of Navigator, each group develops its own unique project. Throughout the development and implementation of projects the process asks participants to identify their objectives and to determine whether they have been met. This information is available at any time for participants to institute a more formal evaluation processes.

Previous PageTop Of PageNext Page