1Department of Natural Resources and Environment, Private Bag 15, Scoresby Mail Centre, Victoria, 3176
2Department of Natural Resources and Environment, Chiltern Valley Road, RMB 1145, Rutherglen, Victoria, 3685
3Department of Natural Resources and Environment, Private Bag 1, Tatura, Victoria, 3616
4Department of Natural Resources and Environment, PO Box 905, Mildura, Victoria, 3502
Abstract
Bennett’s Hierarchy was used to plan and evaluate the Victorian viticultural best practice extension program “Grapecheque”. Key evaluation questions were developed in three areas, namely group health (which measured participation), practice change and process improvement. Irrigation, nutrition and grape quality were the issues that all growers, regardless of region, wanted current research information about. Quantitative evaluation showed adoption of new practices (practice change) in each of these three technical areas. Qualitative evaluation illustrated positive advances in group health and process improvement over the three years of the program. Grapecheque has been used as a model for horticultural extension in Victoria, with the Bennett’s Hierarchy process crucial to effective implementation of change within the industry.
Introduction
Grapecheque is a Victorian State Government funded viticultural extension program that began in July 1997 through the Department of Natural Resources and Environment (DNRE). It began in response to recognition by State government and industry in the mid 1990’s that the rapidly expanding winegrape industry required technical support and access to recent research results. Grapecheque aimed to introduce new and existing management strategies to growers, and to assist them with implementing technological developments, developing best practice viticulture and sustainable businesses. This paper describes the process of planning and evaluating the program using Bennett’s Hierarchy.
Materials and methods
Victoria's agricultural industries have a history of group extension programs dating back to the early 1970's, when there were many groups operating in different industries across the state. Most of the viticultural groups disbanded when extension officers ceased to organise them. Group reformation began in 1994, following a workshop between industry and government where viticultural group extension emerged as a priority (Buchanan et al., 1995).
Grapecheque was established in 1997 when three facilitators were employed to run grower groups using adult learning principles. The program emphasised best practice viticulture and sustainable business development. Each facilitator was based in a different region to service the grape growing areas of Victoria: North West Victoria, Port Phillip and Central Victoria. Facilitators were expected to run between four to six groups, meeting at least four times a year. Fifteen grower discussion groups were established in the first year of the program through contact with local grower associations, and by word of mouth.
Group size was unlimited, in practice varying between six and fifty participants, with optimum participation being fifteen to twenty people. The Grapecheque audience included winegrape growers: vineyard owners, managers and vineyard field staff. The program interacted with scientists, and the Cooperative Research Centre for Viticulture’s Research to Practice™ Program. Scientific interaction brought research results directly to the grower, and identified areas where more research was needed.
The groups’ set the meeting topics, ensuring that common local issues were addressed. The facilitator developed a schedule of events to suit the particular needs of each group that included vineyard walks, focussed discussions, presentations by technical experts, bus tours and equipment demonstrations. Meeting schedules dealt with the high priority issues first, and matched seasonal vineyard requirements with meetings.
Grapecheque was chosen to participate in a statewide evaluation pilot program using Bennett’s Hierarchy (Figure 1) as the quantitative tool for planning and evaluating extension programs. Other qualitative evaluation tools, including global assessment scales, were used to assess the adoption of new technology and the ongoing progress of the program.
Bennett's Hierarchy has been widely used in planning and evaluating extension programs, particularly in Australia (Dart et al., 1998, Van den Ban and Hawkins, 1996). The seven prescribed steps (Figure 1) describe the social, environmental and economic consequences of the program (Level 1), the grower practice change required to achieve the outcome (Level 2), and what is required of the growers to make the practice change with respect to knowledge, attitudes, skills and aspirations (Level 3). Subsequent levels are grower reactions (Level 4), defining the audience (Level 5), determining the activities required to get grower involvement (Level 6) and the inputs required to make it all happen (Level 7). Working from Level 1 to Level 7 uses the hierarchy as a planning tool. Working from Level 7 to Level 1 uses the hierarchy as an evaluation tool.
Figure 1. Bennett’s Hierarchy process used for project development and evaluation (after Bennett and Rockwell, 1995).
Three technical topics of high priority common to the original fifteen groups used by the Grapecheque facilitators working as a team to develop best practice objectives for the program, following Bennett's Hierarchy (Bennett and Rockwell, 1995). These topics were:
- irrigation for efficient water use;
- grape quality to produce high quality wine, and
- nutrition for sustainable vine growth.
The practice change objectives are listed in Table 3.
The systematic approach to evaluation aimed to use the results for decision making about program performance, program improvement, and to provide feedback to the Department. Three key evaluation questions (KEQ) were established to evaluate the program:
- Group Health measured participation (Level 3 of Bennett's Hierarchy). KEQ’s were "How healthy are Grapecheque groups? How many people attend meetings and how long do they attend for?"
- Practice Change (Level 6 of Bennett's Hierarchy). KEQ: "How many people were adopting or not adopting the recommendations of the program and why?"
- Process improvement KEQ: "What improvements can be made to the project as it proceeds to ensure that it maximises the outcomes?"
Table 1. Criteria for global assessment scale of Grapecheque group health.
Scale1 | |||||
Criteria |
1 |
2 |
3 |
4 |
5 |
Interest of members in group |
|||||
Group attracts new members |
|||||
Regular meetings |
|||||
Different members host meetings |
|||||
Leadership in group |
|||||
Group has goals and objectives |
|||||
Grapecheque goal awareness |
|||||
Focused discussion group awareness |
|||||
Best Practice Identification |
|||||
Final Score (total/9) |
|||||
1The scale rating is from 1 to 5, 1 being poor and 5 excellent
Group Health and Global Assessment Scales
For Grapecheque to be effective, the program required “healthy” groups. A Global Assessment Scale (Glas) (Hargreaves and Attkisson, 1978) was developed to measure group health (Table 1) using the criteria of commitment to the group, involvement, function, new members attracted to the group, the sharing of findings, adoption of best practice, and continuous improvement of best practice. The higher the scale, the healthier the group. Facilitators rated the groups by the scale twice a year, and used the information to guide and motivate the groups.
Results
Group Health
After three years, the number of facilitators employed had increased from three to four and the original 15 groups had increased to 23, both significant indicators of the health of the program. Approximately 35% of Victorian winegrape growers participated in Grapecheque and more than half of those attended regularly (3 meetings or more per year). Over the three years, participation remained the same in North West Victoria, averaging 11.5 people per meeting, and rose from an average of 16 participants to 27 participants in Greater Victoria. The 228 events run by Grapecheque between 1997 and 2000 were attended by more than 4000 participants.
The composition of the groups changed in the first three years of the program. In North West Victoria some original participants left the groups, but others joined. In Greater Victoria, original members remained, and they were active in enrolling new members. Discussion with irregular participants and non participants identified that meeting times excluded certain participants, some had their needs met by company programs, while others preferred not to belong to a group.
Groups differed in their dynamics (Table 2). Monitoring group health gave the facilitators clues about where differences between the groups lay, and identified when changes need to be made to improve group meetings. For example, the Mornington Peninsula group (final Glas score 4.5) was a large group, with between 12 and 50 participants per meeting, and had an association with a history of running training days for members. The Macedon group (final Glas score 2) was small, averaging 8 participants per meeting, and with an association that occasionally organised training days for members.
Table 2. Global Assessment Scale scores for Grapecheque group health from 1997-2000 (- = group not operating).
Zone |
Group |
Jan ‘98 |
June’98 |
Jan ‘99 |
Jun ‘99 |
Dec ‘99 |
|
Dareton |
4 |
4 |
3 |
2 |
4 |
Gol Gol |
3 |
3 |
1 |
2 |
2 | |
Merbein |
3 |
3 |
3 |
2 |
3 | |
Red Cliffs |
3 |
3 |
3 |
3 |
4 | |
Robinvale |
3 |
2 |
1 |
- |
- | |
Swan Hill |
3 |
3 |
3 |
1 |
4 | |
|
Yarra Valley |
3 |
3 |
4 |
4 |
4 |
Mornington |
4 |
3 |
4 |
4.5 |
4.5 | |
Gippsland |
2 |
2 |
3 |
3 |
4 | |
Geelong |
2 |
2 |
3 |
3 |
4 | |
Far South West |
- |
- |
- |
3 |
3 | |
Macedon |
2 |
2 |
2 |
2 |
2 | |
|
Bendigo |
2 |
3 |
2 |
3 |
4 |
Central Victoria High Country |
- |
2 |
2 |
3 |
3 | |
Colbinabbin |
- |
2 |
1 |
2 |
3 | |
Grampians |
2 |
3 |
3 |
4 |
3 | |
Goulburn Valley |
3 |
3 |
2 |
1 |
2 | |
Rutherglen |
- |
2 |
3 |
3 |
4 |
1In this paper, North West Victoria includes the locations Dareton, Gol Gol, Merbein, Red Cliffs, Robinvale and Swan Hill.
Practice change
Participants were surveyed about current practice at the beginning of the program, and regular participants were resurveyed at the end of the program. The quantitative data collected from the questionnaires showed that the rate of practice change varied with both the issue and the region (Table 3).
Irrigation
Most of Victoria’s 32,300 hectare wine grape industry is irrigated, and most vines are grown in the Murray Darling Basin. Impending caps on water and environmental concerns about salinity mean that growers need to be able to irrigate efficiently. Efficient irrigation means the vine receives enough water to meet its needs without wasting water through runoff or drainage to ground water tables. Efficient water application depends upon measuring soil water to decide when to irrigate and how much water to apply. Our survey data showed a 21% increase (from 68% to 89%) in the use of soil water monitoring to schedule irrigations among Grapecheque participants in the North West, and a19% increase (from 38% to 57%) in the rest of Victoria. In addition, irrigation scheduling techniques that influence grape quality, such as partial rootzone drying and regulated deficit irrigation, were adopted by 39% of Grapecheque participants across the state. No participants used these systems in 1997. More efficient use of irrigation water was also reflected in a 52% increase in monitoring irrigation systems across the state during the first three years of Grapecheque.
Table 3. Summary of practice change data for irrigation, quality and nutrition from June 1997 to May 2000 for participants in the Grapecheque program.
Irrigation | ||||
Adoption | ||||
North West Victoria |
Greater Victoria | |||
Objective: By the end of June 2000, of grapegrowers participating in Grapecheque, 75% of North West Victoria growers and 25% of Greater Victoria growers will: |
1997 |
2000 |
1997 |
2000 |
|
|
|
| |
Grape quality | ||||
|
Adoption in all of Victoria | |||
1997 |
2000 | |||
|
| |||
Nutrition | ||||
|
Adoption in all of Victoria | |||
1997 |
2000 | |||
-- |
85% | |||
1missing data represented by -
The difference in use of irrigation technologies between regions at the beginning of the program was attributed to the North West being a traditional growing region, reliant on irrigation for fruit production. Programs designed to facilitate the adoption of best practice irrigation technology in the region had operated since the mid eighties e.g. Waterwatch. In Greater Victoria, irrigation technology and grape growing were recent phenomena.
Quality
Both domestic and international consumers expect Australian wine to be of known quality at particular price points. Several key components of grape quality have been addressed during Grapecheque, such as freedom from chemical residues and optimum maturity to maximise fruit flavours. Responsibility for producing quality grapes lies with the grapegrower, who needs to be able to supply grapes to the winemaker to meet specifications. Potential chemical residues are monitored by asking growers to complete and submit spray diaries. Use of spray diaries increased by 21% across the state, and was mostly due to wineries requiring spray diaries before accepting fruit. The measurement of fruit ripeness is important in determining the optimum harvest time and during the Grapecheque program this increased by 16%. Grapecheque was the enabler for this process rather than the driver, as the wineries were demanding higher quality fruit (Kym Ludvigsen, pers. comm.)
An important facet of producing quality wine grapes for both viticulturist and wine maker is having an accurate estimate of yield. Research has shown that within a uniform block of vines, grape quality declines as yield increases. Forecasting yield is a valuable tool for the viticulturist to use when aiming for a yield target to produce the grape quality desired. Knowing an estimate of yield before harvest is important for the winemaker to be able to schedule harvest and winemaking operations. Our data show that use of recognised crop forecasting systems increased by 40% amongst Grapecheque participants. Data were not collected on the rate of change amongst non Grapecheque participants.
Nutrition
Some growers were interested in vine nutrition. Growers are interested in having healthy vines that produce high quality fruit. The community is interested in minimal nutrient leaching. Our data showed that 85% of participants used petiole analysis at flowering to monitor vine nutrition, an increase of 28% from the start of Grapecheque (Table 3). Growers were less clear on optimum timing of fertiliser applications and some (17%) were using nutrient balance as a method of maintaining vine health.
Process improvement
Qualitative evaluation was used to show why adoption of some technology has been slow, and why there were differences between regions. These tools were used during the program to improve delivery and adoption. A successful case study of this is shown with crop forecasting. The procedure was trialed by a North West Victoria group, who found it difficult to use and only 8% of participants the group adopted the technology. Qualitative evaluation showed that growers found the crop forecasting methodology very complex, and they wanted a more practical approach. The facilitators gave this feedback to the scientists, and assisted with simplifying the method. This flexible approach resulted in 36% of the growers in the group using the technique by the end of the growing season.
The use of recognised design and implementation method of the program played an important role in the success of Grapecheque. Quantitative and qualitative evaluation data were powerful tools for planning, developing and reporting on the program. Using Bennett's Hierarchy to plan the Grapecheque program clarified its objectives. It was a particularly valuable exercise for the facilitators, as the participatory approach in planning meant they had strong ownership of the program and were clear about what they were trying to achieve.
Quantitative data such as "how many participants, what have they changed" have demonstrated that winegrape growers have altered practices as a consequence of their involvement in the Grapecheque program. However, the program lacked a control group of non participants — this oversight has been addressed in Grapecheque mark 2.
The groups examined a variety of complex issues including pest and disease management, control of grape quality, irrigation management and rootstock/scion interactions. The methods adopted by Grapecheque fostered progressive learning. As the groups tackled their chosen issues systematically, demands for complex information from and the depth of questioning of presenters indicated that the understanding of each issue had increased.
The group environment fostered learning and confidence amongst growers. This combination was critical for accelerated adoption of new practices by winegrape growers. The groups evolved over the first three years of the program. In the first two seasons, growers wanted information on technical matters, and were happy to rely on the Facilitator to direct sessions. In the third season, they became much more pro-active, seeking to benchmark vineyard sites, management practices and winestyle between local vineyards. Participating growers were also keen to begin experimenting with new technologies to test their regional appropriateness.
Lessons learnt from evaluation and planning
The program would have benefited from having a control group of non participants against which practice change amongst participants could be referenced. Observation about differences in the information demands of groups within and between areas need to be quantified so that analysis is more rigorous. Different production issues faced in each region mean that direct comparison of practice change has to be understood in terms of local priorities. Facilitators and presenters need to be flexible in their approach to running the groups, and in delivering information to them.
The wine industry is satisfied with the program. At the 1999 industry R&D planning session the wine industry representatives wanted to see continued support for Grapecheque. The funding provider (State Government) is pleased with the way the program is running, and sees it as a model for other horticultural extension activities in the state (Buchanan, pers. comm.).
Education within the industry is regarded as a key to developing and maintaining Victoria’s competitive edge in the world wine market. The Grapecheque program offers growers recent research results packaged in various ways, such as multimedia presentations, handouts, hands-on experience in the laboratory and in the field, and most important of all, the opportunity to interact closely with the people who have undertaken the research. Scientists and viticulturists especially welcome this opportunity for interaction with end-users as it gives them the opportunity to find out first-hand the problems experienced by growers implementing research in the vineyard. This in turn provides researchers with the opportunity to give feedback on how research results can be more effectively used, and to develop or refocus research projects so that they are more relevant to the growers’ needs.
References
- Bennett C. & Rockwell K. (1995) Targeting Outcomes of Programs (TOP): An Integrated Approach to Planning and Evaluation. Draft.
- Buchanan G., Walker R., Bevington K. & Davies G. (1995) Rethinking Technology Transfer. A report of a workshop held at Mildura on 9-10 December 1993. Agriculture Victoria and Riverlink, Mildura.
- Dart, J., Petheram, R. J. & Straw, W. (1998) Review of Evaluation in Agricultural Extension, Rural Industries Research and Development Corporation, Canberra, Australia.
- Hargreaves, W.A.& Attkisson, C.C. (1978) Evaluating Program Outcomes In Attkinson, C.C. Hargreaves, W.A., Horowitz, M.J. and Sorenson J.E. (Eds.), Evaluation of Human Services Programs. New York: Academic Press.
- Marks, N. (2000) A Qualitative Evaluation of Grapecheque, June 2000
- Pywell, M. (1999) Compilation of statistical data on Australian and Victorian Grapes ExpHORT 2000 Publication No.60, Department of Natural Resources and Environment, Mildura, Australia.
- Van den Ban, A.W. and Hawkins, H.S (1996) Agricultural Extension. 2nd Edition. Blackwell, Berlin, Germany.
