Previous PageTable Of ContentsNext Page

Evaluation of written extension texts

T.F.Smith

Agriculture Protection Board, Baron-Hay Court, South Perth, W.A. 6151.

Although a motivated reader may plough through obscure prose", others will reject unread what they cannot easily understand. Most extension texts so far examined are too unreadable for the average farmer"'. This paper discusses ways of measuring the readability of a document before publication.

1 Readability formulae

The only evaluative Methods in common use are readability formulae, for example: the Flesch Reading Ease Score, Dale-Chall Formula, Fog Index and Fry's Readability Graph. These are based on correlations between word length (or frequency) and sentence length. However shortening sentences only makes them more readable if sentence structure is also simplified"', and short words are not always easier to read than long words. Readability formulae provide a quick and crude guide to readability but are no help in text rewriting.

2 Close procedure

In the cloze procedure every nth word is deleted from a text... Readers then read the mutilated text, filling in the missing word blanks. The proportion correctly filled is an estimate of text readability. Validity has been established using multiple choice tests as criteria"'. Cloze can be used to estimate readability of a text by a specific group of readers. Its disadvantage is that it measures word redundancy"' rather than readability per se. and the relationship between sentences has little effect on scores''. Moreover, cloze tests of hard passages may cause readers to give up in frustration (pers. observation). The cloze proceedure is a useful way of measuring readability of a text by a given readership. However, its poor ability to respond to whole text structure and unacceptability to many readers limits its value in extension text evaluation.

3 Sentence verification test (SVT)

This test consists of random text sentences modified in four different ways:- unchanged; paraphrased; meaning changed and different topic'''. After reading the text, subjects say whether the test sentences contain new information. The number of correct choices is their SVT score. This is sensitive to text readability and reader's skill and background knowledge'''. SVT is well accepted by subjects, relatively easy to construct and objective. It requires further comparison with established Methods of measuring comprehension, but looks likely to provide a valuable and acceptable tool for measuring the understanding of farmer audiences.

4 Reading speed

The length of time the reader spends looking at an individual word depends on the time taken to understand it in context(10). Therefore a text which takes longer to read is less readable. Single words or sentences may be presented on a computer screen and changed when the reader presses a key. Varied levels of attention or interest may affect reading time, therefore subjects with a strong "set to learn" produce more reliable reading times'"'. Reading speed is easy to measure on a computer and it is sensitive to text coherence and sentence structure. However measuring techniques require further development to define the most reliable test conditions.

5 Recall

Subjects may be given a text to read and their comprehension deduced from a written summary. Recall summaries are difficult to mark objectively and tests of this type are not always acceptable to adult subjects. However, recall may be predicted from an analysis of text structure, giving the test validity'".

1. Gunning R. (1952) The Technique of Clear Writing. New York: McGraw-Hill.

2. Paul, L.C. (1970). OECD Review, 17, 56 - 62.

3. Hull, L.C. (n of technical writing. 26th Tech. Comn. Conf., Los Angeles.

4. Taylor, W.L. (1953). Journalism Quart. 30, 415-433.

5. Bormuth J.R. (1968). J. Educ. Measurement, 5, 189 - 196.

6. Klare, G.R., n. (1972). Int. Rev. Appl. Psychol., 21, 77 -106.

7. Kintsch, W., and Yarbrough, J.C. (1982). J. Educ. Psychol., 74, 828 - 834.

8. Royer, J.M., Reading Hastings, C.N., and Hook, C. (1979Behavior, 11, 355 - 363.

9. Just, M.A., and Carpenter, P.A. (1980). Psychol. Rev., 87, 329-354.

10. Klare, G.R., Shuford, E.H., and Nichols, W.H. (1957). J. Appl. Psychol., 41, 222 - 226.

11. Miller, J.R. and Kintsch, W. (1980). J. Exp. Psychol.: Human Learning and Memory, 6, 335 - 354.

Previous PageTop Of PageNext Page