We can use easy discourse measures to demonstrate improvement that may not be detected by standardized measures. I don’t know about you, but I find it frustrating to know that my patient is improving, but their scores on the BDAE or WAB don’t change much.
Free DIRECT download: Discourse measures scoring sheet (cheat sheet). (Email subscribers get free access to all the resources in the Free Subscription Library.)
Outline:
- Why measure informativeness and efficiency?
- What are the 6 easy discourse measures?
- Obtaining a discourse sample.
- How to obtain the 3 descriptive measures.
- How to obtain the 3 calculated measures.
- Lack of clear norms?
- Try measuring informativeness or efficiency.
- Related Eat, Speak, & Think posts.
- References.
Why measure informativeness and efficiency?
Informativeness is the amount of information conveyed, and efficiency is how concisely a person conveys that information.
Discourse is verbal or written communication used for some purpose. We engage in discourse when we ask and answer questions, tell stories, give instructions, describe events, share our point of view, argue our case, and so much more.
When we work with patients who’ve lost their ability to freely communicate in a normal manner, their main concern is typically to regain their ability to engage in discourse. Often, unfortunately, a full return to normal communication isn’t possible.
In those cases, our goal is to help them to communicate their intended messages in the best way possible. We provide direct intervention to improve their communication ability, as well as teach strategies to assist them to convey their message when the words don’t flow.
Standardized test batteries often don’t capture the improvements our patients make in their functional communication skills. They don’t provide discourse-level measures that are sensitive to change. (Wright, n.d.)
So why should we measure informativeness and efficiency of discourse? Because we communicate at the level of discourse. When people lose their ability to communicate normally, they lose their ability to communicate meaningful information in a socially-appropriate concise manner. Standardized assessments don’t directly measure this.
What are the 6 easy discourse measures?
While there are many discourse measures, I’m focusing on the six from Nicholas and Brookshire (1993). They defined a system to quantify the informativeness and efficiency in connected speech, or discourse. Their paper is available to ASHA members for free and contains a thorough tutorial with examples, as well as the picture stimuli used to elicit discourse samples.
Once we have a discourse sample, we can obtain the following measures:
- Number of words (#words).
- Number of correct information units (#CIUs).
- Length of speaking time (Time).
Using the above three direct measures, we can calculate the following measures:
- Percentage of words that are correct information units (%CIUs).
- The number of words per minute (WPM).
- The number of correct information units per minute (CIUs/minute).
%CIUs measures informativeness
A correct information unit (CIU) is a word that is intelligible, accurate, relevant, and informative. So each CIU is a word, but not all words are CIUs.
If a person speaks clearly and on topic, then they will have a high percentage of CIUs. Most of their words will be accurate, relevant, and informative.
On the other hand, if a person has word retrieval difficulty or tends to wander off topic, they’ll have a low percentage of CIUs. Many of their words will be inaccurate, not relevant, or non-informative.
Once you determine which words in the sample count as CIUs, it’s easy to calculate the percentage of CIUs (%CIUs) compared to the total word count.
CIUs/minute and WPM measure efficiency
Regardless of how informative a person is, they could be efficient or inefficient in their communication. A person who speaks quickly is efficient, while one who speaks slowly or has excessively long pauses is inefficient.
Once you know the number of words in a discourse sample and the time it took the person to say those words, it’s a simple matter to calculate words per minute (WPM).
After you’ve calculated the %CIUs, it’s a simple matter to calculate the percentage of CIUs per minute (%CIUs/minute).
Obtaining a discourse sample
The first thing to do is to obtain a discourse sample. There are many ways to do this, and you can choose how you want to go about it. It’s good to be aware that many things can affect the sample, including the task, the stimuli, the instructions, who’s present, and the setting (Doyle et al., 1995).
Nicholas and Brookshire (1993) collected ten samples from each participant during structured discourse tasks. You could choose to divide this list and use half for verbal discourse and half for written discourse (Edmonds, n.d.).
- Describing 4 individual pictures.
- Narrating 2 picture sequences.
- 2 personal narratives.
- Describing 2 procedural tasks.
They used the picnic scene from the WAB and the cookie theft scene from the BDAE, which are available online. They include in the Appendix the other two individual pictures and the two picture sequences.
The two personal narrative prompts were “Tell me…”:
- What you usually do on Sundays.
- Where you live and describe it to me.
The two procedural narrative prompts were “Tell me how you would go about…”:
- Doing dishes by hand.
- Writing and sending a letter.
I described five tasks in addition to those above that we can use to elicit discourse samples in How to assess pragmatic skills in adults with brain injury. Some of these tasks are more open-ended conversational discourse tasks.
In clinical practice, we also have to be efficient. Researchers such as Heidi Iwashita and McKay Moore Sohlberg (Instagram) are actively working to determine best practice for eliciting and analyzing discourse samples. In the meantime, we should at least be consistent between test and re-test.
How to obtain the 3 descriptive measures
You don’t have to report all six measures. Choose the measure(s) that match your therapy goals: informativeness and/or efficiency (Wright, n.d.).
Number of words (#words)
First, omit any comments your patient makes before or after their actual performance of the task. Also omit any non-words and unintelligible words.
Then count the words. All word processors likely have a word count tool. You could also copy the text into an online word count tool, such as WordCounter.net.
Number of correct information units (#CIUs)
Eliminate any word that is not intelligible, accurate, relevant, and informative. That sounds straightforward, but it can be tricky to decide what to count as a CIU.
Nicholas and Brookshire (1993) provide a detailed account of what does and does not count as a CIU, with examples.
What Nicholas and Brookshire (1993) count as a CIU
Here’s a summary of what Nicholas and Brookshire (1993) consider to be a CIU. You can find the full explanation along with examples in their appendix.
- A CIU doesn’t have to be grammatically correct.
- A CIU can be a paraphasia, as long as the production would be intelligible as the target word in context (ex. “school” for “stool”).
- Only consider the final attempt in a series of attempts to correct a sound error.
- If several people are involved, but the patient only mentions one, do count it.
- Count informal terms if they convey information about the content of the picture or topic (ex. “Yup”, “nope”, “uh-huh”, “un-uh”).
- Embellishments, if they add to the events portrayed or express a moral.
- Words that express legitimate uncertainty.
- An auxiliary verb and the main verb count as two separate CIUs.
- Contractions as two separate CIUs.
- Count each word separated by hyphens as separate CIUs.
A summary of what does not count as a CIU
And here is a summary of what they say does not count as a CIU. You’ll find the full description and examples in the appendix in Nicholas and Brookshire (1993).
- Words that are unintelligible or not relevant.
- The word “and” is never counted as a CIU.
- Grammatically incorrect words if they lead to misunderstanding or uncertainty about the meaning.
- Repeated attempts to produce a word (only the final attempt is considered).
- Dead ends, false starts, or revisions that are uninformative.
- Repetition of words or ideas that don’t add new information and aren’t necessary for cohesion or grammatical correctness.
- First use of a pronoun if the antecedent hasn’t been established.
- Vague or nonspecific words that aren’t necessary for grammatical completeness if a more specific word or phrase could have been used.
- Conjunctions, qualifiers, and modifiers used indiscriminately as filler words (ex. “so’, “then”, “I think that”, “it looks like”, “apparently”, “of course”, “sort of”).
- Filler words, interjections, and tag questions.
- Commentary about the task, the patient’s own performance, or personal experience.
Duration of sample (Time)
To measure how long it took your patient to speak, you’ll want to omit any time you spent providing instructions or cues. Also omit any statements your patient made before or after actually doing the task.
It’s easy enough to calculate the duration if you record the sample. You’ll know how long the recording is. You’d just need to subtract the time associated with you speaking or your patient making extraneous comments at the start and end.
How to obtain the 3 calculated measures
Percentage of correct information units (%CIUs)
Take the number of CIUs and divide by the number of words. Then multiply by 100.
In formula form:
%CIUs = #CIUs/#words x100
For example, if the sample is 302 words and 104 of them are correct information units, then %CIUs would be 34.2%. This is 104 divided by 302, multiplied by 100.
This means that 34% of the patients words were informative in that sample.
Words per minute (WPM)
If you’re lucky and the duration of the sample (Time) is in even minutes, you simply divide the number of words by the number of minutes. This is #words/number of minutes.
If our patient in the example above took exactly 3 minutes to produce 302 words, their rate would be 100.7 words per minute. (302 words/3 minutes = 100.7 words per minute.)
But chances are your measure of duration will include some leftover seconds. In this case, you’ll need to convert the minutes to seconds to do the calculation.
- Convert Time into seconds.
- Take the minute part of your measure of duration and multiply by 60.
- Then add your leftover seconds.
- Divide the number of words by the number of seconds. Now you have words per second.
- Then multiply by 60 to convert to words per minute.
For example, if our patient took 3 minutes and 10 seconds to produce the sample:
- 3 *60 = 180 seconds + the 10 leftover seconds = 190 seconds.
- 302 words / 190 seconds = 1.59 words per second.
- 1.59 * 60 = 95.4 words per minute.
CIUs per minute (CIUs/minute)
We calculate CIUs/minute in the same way as WPM. Simply substitute the number of CIUs for the number of words.
In our example, our patient produced 104 CIUs. If Time is exactly 3 minutes, then our patient produced 34.7 CIUs/minute.
If Time is 3:10, then we would calculate 104 CIUs / 190 seconds = 0.55 CIUs per second in step 2 above. Then we’d multiply by 60 to get 33 CIUs per minute.
Lack of clear norms?
I would dearly love a clear list of norms for these discourse measures. I’ve read a handful of papers and taken two continuing education courses to write this post, and I haven’t found clear answers for what is considered normal. In a way, this isn’t surprising given that there are so many factors that can influence the results.
But here are some guidelines:
WPM norms
I’ve seen the average speaking rate for conversations to be between 120 – 150 wpm.
Nicholas and Brookshire (1993) found that their group of 20 non-brain-damaged adults spoke at a rate of 105 – 202 WPM. And their group of 20 people with aphasia spoke at a rate of 12 to 150 WPM.
Joseph Duffy suggests 150 – 200 wpm for reading aloud, when he says we can expect a normal speaker to read the 115-word Grandfather Passage aloud in 35 to 45 seconds. (Duffy, 2005, p. 105)
%CIU norms
Heather Harris Wright provided a chart in Improving Functional Outcomes in Aphasia that summarized findings of three studies. Looking across studies, non-brain-damaged adults produced 55% – 98% CIUs, with a mean of 88.5% in one study. People with aphasia produced 22% – 85% CIUs, with a mean of 64.6% in one study. (Wright, n.d.)
CIUs/minute norms
In the same chart as above, health adults produced 61 – 209 CIUs/minute. People with aphasia produced 10 – 109 CIUs/minute. (Wright, n.d.)
So there’s overlap between what’s “normal” and what’s impaired.
That can be frustrating, but it’s part of skilled assessment to identify if a problem exists and then characterize that problem. In addition to the measures above, we can document a language impairment from a standardized test and/or a functional communication questionnaire such as the ACOM or the CETI.
Try out one or more of these easy discourse measures!
The next time you collect a language sample, why not try calculating one of these easy discourse measures? Let me know how it goes!
Related Eat, Speak, & Think posts
References
- Doyle, P. J., Goda, A. J., & Spencer, K. A. (1995). The Communicative Informativeness and Efficiency of Connected Discourse by Adults With Aphasia Under Structured and Conversational Sampling Conditions. American Journal of Speech-Language Pathology, 4(4), 130–134. https://doi.org/10.1044/1058-0360.0404.130
- Duffy, J. R. (2005). Motor speech disorders: Substrates, differential diagnosis, and management. St. Louis, Mo: Elsevier Mosby.
- Edmonds, L. A. (n.d.). Word Retrieval Training with Nouns and Verbs: Treatments for Facilitating Generalization. Part of the eWorkshop: Aphasia Treatment: Maximizing Functional Outcomes. ASHA Store: https://bit.ly/2D1l2vl.
- Nicholas, L. E., & Brookshire, R. H. (1993). A system for quantifying the informativeness and efficiency of the connected speech of adults with aphasia. Journal of speech and hearing research, 36(2), 338–350. https://doi.org/10.1044/jshr.3602.338
- Wright, H. H. (n.d.). Moving Beyond Single Words: What We Can Learn from Discourse. [Seminar]. Part of the eWorkshop: Aphasia Treatment: Maximizing Functional Outcomes. ASHA Store: https://bit.ly/2D1l2vl.
Free DIRECT download: Discourse measures scoring sheet (cheat sheet). (Email subscribers get free access to all the resources in the Free Subscription Library.)
Lisa earned her M.A. in Speech-Language Pathology from the University of Maryland, College Park and her M.A. in Linguistics from the University of California, San Diego.
She participated in research studies with the National Institute on Deafness and other Communication Disorders (NIDCD) and the University of Maryland in the areas of aphasia, Parkinson’s Disease, epilepsy, and fluency disorders.
Lisa has been working as a medical speech-language pathologist since 2008. She has a strong passion for evidence-based assessment and therapy, having earned five ASHA Awards for Professional Participation in Continuing Education.
She launched EatSpeakThink.com in June 2018 to help other clinicians be more successful working in home health, as well as to provide strategies and resources to people living with problems eating, speaking, or thinking.
Be First to Comment