A cloze test is an evaluation test, consisting of a portion of text from which some words have been removed to measure the ability to understand the text itself. That’s very useful nowadays with digital products. We’ve been learning how to test design, but what about testing content? how to test the copy and micro-copy of an app or website?
For NNG: “Cloze Tests provide empirical evidence of how easy a text is to read and understand for a specified target audience — measuring reading comprehension, not just a readability score.”
[This article is taken from one of our lessons. If you want to learn UX Writing and start a career in this field, sign up for our UX Writing course]
A close test it’s something like this:
Table of Contents
Why we need a cloze test?
Everything stems from the concept of readability and comprehensibility.
The difference is subtle.
- Readability is the ease with which a reader can understand a written text. The complexity of the vocabulary and syntax generally affects the reader’s ability to accurately understand the text. The typographic presentation (font size, line length) also has a considerable influence on readability.
- Comprehensibility is how a reader can process a text, get a meaning from it, and include it in a larger context.
The difference between readability and comprehension can be summarized as: a sentence is legible if you can read the words, but it’s only understandable if you can understand the meaning and purpose of those words.
How can we understand when content is understandable and readable?
We run the cloze test.
How to do a cloze test? (generic)
- Recruit attendees within your target demographic (your personas).
- Test at least 10 participants to increase the significance of the test.
- Select a text with clear keywords.
- Both paper and digital texts can be used with the possibility for the user to select words.
- Time limits are optional but they are useful for understanding the time of understanding the text.
- Guide the participant to write only one word in each blank and try to fill in all the spaces.
- Guessing can work well because it helps us understand if many participants understand a sentence differently from the original intent.
- Warn users that spelling and punctuation don’t matter.
- To perform a cloze test we have to select a text, the ideal would be about 125-250 words and replace an empty space every 5 words. UX Writers working with microcopy will need to use a smaller number.
- Ask the user to read the modified text and insert the missing words in the blanks using multiple choice answers (objective cloze test) or open answers (subjective cloze test).
Usually, a cloze test is open; attendees can enter any words they think fit the blank.
In the example below, we have decided to try a closed cloze test in which we will provide a list of options to choose from.
For each word, we have provided a list of options to choose from.
Participants could also insert words that they considered most appropriate even if not present in the list.
How do you know which word is best suited to the context?
According to NNG, a word can be defined correctly in a cloze test if it is used by at least 60% of the participants.
If users on average get 50-60% or more of the words right it can be assumed that the text is reasonably understandable for the specific user profile used.
Calculation of results
For each empty space, we draw up a list of words and calculate the percentage with which each word was chosen by the participants.
The word that has the highest percentage is the most suitable to be inserted in the text.
We have to try and try again until we find the word that has at least a percentage of 50-60% of consensus among the participants.
We can also use colors to distinguish the words that pass the test:
- green: 61-100%
- yellow: 31-60%
- red: 0-30%
Cloze Test applied to UX Design
[This article is taken from one of our lessons. If you want to learn UX Writing and start a career in this field, sign up for our UX Writing course]
If you are looking to test marketing and product content, we recommend that you modify a cloze test to fit the context of the web page.
- You can evaluate how typographical attributes (line-height, size, line break) affect understanding.
- You can evaluate how supplemental visuals, colors, and graphics help or harm understanding.
- You can see how understanding varies by device type: desktop web, mobile web, and native mobile.
How Cloze testing works in UX Design
The same as a traditional cloze test (above) with some considerations:
- You can choose the user to speak aloud or silent reading. Speaking aloud could provide more insights, but it could also make the participant nervous and hesitant. A silent reading offers a more natural reading experience, but will not provide information on the participant’s reasoning.
- The participant must observe the content in the context of the web page.
- The web page does not need to be live, but it can be a static copy with one word omitted every few terms. Make sure you focus on your product keywords.
- Tell the participant to fill in the missing words, with or without a word list.
- Make sure the participant feels comfortable and relaxed, it should not be a stressful activity. It’s also okay not to get 100%.
- You can also add an optional explanation at the end of the test to learn more about the participant’s general thoughts on the content: i.e. what confused you?
Calculation of results in UX Design
The same as a traditional cloze test but with some considerations:
- You could also evaluate other factors such as the speed of comprehension, qualitative evaluation of the participants’ reasoning, and the impact of different content variants on comprehension.
[This article is taken from one of our lessons. If you want to learn UX Writing and start a career in this field, sign up for our UX Writing course]
Extra Resources
- Join our FB Exclusive Group to get access to extra resources, it’s FREE.
