Practice Testing
Introduction
One of the most effective ways of studying is to utilise the so-called testing effect. This refers to the fact that quizzing yourself on previously learnt material promotes greater retention than simply restudying the same material for an equivalent amount of time [1]. In other words testing not only assesses learning, it enhances it [2]. Some articles refer to a retrieval effect, in the vast majority of cases the testing effect and the retrieval effect are essentially the same thing.
Practice testing is thought to work because retrieving information strengthens memory. If no feedback is provided and the success rate of retrieval is too low, then testing might not be effective. However, this is easily solved by providing feedback. As long as the correct answer is provided then practice testing will be effective [3].
If a student can’t answer the question in a practice quiz, it’s unlikely they can answer the question in a real quiz. Repeated practice quizzing with feedback enables errors to be corrected and enhances retention. Most students aren’t good at judging how well they have mastered a given topic. Not only does retrieval practice help students learn material, at the same time it provides accurate feedback regarding how well they have mastered it.
A review of 50 classroom studies by researcher Pooja Argawal and associates found consistent evidence of a testing effect. 35% of experiments showed a small effect, 24% a medium effect, and 33% a large effect. The remainder showed a small or negative effect [4]. The testing effect has been shown to be effective with a variety of study materials including word lists, text passages, and nonverbal materials [5].
In addition, practice testing has been found to be more effective than other study methods such as concept mapping, restudying, and note-taking [6]. The testing effect has been found with a variety of test formats including multiple-choice, fill-in-the-blank, short-answer, cued-recall, and free-recall [7]. So regardless of how the material is tested, a learning benefit still results.
|
Tip
|
Practice testing with feedback is one of the most effective study methods. |
Practice testing isn’t just about "learning the test". Testing not only helps retention of directly tested content, but learning does actually transfer to similar but not identical items [6, 8, 4]. The mechanism for the testing effect isn’t entirely understood. One explanation is improved motivation, essentially students are dissatisfied by poor performance on practice quizzes, so make more effort to improve on subsequent tests.
|
Tip
|
Practice testing with feedback goes beyond "learning the test" and assists with retention of related material. |
Despite practice testing being one of the most effective study strategies, it is not used by the majority of students. Karpicke and associates found that repeated reading was the most common technique (84%), with 55% ranking it as their number one strategy. 43% answered practice problems, 40% used flashcards, and 11% practiced retrieval [9]. Although practice problems and flashcards can be a form of self-testing, this isn’t always true. For example, students can use flashcards to just present themselves with a series of facts rather than use a question and answer (Q&A) type format. Students are largely unaware of the benefits of self-testing. If they do test themselves, it is mainly to check the status of their learning rather than to enhance retention. Even if students want to use other study methods, it’s beneficial to combine them with testing. Hence it is recommended that students alternate between testing and other study techniques [2].
Some laboratory experiments found larger effect sizes when the delay between practice quizzes and the final test was longer. In contrast, a review of classroom studies found larger effect sizes for shorter delays (one - three days). This is useful for students as it aligns with the normal study practices of students [4]
|
Tip
|
When preparing for a test or exam, try quizzing yourself on the material every one to three days. |
Adding a series of questions to any notes made can be a useful strategy. One note-taking system that incorporates this is the Cornell note-taking system which is discussed elsewhere in this book. These questions can then be used as part of a later revision strategy. Another popular way to incorporate practice testing is using flashcard software.
Admittedly, classroom studies that show evidence of a testing effect typically compare it with weak study strategies like rereading or highlighting. When it is compared to stronger strategies not every study shows evidence of the testing effect [10]. Yet no other study strategy is currently supported by more evidence. That doesn’t mean that students should exclusively use testing, especially if they find success with other techniques like concept mapping, but it is recommended to use it as part of an overall study strategy.
|
Note
|
No other study strategy has as much support as the testing effect. |
How many times should I test each item in a practice test?
Generally speaking the more times an item is retrieved the better the learning [6]. However, it is a case of diminishing returns, with one study finding sizeable gains in final test performance for the first few correct recalls in practice, but after that showing minimal gains [11].
This finding was expanded by a series of experiments by researchers Rawson and Dunlosky which investigated the optimum number of correct recalls for students. Participants were required to learn the definitions of key psychology terms such as "Sensory memory is a memory system that retains large amounts of sensory input for very brief periods of time". Three correct recalls during the initial session enabled faster learning in later study sessions. To avoid any ambiguity, three correct recalls doesn’t mean an item was studied three times, it means study an item as many times in a session as necessary to get the item correct three times. Learning to a criterion of three correct recalls took six trials on average. Once a certain number of correct recalls is reached, additional correct recalls don’t really help much. The researchers found four correct recalls during the initial learning session was no better than three.
The results of the experiments led to a recommendation of a "3+3" study schedule. This means in the initial learning session keep practicing until an item is recalled correctly three times. Then study the item again in three subsequent study sessions, learning to one correct recall in each session. This is suitable for students with short-term retention goals for a final exam, while also supporting long-term retention [12].
|
Tip
|
Researchers recommend a 3+3 study schedule. In the first study session, keep studying until you can recall an item correctly three times. Then study the item again in three separate study sessions until you get it right at least once in each session. |
Can I drop an item from practice testing once I get it right?
It is commonly believed by students that once an item can be successfully recalled, it can be dropped from practice. The time can then be devoted to other items. But research suggests that dropping those items from testing reduces long term retention. In other words, if it is important to remember an item, keep testing it even if you get it correct. One study trained undergraduates on Swahili vocab. Students were trained on the vocab until an item was correctly recalled once. After that, one group restudied each item and another group was retested on each item. The continued testing group did much better than the continued study group on a final test one week later [13]. To be clear, the continued study group were still studying the item, they just weren’t being tested on it. This highlights the value of the testing effect.
|
Tip
|
Keep testing all items. Don’t remove an item from testing even if you get it correct. |
What kind of topics and formats is the testing effect useful for?
Positive testing effects have been found for a wide variety of topics. Students in a university anatomy and physiology course retained information from a passage better when incorporating self-testing versus other approaches where they just reread or read and took notes [14]. Similarly, Batsell and associates compared two groups of students in an introductory psychology course, one group did daily readings and completed a multiple-choice quizzes on the readings, the other group just did the readings. The students in the quiz group performed better in a final exam even when the questions were entirely new or just similar to the ones in the earlier quizzes [8].
Robust evidence exists for the testing effect across different formats, including those students commonly encounter such as foreign-translation word pairs, text passages, and lectures [6].
|
Note
|
The testing effect works for a wide variety of materials. |
If I suffer from test anxiety will practice testing still work for me?
Test anxiety is quite common. There is some evidence that when the practice tests involve high stakes, students don’t learn as well. For instance, one study by Hinze and Rapp had participants complete both high stakes and low stakes quizzes before measuring how they performed in a final test. Those in the high stakes group performed worse. Moreover, only the low stakes group performed better than a group that performed rereading instead of quizzes [15]. On the other hand a recent review of studies found no difference between high and low stakes quizzes [6].
Generally speaking though, students using practice testing will be doing it at home or in a somewhat relaxed environment like the library, so it qualifies as low stakes testing. Regardless of any evidence that high stakes practice testing isn’t quite as useful, this won’t affect most students who are using it as a study technique.
|
Tip
|
Even those with test anxiety can still get benefits from practice testing. Most practice testing is low stakes so won’t provoke anxiety like a test that forms part of a course assessment. |
How does practice testing compare to other study methods?
Students in the university environment need to learn a large amount of information, so it is important to use study time efficiently. While students can use a variety of study strategies such as re-reading, highlighting or concept mapping, a review of research found taking practice tests boosts learning and retention compared to other popular strategies [16].
Does the type of practice test matter?
Practice testing has been shown to enhance retention using a variety of formats including multi-choice questions, fill-in-the-blank questions, short-answer questions, cued-recall, and free recall [6, 10]. Additionally, some research has shown that practice testing works even when the practice tests are open book. Nevertheless, the evidence does point to simpler types of practice tests, like fill-in-the-blank or multichoice, as not being as effective as more generative responses such as short answer or recall [17]. A free recall test involves retrieving information about a certain topic without any cues or prompts. A cued recall test involves retrieving information given some sort of prompt.
For students preparing for a particular style of test, research has also shown that the format of the practice test doesn’t need to match that of the final test although there is evidence of a superior effect when the two match [6, 4]. In practical terms, the simplest approach is to design your practice tests to be similar to the style of question the university uses.
|
Tip
|
Practice tests that involve generative responses such as short answer or recall may be more effective in general. In practical terms, it makes sense to practice answering questions of the same style that the university uses in its tests and exams. |
Another key finding is that the more practice testing the better. However, this practice testing should be spaced out rather than all in one session, even if the total time invested remains the same, with longer intervals between testing being better. There is no strong evidence that a particular spacing interval such as fixed or expanding is superior.
Where do I get the questions?
If available, one of the simplest and most logical sources of questions are those from previous semester offerings of the same course. This includes tests, quizzes, exams, assignments and so on. A course typically doesn’t vary much between offerings, and this approach will guarantee good coverage of the material. The lecturer might also provide sets of questions. Also, students who have previously taken the course may share questions. Another possible source is the prescribed textbook.
Sometimes universities will introduce a new course, if this is the case look for similar offerings at other universities, as well as the textbook, for a source of questions. Being realistic though, a new course is rarely completely new, instead it will often be a reorganisation of material from previously existing courses.
Another possibility is to make your own questions. This can also be useful, but the downside is they might not give good coverage of the exam material. Moreover, students new to the material may not understand which ideas are the most important.
|
Tip
|
There are many sources of questions for practice quizzing. Be careful about inventing your own questions as you may emphasise the wrong topics. |
Flashcards
One of the most obvious approaches to self-testing is to use flash cards. There’s a variety of free or low cost software available. If a student isn’t interested in making their own flashcards a variety of premade decks can often be found online. Flashcards are discussed in more detail in the chapter on flashcards.
Is it important to create my own practice questions?
Obviously, if you have no other source of questions, you have no option but to generate your own questions. As mentioned previously though, it’s relatively common though to be able to access questions from exams and tests from previous semesters. Similarly, the lecturer might provide sets of practice questions, or they might be available from the textbook. Another option is to look for similar courses online and use those as a source of questions.
Education professionals will often recommend students create their own practice questions believing that it encourages engagement with the material on a deeper level. Ask other students to share their study materials, and they will give similar reasons why you should generate your own questions. It’s possible though that they also have other motives such as not wanting you free-riding off their efforts or getting an advantage in a competitive course. The obvious problem with creating your own questions is the time taken, also you might find the material challenging and struggle to create high quality questions.
One study by Weinstein and associates compared three strategies: re-reading, answering provided questions, and generating and answering their own questions. They found that answering questions was superior to re-reading. However, there was no difference in final test performance between answering questions provided by the experimenter and answering self-generated questions. In addition, generating questions needed approximately three times as much time [18].
A study by Myers and associates sheds additional light on this. They found that self-generating questions performed worse than restudying or answering provided questions. This was likely due to only having a fixed amount of time in all three conditions. For self-generating questions to be beneficial there needs to be adequate time to generate high quality questions [19].
|
Tip
|
Don’t generate your own questions unless you have no choice. There’s no large benefit, especially accounting for the additional time required. Notably, don’t be afraid to use pre-existing decks of flashcards if available. |
A crucial factor is how well the practice questions overlap the material covered in the final test. Students can have trouble distinguishing between important and unimportant material when generating their own practice questions. Questions that focus on the wrong material will be little use if the goal is to do well in a test or exam.
To sum up, there is no need to spend a lot of time making up your own questions for testing if suitable questions can be obtained from tests or exams from previous semesters or other sources. Often the style of questions remains the same over many years, sometimes the exact same questions are reused.
How much time should I leave before I retest the same item?
Consistent with what we would expect, another key finding is that more practice testing is better. Furthermore, this practice testing should be spaced out rather than all in one session, with longer intervals between testing being better. Spacing the practice sessions provides a benefit even when the total amount of time is fixed. Additionally, there is no strong evidence that a particular spacing interval such as fixed or expanding is better.
Since the testing effect was found for a variety of timings around practice tests as well as for differing delays between the last practice test and the final exam, it is recommended to incorporate regular practice testing but not to obsess about the exact timing [4].
Is an open book test still beneficial?
Some readers might be wondering if they can still benefit from the testing effect with open book testing. A study by Agarwal and associates found that both open-book and closed-book testing enhanced retention and that neither was superior to the other, but both were superior to restudying [20]. Despite this study, based on theoretical considerations, it is likely that closed-book testing is superior to open-book. If however you are struggling with closed-book testing early in the learning process, benefits can still be obtained from open-book testing.
|
Tip
|
Even open-book practice testing can be helpful if closed-book testing is initially too difficult. |
My preferred study strategy is note-taking how does it compare with testing?
A common study strategy when initially learning a topic is to make notes from the lectures or textbook. Some students like to then rewrite their notes one or more times. Researchers conducted an experiment that compared repeated reading, repeated testing, and repeated note-taking and measured the performance after several different delays. After a two-week delay, repeated testing was found to be superior to repeated note-taking, although they were equivalent after a one-week delay. Repeated reading was inferior to both. In a second experiment, three conditions were compared: note-taking with note-reading, note-taking with testing, and repeated testing. The conditions with testing lead to the best performance [21].
To summarise, it is fine to use note-taking as part of an overall strategy, but testing should always be included as well for optimal results. Many students rely on a strategy of note-taking plus note-reading which is outperformed by conditions that include testing.
|
Tip
|
Rewriting notes can be useful as part of an overall strategy, but incorporate testing for optimal results. |
Why does the testing effect work?
Readers may be wondering why the testing effect works. Although there is currently no definitive answer, a number of theories have been proposed [6].
-
Transfer-appropriate processing: the mental processing steps are similar between the final assessment and the practice quizzes. This produces bigger learning gains compared to when the final assessment and study technique don’t match.
-
Motivation: Frequent tests may expose gaps in knowledge and improve focus hence boosting learning. If students know they are going to be quizzed on material, they increase their effort.
-
Additional exposure: One factor contributing to the testing effect is additional exposure, although this is not the only reason as the testing effect is superior to other techniques like rereading.
Does the testing effect only benefit the learning of basic facts?
Some might claim that quizzing is only useful for learning basic facts. In contrast, the evidence shows that the testing effect also boosts conceptual learning and problem-solving [6].
Does the level of the test items matter?
Jensen and associates compared the effect of quizzes at different levels. Some students in an introductory biology course answered low-level questions that required only remembering basic facts. Another group answered high-level questions that involved "application, evaluation, and analysis". Those in the high level group performed better on both high and low-level questions in the final exam [22].
Does the testing effect still work with complex information?
One caveat about the testing effect is that the benefits may be reduced if the material is complex and the student doesn’t already have a reasonable understanding. This suggests that in the case of initially learning complex material, it’s best not to rely on quizzing alone. Instead, use other techniques to help enhance understanding first [7].
Start by studying plenty of examples, creating concept maps, and considering alternative presentations of the material. Sometimes it is the case that the course materials provided by instructors are too abbreviated and contain too few examples to assist students in understanding. If this is the case, seek out additional sources of knowledge like textbooks, course materials from other universities, or online videos. Universities often assume that their course materials do an adequate job of explaining a topic when that is not the case. This means students may need to invest additional time over what is commonly allocated per course due to the need to seek out and review additional learning materials.
|
Tip
|
If the material is complex, and you don’t understand it well, use techniques other than testing to first build a basic understanding. |
What are some key points for using practice tests?
Practice quizzing has been shown to be one of the most effective techniques for learning material. Some points to keep in mind when using practice tests include:
-
Focus on testing the most important ideas and concepts.
-
Free recall or short answer questions are superior to formats that simply require recognising the answer such as multi-choice. Multi-choice practice tests are still useful though.
-
Provide feedback regarding incorrect answers.
-
Detailed explanations as to why an answer is incorrect are better. Ideally have worked solutions available.
-
Try a 3+3 study schedule. In the first study session, keep studying until you can recall an item correctly three times. Then study the item in three more separate study sessions ensuring at least one correct recall in each session.
-
Save time by using existing questions rather than making your own. Making your own questions isn’t wrong but can be time-consuming.
-
If the material is complex, take time to understand the material before starting practice testing. Practice testing will help commit it to memory after the initial understanding phase.
Summary
Students may be reluctant to use self-testing as they are not aware of the benefits, and additionally, it takes more effort than alternative techniques like highlighting passages or re-reading. The value of self-testing goes far beyond that of a diagnostic tool. The act of retrieving the information from memory helps learn the information. Self-testing is far more effective as a study strategy than other more popular techniques as re-reading.
There is some debate as to the benefit of generating your own questions. There is also a trade-off since generating your own questions takes a substantial amount of time. Given a limited amount of time, it is better to spend the time answering questions than creating them. Generating and answering questions remains a viable option if a ready source of existing questions is not available. Most undergraduate courses follow fairly standard syllabuses, so questions should be relatively easy to source.
Some may be concerned that the testing effect is just due to practicing identical items to those that appear in the final exam. This is not the case as many studies involve rephrased questions when evaluating the benefits. In practice exams at universities tend to be relatively similar from semester to semester, this means using questions from previous semesters to help learn will be an effective approach. Whatever your preferred study technique, it’s recommended to add testing to your portfolio.
References
[1] A. C. Butler, “Repeated testing produces superior transfer of learning relative to repeated studying.,” Journal of Experimental Psychology: Learning, Memory, and Cognition, vol. 36, no. 5, p. 1118, 2010.
[2] J. D. Karpicke and H. L. Roediger III, “Repeated retrieval during learning is the key to long-term retention,” Journal of memory and language, vol. 57, no. 2, pp. 151–162, 2007.
[3] Y. Weinstein, C. R. Madan, and M. A. Sumeracki, “Teaching the science of learning,” Cognitive research: principles and implications, vol. 3, pp. 1–17, 2018.
[4] P. K. Agarwal, L. D. Nunes, and J. R. Blunt, “Retrieval practice consistently benefits student learning: A systematic review of applied research in schools and classrooms,” Educational Psychology Review, vol. 33, no. 4, pp. 1409–1453, 2021.
[5] C. A. Rowland, “The effect of testing versus restudy on retention: a meta-analytic review of the testing effect.,” Psychological bulletin, vol. 140, no. 6, p. 1432, 2014.
[6] C. Yang, L. Luo, M. A. Vadillo, R. Yu, and D. R. Shanks, “Testing (quizzing) boosts classroom learning: A systematic and meta-analytic review.,” Psychological bulletin, vol. 147, no. 4, p. 399, 2021.
[7] M. Trumbo et al., “Is the testing effect ready to be put to work? Evidence from the laboratory to the classroom.,” Translational Issues in Psychological Science, vol. 7, no. 3, p. 332, 2021.
[8] W. R. Batsell Jr, J. L. Perry, E. Hanley, and A. B. Hostetter, “Ecological validity of the testing effect: The use of daily quizzes in introductory psychology,” Teaching of Psychology, vol. 44, no. 1, pp. 18–23, 2017.
[9] J. D. Karpicke, A. C. Butler, and H. L. Roediger III, “Metacognitive strategies in student learning: do students practise retrieval when they study on their own?,” Memory, vol. 17, no. 4, pp. 471–479, 2009.
[10] B. F. T. Moreira, T. S. S. Pinto, D. S. V. Starling, and A. Jaeger, “Retrieval practice in classroom settings: A review of applied research,” in Frontiers in Education, 2019, vol. 4, p. 5.
[11] M. A. Pyc and K. A. Rawson, “Testing the retrieval effort hypothesis: Does greater difficulty correctly recalling information lead to higher levels of memory?,” Journal of Memory and Language, vol. 60, no. 4, pp. 437–447, 2009.
[12] K. A. Rawson and J. Dunlosky, “Optimizing schedules of retrieval practice for durable and efficient learning: How much is enough?,” Journal of Experimental Psychology: General, vol. 140, no. 3, p. 283, 2011.
[13] J. D. Karpicke and H. L. Roediger III, “The critical importance of retrieval for learning,” Science, vol. 319, no. 5865, pp. 966–968, 2008.
[14] J. L. Dobson and T. Linderholm, “Self-testing promotes superior retention of anatomy and physiology information,” Advances in Health Sciences Education, vol. 20, pp. 149–161, 2015.
[15] S. R. Hinze and D. N. Rapp, “Retrieval (sometimes) enhances learning: Performance pressure reduces the benefits of retrieval practice,” Applied Cognitive Psychology, vol. 28, no. 4, pp. 597–606, 2014.
[16] O. O. Adesope, D. A. Trevisan, and N. Sundararajan, “Rethinking the use of tests: A meta-analysis of practice testing,” Review of educational research, vol. 87, no. 3, pp. 659–701, 2017.
[17] J. Dunlosky, K. A. Rawson, E. J. Marsh, M. J. Nathan, and D. T. Willingham, “Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology,” Psychological Science in the Public interest, vol. 14, no. 1, pp. 4–58, 2013.
[18] Y. Weinstein, K. B. McDermott, and H. L. Roediger III, “A comparison of study strategies for passages: Rereading, answering questions, and generating questions.,” Journal of Experimental Psychology: Applied, vol. 16, no. 3, p. 308, 2010.
[19] S. J. Myers, H. Hausman, and M. G. Rhodes, “Testing effects for self-generated versus experimenter-provided questions.,” Journal of Experimental Psychology: Applied, vol. 30, no. 2, p. 241, 2024.
[20] P. K. Agarwal, J. D. Karpicke, S. H. K. Kang, H. L. Roediger III, and K. B. McDermott, “Examining the testing effect with open-and closed-book tests,” Applied Cognitive Psychology: The Official Journal of the Society for Applied Research in Memory and Cognition, vol. 22, no. 7, pp. 861–876, 2008.