Close Your Books and Take a Test

The following guest post from Joshua Courtney, DO compares learning methods for test preparation and discusses the research behind the various techniques.
For many students, springtime equates with hundreds of hours preparing for standardized examinations spanning over a decade of medical education at many levels. While exams such as the SAT, MCAT, USMLE, COMLEX, NCLEX and NAPLEX all possess their own unique attributes, they are all defined as high-stakes standardized tests. Although most standardized exams within health care are primarily given to ensure a certain minimal level of competency amongst practitioners, it is common knowledge that these examinations are also used as selection criteria by institutions and residency programs to distinguish between applicant pools. A score alone, in some cases, may function as the sole factor in limiting a physician’s future specialty or even the ability to gain licensure - warranting the “high-stakes” descriptor. Surveys published by the National Resident Matching Program (NRMP) have identified USMLE/ COMLEX Step 1/Level 1 scores as the number one selection criteria used by residency program directors to identify candidates for which to offer interviews.

Preparing for any standardized test is daunting, to say the least. In the recently published How To Study For Standardized Tests, Sefcik, Bice & Prerost point out that students who have received low (or failing) scores on standardized exams in the past often describe themselves as “poor standardized test-takers.” The authors further note, “the phrase itself creates the impression of a mysterious innate quality (or lack thereof) that renders someone constitutionally incapable of taking a standardized test.” While it is true that some individuals inherently do better than others on these tests, the more accurate phrase should describe certain students as being “poor preparers for standardized tests.” Now we are saying the individual is inherently capable, just lacks effective methods to achieve their high scoring potential. This speaks highly of the importance of simply believing in one’s self.

Standardized tests are very different from teacher/professor written exams on many levels. Teachers write tests that rewards the student who pays attention in class, studies adequately, and can easily pick out important lecture points – all behaviors a “good student” should possess. Recall of highlighted facts and memorization of the material, alone, are usually enough to sufficiently perform on professor-written tests. Because they are limited in focus, cramming knowledge over a short timespan may be adequate. When it comes to standardized test-taking, traditional methods that have been effective in the classroom may not work.

Preparing for a standardized exam is an entirely different experience, requiring an entirely different strategy. Unlike teacher-written exams, a well-designed standardized test should have global focus with emphasis on integration, associations, and application measures. Standardized exams are written by individuals you have never met. Their purpose is to test a your global knowledge base. There may even be intention to evaluate specific cognitive skills, such as verbal reasoning or analytical thinking. The high-level thinking items tested on standardized exams require a deeper understanding of principles and concepts, leading to a greater degree of difficulty.

The weeks leading up to any high-stakes examination are stressful, to say the least. Most students use a multimodal approach to exam preparation, consisting of review books, practice questions, flashcards, review courses and other methods recommended by upperclassman that best fit their learning style. Students often default to the same study behaviors that have work well for them in the past – an approach that makes sense on the surface. Although individualization of any study plan is a must, there are certain preparation behaviors that are common amongst “high-scorers.”

Rodin's The Thinker

In a recent study performed at Purdue University, and published in Science, researchers looked at the effectiveness of three distinctly different study methods they termed (concept mapping, retrieval practice, and repeated study) to assess their impact on knowledge retention. Below are clear descriptions of each of these learning methods:

Concept mapping: recreating subject matter through diagrams to organize and encode meaningful relationships among concepts.

Retrieval practice: process of using cues to actively recall and reconstruct knowledge (ex: group discussion, reciprocal teaching, taking practice tests)

Repeated study: act of rereading notes, books, or other study materials

When comparing each of these methods head-to-head, it became clear that time spent rereading notes and review materials would be better spent practicing retrieval to ensure better learning. Of the three groups tested, students who took practice tests were found to have retained nearly 50 percent more of the information a week later than students who used the other two methods of studying. These results have been supported by further studies, and are shown to contradict the perception of many students preparing for standardized exams.

In repeated scenarios, when students were asked to make predictions of the long-term learning benefits of different methods of study, they have indicated that both rereading of information and concept mapping (diagrammatic reconstruction of knowledge) would produce more learning than active retrieval. While students believed they would do far better after elaborative studying as compared to taking practice exams, the later group (retrieval practice) was found to have a greater impact on both learning and examination performance. This phenomenon has come to be known as the “testing effect” and shows that acts of retrieval have a potent effect on learning while enhancing long-term retention of the information being tested. These findings parallel research from Washington University in St. Louis suggesting that the common practice of rereading material is no more effective in improving learning than performing a single, initial reading of the text. The practice of spending large amounts of time rereading materials despite limited benefit to memory and retention is a phenomenon coined “labor-in-vain” learning. This method of preparation is pervasive among test-takers, despite being ineffective.

Students and residents preparing for licensure and specialty boards alike often incorporate practice questions into their exam preparation as a means to assess areas of strength and weakness, as well as to assess exam preparedness. Although using practice tests for this purpose can be an effective means to both self assess and drive one’s focus, this differs from the concept of using testing as a learning device of its own. The process of practice retrieval has been shown to be of far greater value to impactful learning than previously thought.

Although medical schools and other institutions provide some level of guidance for students undergoing high-stakes exam preparation, self-study and self-guidance are implicit in great extent to the process of preparing for any standardized test. The incorporation of taking practice items is important for diagnostic purposes, but the true impact comes from the actual learning that takes place during the process of testing itself. This may seem obvious, however, in repeated studies, students have been shown to lack awareness of the testing effect, and its potent impact on learning itself. In multiple cases, when asked to select only one learning modality by which to prepare, the majority of students resort to traditional study of material through rereading of books. Using active practice retrieval is typically thought of as an adjunct means of preparation, rather than a primary study tool.

Thinking back to my initial days as a first year medical student, I remember a discussion with an upperclassman that offered a candid portrayal of what my next two years of medical school would look like. He told me I would feel as though I was standing openmouthed in front of a fire hydrant spraying full blast. This unappealing analogy is surprisingly accurate in correlating the amount of information presented in medical school to the amount of material retained over time. Information learned in a manner that allows individuals to make inferences and apply knowledge in a robust and enduring way is known as “meaningful learning.” This is often described in contrast to “rote learning,” which is considered brittle and transient. Rote learning is short-term and is reflected in failures to make inferences and incorporate knowledge for problem-solving and inference-related tasks.

Rote learners may do well on course exams and get by on their ability to memorize short bursts of material for transient periods of time. This method may work on teacher-generated tests, which limit focus and emphasize specific details of course material, resulting in adequate or even high course grades. By contrast, however, rote learning is detrimental to generating a high performance on a standardized exam that has a more global focus and requires application of learned concepts. Rote learning often becomes apparent with students who’s standardized test scores appear to be considerably sub-par to their classroom performance. Meaningful learning, by contrast, is both impactful and preparatory for the depth and complexity of questions tested on standardized examinations. Students who engage in meaningful learning methodologies are likely to achieve standardized test scores more on-par with course grades, and may even outperform their score expectations. Recent evidence has shown that practicing active retrieval enhances the performance standards used to measure meaningful learning.

Most students I have worked with during my career seek a give-it-my-all approach when preparing for a high-stakes standardized test. The last thing any student wants to do is settle for a different specialty or career path because of a lackluster study effort that resulted in a lackluster score. Students sometimes spend upwards of 10-15 hours per day studying for their board exams in the days and weeks prior to their test. Much of this time is spent rereading popular board review texts 3,4 or even 5 times. Although this approach may be perceived as tried-and-true methodology to some, recent evidence suggests this may be a “labor-in-vain” method of learning. Time spent participating in learning activities that promote active retrieval are shown to have a more potent effect on meaningful learning. This leads to both higher board scores and better long-term retention. Participating in group discussions, reciprocal teaching, and taking practice exam items are all likely to engage active learning processes. We may believe we have mastery of a particular topic, but the most effective way to examine knowledge through self-assessment is by engaging in the process of retrieval.

Although much work is needed to further assess the most effective means by which to utilize retrieval as a tool to enhance meaningful learning, our preliminary data suggest that taking a larger quantity of questions over a sustained period of time is correlative with improved standardized test performance. Further work is needed to assess the extent to which this correlation exists. Taking practice items that are constructed to reflect the types of questions specific to the actual test, and similar in the level of inference required for completion, appear to offer an effective approach to practice retrieval.

So the next time you are tempted to reread your notes or the biochemistry chapter of your favorite review book, stop and consider a different approach - take a test.


Joshua Courtney, DO is an anesthesiologist and the founder/CEO of COMBANK, a leading online q-bank for the COMLEX exams.