Open Book Exams

---

Most bar exams are closed book, reflecting the assumption that new lawyers must be able to recall legal principles from memory. Research, however, suggests that open-book exams may be more appropriate for testing a new lawyer's minimum competence. We summarize here research related to this question, with a particular focus on studies related to college and professional students, licensing exams, and high-stakes exams.

During the pandemic, the medical faculty at the Imperial College London decided to administer two final-year exams in an open-book format. The exams, which assess applied knowledge of medical principles, had previously been administered as closed-book exams. No additional time was provided for the open-book format.

The researchers compared results on the open-book exams to closed-book exams on the same topics administered over the previous three years. Median scores on the exams were identical, and the "average discrimination of the [open-book exams] was comparable to that of the [closed-book exams]." Reliability of the open-book exams, as measured by Cronbach's alpha, was similar to that for the closed-book exams (over 0.80).

The authors concluded that "concerns about the use of [open-book exams] in high-stakes assessments may be unfounded." Indeed, an open-book exam "encourages the use of problem-solving skills more akin to those used in real life." 

The researchers in this study compared closed-book and open-book exams across several sections of a college psychology course. The same instructor taught all sections and used the same materials for each section. Exam formats were counterbalanced so that students in each section took some open-book and some closed-book exams. By pooling results for the full semester, the researchers reduced the impact of any differences among the sections.

Students scored significantly higher on the open-book exams (mean = 80.11) than on closed-book ones (mean = 72.52). They also reported less anxiety for the open-book exams. Retention (as measured 2 weeks after the initial exam) did not differ significantly between the groups. Somewhat surprisingly, students reported studying more for the open-book exams than the closed-book ones, but the difference was not statistically significant.

The researchers concluded that professors should consider open-book exams because "[a]n open-book exam still distinguishes good students from poor ones, is preferred by the students, does not seem to decrease learning and retention, and decreases anxiety levels."

Note: This study also included a condition in which students could use "cheat sheets" (a double-sided piece of 8-1/2 x 11" paper) during the exam; this portion of the study included students enrolled in an introductory statistics course. In the psychology course, performance on the cheat-sheet exam fell between that of the closed-book and open-book conditions, and the difference was statistically significant. The statistics course did not include a closed-book condition. There, no statistically significant difference emerged between the open-book and cheat-sheet conditions. Studens who used cheat sheets, like those who took an open-book test, showed as much retention as students who took a closed-book test.