For some jobs, speed is an element of competence. A slow typist, for example, is not competent to fill positions that require higher typing speeds. Tests designed to assess competence for this kind of work are "speeded." That is, they offer more material than any examinee can complete and the administrator measures the examinee's speed as well as other aspects of performance. Tests that measure knowledge or skills without regard to speed are known as "power" tests. A power test that implicitly requires candidates to compromise accuracy for speed is inappropriately speeded.
Speed is not a component of minimum competence for lawyering. As we explain further below, however, the traditional bar exam implicitly tests for speed--and penalizes candidates who are not able to match the pace set by that exam. We also discuss ways in which alternative assessments can avoid speededness. First, however, we outline key testing standards and research about speed, speededness, and exams.
Standard 4.14 of the Standards for Educational and Psychological Testing, which outlines best practices for test design, states: "For a test that has a time limit, test development research should examine the degree to which scores include a speed component and should evaluate the appropriateness of that component, given the domain the test is designed to measure." A comment to that standard adds: "When speed is not a meaningful part of the target construct, time limits should be determined so that examinees will have adequate time to demonstrate the targeted knowledge and skill."
The Standards also stress that allowing adequate time for an exam is part of universal design. On p. 58, as part of the Standards' discussion of fairness in testing, the editors state: "Universal design processes strive to minimize access challenges by taking into account test characteristics that may impede access to the construct for certain test takers, such as . . . by providing extended administration time when speed is not relevant to the construct being measured."
Despite Standard 4.14 and growing interest in universal design, researchers note that many test-makers pay insufficient attention to the speededness of their exams:
When test-makers do consider speededness, they often rely upon outdated measures such as calculating (1) the percentage of examinees who did not complete the exam or (2) the percentage who appear to enage in rapid guessing near the end of the exam. Scholars have criticized these measures, especially when applied to high-stakes exams. For those exams, test-takers may pace themselves by working faster than is optimal throughout the exam. Indeed, prep courses for these exams often stress the need to work quickly. High-stakes exams, therefore, may be inappropriately speeded (assessing speed as well as accuracy) even if most test-takers complete the exam without engaging in rapid guessing.
For these reasons, contemporary researchers suggest that the speededness of an exam is best assessed through controlled experiments. The researcher may vary the amount of time that subjects have to complete a portion of the exam. Alternatively, a controlled experiment may vary the number of items that subjects must complete within the same time frame. Polina Harek et al. (cited above) conducted this type of experiment with part of the United States Medical Licensing Exam (USMLE). They found that the exam was inappropriately speeded and that the impact was greatest on the lowest scoring examinees. In response to their study, the test-maker increased the amount of time per item and established an ongoing program to assess time limits on the exam.
Here are some additional sources related to speededness:
NCBE acknowledges that "[f]or the most part, licensing tests like the MBE and MPRE are not designed to be speeded." That careful phrase, however, recognizes that these tests are speeded to some extent. Marilyn Wellington, NCBE's Chief Strategy and Operations Officer, recently explained that "the intent [on the bar exam] is for examinees to exhibit their ability to work at a speed that is reasonable for legal work while fully demonstrating their skills and knowledge." (FN 2) But what is "a speed that is reasonable for legal work," and how does NCBE determine that speed?
Research by the AccessLex Institute and New York Board of Law Examiners demonstrates that the UBE is speeded for many candidates. Twelve percent of candidates taking that exam for the first time reported that they ran out of time to complete the MBE (the multiple-choice portion of the exam). Nineteen percent of second-time takers reported running out of time on that section. P. 25. That exam experience correlated significantly with failure--even after controlling for a host of other variable. P. 80. Similarly, running out of time on the essay portion of the exam correlated significantly with failure after controlling for other variables. Id.
To address this problem, the AccessLex report recommends that candidates receive more instruction on strategies for answering questions quickly, complete more practice exams under timed conditions, and obtain more feedback on their performance. This is good advice for candidates forced to take a speeded exam. But this extensive preparation drives up the costs of taking the bar exam, and there is no evidence that it develops a skill that is essential for entry-level law practice.
Unfortunately, the NextGen bar exam may be even more speeded than the UBE. NCBE has announced that the exam will require nine hours of testing time, rather than the twelve currently required for the UBE. Although some test-takers may welcome a shorter exam, NCBE's announcements suggest that candidates will have to work more quickly on the nine-hour exam. The performance tests on NextGen, for example, may be allotted just 60 minutes apiece, rather than the 90 minutes provided for each of these tasks on the UBE.
This section is under construction.