College Board’s own research found Hispanic students sometimes scored better with pencil and paper
From The Hechinger Report
By Jill Barshay
January 31, 2022
The College Board’s announcement that it is ditching the paper-and-pencil SAT in 2024 and administering the college admission exam only by computer raises a confounding question: do students perform better on paper or digital tests?
Back in 2016, Ben Herold of Education Week reported that grade-school students who took the computerized version of a different test, the Partnership for Assessment of Readiness for College and Careers (PARCC), in 2014-15 scored lower than students taking the paper-and-pencil version. Educators and testing experts surmised that many students weren’t comfortable working on a computer, or weren’t familiar with the computer they had to use for the exam that day.
Research has shown mixed results. A 2009 analysis of 81 multiple choice tests given to elementary, middle and high school students directly compared paper versus digital versions. Students seemed to do better on computers in English and social studies, but better with pencil and paper in math. An earlier 2006 study found that low-income students scored better on paper exams than digital ones. Other studies have shown almost no difference between the two modes.
It may be most instructive here to look at the College Board’s own research. The organization has been pilot testing computerized versions of the SAT for several years and in 2019, it published the results of an October 2016 experiment that directly compared paper-and-pencil and computerized versions among more than 5,000 11th and 12th grade students at 55 schools. Half the students were randomly assigned the computerized version and the remaining half took the test, as usual, on paper.
To my surprise, reading scores were consistently higher on the computerized test – about a point higher. This was surprising to me because a separate body of research has overwhelmingly found that reading comprehension is better on paper than on screens, particularly for absorbing nonfiction passages.
College Board researchers dug deeper into why students were better at reading on screens on the SAT. They found that students were more accurately answering one type of reading question, in which students are asked to identify the portion of a text that serves as the best evidence for an answer to a previous question. The same happened on history and social studies tests; students were better at finding text evidence on a screen.
Screens also appeared to help students answer more questions. Slightly more students on computers answered questions at the end of a test, leaving fewer questions blank, especially in math.
However, the advantages of screens appeared not to be as true for Hispanic students. The College Board researchers noted when Hispanic students took the SAT on screens that their reading scores didn’t go up by as much as they did for white, Black and Asian-American students. Indeed, on the PSAT 10, a similar test given to 10th graders, Hispanic students tended to score higher on the paper-and-pencil test than they did on the computerized version.
A similar phenomenon was also seen for students who speak another language better than English. They consistently scored better on the paper version of reading tests.
For those who are curious about writing, students in the study tended to produce a better essay on paper than on computers. The essay portion of the SAT wasn’t part of the student’s score and has since been discontinued. When the National Assessment of Educational Progress looked at differences in essay writing on screens, it found that some students benefited and others suffered. High-achieving fourth graders wrote better on computers while low-achieving fourth graders wrote better on paper. It appeared that kids who were already accustomed to writing on a computer took more advantage of copy and paste commands and were able to revise their essays.
The College Board’s decision to scrap paper-and-pencil testing is the third major revamp of the college admissions exam in the last 20 years. It’s also shortening the test from three to two hours and allowing students to use calculators during the entire math section. Many colleges have made the SAT an optional part of the college application and the exam seller is trying to reduce costs and hold on to as much of the lucrative student testing market as it can. The College Board says that the shift to computerized testing will make the test “easier to take, easier to give and more relevant.” While the format change affects students, there may be even more compelling business reasons to make the switch.
Photo: U.S. News Global Education
Read this and other stories at The Hechinger Report