Popular policy proposal is plagued by poor attendance, quality concerns
From The Hechinger Report
By Jill Barshay
March 29, 2021
Summer school may seem like a common sense way to help children make up for the months of lost school time during the COVID-19 pandemic. Former New York City Mayor Mike Bloomberg urged President Joe Biden to push every school in the country to stay open this summer in a March 2021 Washington Post opinion piece. Governors around the country from Virginia to California are endorsing summer school, as has the powerful teachers union leader Randi Weingarten. More than $1.2 billion of the $1.9 trillion Covid relief package signed into law on March 11 is specifically earmarked for summer school and many policymakers are urging that billions more in federal and state funds be spent to open schools this summer.
There’s just one problem. Research studies done before the pandemic show that summer school usually doesn’t accomplish its purpose of raising reading or math achievement.
“Generally, summer programs are not effective because they don’t really engage young people and they’re not run well,” said Jean B. Grossman, an economist at MDRC, a nonprofit research organization, and Princeton University, who has evaluated summer school programs. “It looks like summer school should help but the research is a mixed bag.”
In a 2020 synthesis of summer school studies, researchers calculated that the benefit to students tends to be close to zero in math or reading. This meta-analysis is a draft paper, meaning it has not yet been published in a peer-reviewed journal, but is the most recent review of the evidence.
Summer school is “boring” and it’s not productive to force kids to learn when they don’t want to, said Robert Slavin, an education researcher at Johns Hopkins University and one of the meta-analysis authors. “Nobody wants to be sitting in school while their friends are out playing,” Slavin said.
When there were initial achievement gains from the extra summer instruction, they were often fleeting and disappeared by the next spring, Slavin added.
One of the biggest summer school studies was conducted in five cities: Boston; Dallas; Jacksonville, Florida; Pittsburgh, and Rochester, New York. Researchers at RAND observed how students did over four years, beginning with third grade in 2013, and found no academic benefits to summer school because of poor attendance. Twenty percent of the kids never showed up and the kids who did come skipped an average of one out of every four days. Many kids skipped more. Academic benefits from summer school were only seen in the fraction of kids who attended faithfully for two summers in a row.
Summer school programs that resembled camp, including sports and arts, didn’t have any better attendance than those that resembled ordinary school. Attendance prizes for families helped, RAND reported. One district spent almost $70,000 in supermarket gift cards for parents and small treats for kids, such as pizza and ice cream.
One of the most rigorous summer school studies looked at an unnamed Northeastern city that made summer school mandatory to elementary school children who failed year-end tests. Economist Jordan Matsudaira of Teachers College, Columbia University compared what happened to students who scored just below and just above the cut off for being sent to summer school. (The Hechinger Report is an independent unit of Teachers College.) Third graders who went to summer school did slightly worse than those who didn’t. Meanwhile, fifth graders benefited from summer school, in both reading and math. Attendance remained a problem. Many of the students who were supposed to go to summer school refused to or didn’t attend regularly.
One can find individual studies that show strong results for summer school. But scholars told me that these positive studies often involved small numbers of children and compared the performance of students who attended summer school with different types of students, which is not an apples-to-apples comparison. In other words, there’s a lot of shoddy research on summer schools out there.
One review of almost 100 older studies of uneven quality, published in 2000, generally found positive results from summer school. In the underlying studies, middle-class children benefited more than disadvantaged children — a disappointment for policymakers who are keen to focus on the neediest cases. And just to point to an example of how muddy the research is, this older meta-analysis found that the gains were strongest for early elementary and high school students. In the mandatory summer school study, which I described above, it was the opposite, with fifth graders benefiting much more than third graders.
Unfortunately, there are few randomized control trials, the gold standard in research, for summer school. Grossman conducted one in 2011-14 in which middle school students were randomly assigned to a five-week summer program with trained instructors using good curricula for reading and math lessons in the mornings and fun afternoon activities such as sports, music, theater and art. Again, persuading kids to enroll was a problem.
Even though attendance for those who did enroll was rather good in this study, the academic results were still disappointing. The program was billed as a way to “accelerate” learning but students learned at the same pace as they did at school. The gains for the five-week summer program were small and not statistically significant compared to kids who didn’t attend.
“My takeaway,” said Grossman, “is, if you’re lucky, for every week of summer school, you get what you get in a week of regular school. And if the summer school is not done as well, you get less than that.”
Often summer programs don’t train the instructors well or adhere to a proven curriculum. In a separate review of summer reading programs, positive results were seen only when instruction was “research based,” according to an analysis of studies that took place between 1998 and 2011 for children in grades kindergarten through eighth grade.
What do the researchers suggest for schools that want to proceed with summer school? Grossman’s advice is to focus on social-emotional skills and relationships to help re-engage students in learning. She thinks the best summer programs don’t make kids sit in a math class for an hour every day in the summer, replicating the structure of traditional school, but engage kids in topics that they’re interested in, such as social justice. “Maybe they have to calculate disproportionality in school discipline and they sort of inadvertently do math while they’re learning about this cool subject,” she said.
Slavin advocates fusing summer camp with intensive, daily tutoring. He noted that two studies in his summer school research review proved to be exceptions to the pattern of ineffectiveness. One is a Los Angeles study, published in 2005, and the other is a study in a small city in the Pacific Northwest, published in 2013. Both summer school programs focused on teaching young children to read in kindergarten and first grade. In both cases, children received group phonics instruction followed by reading practice in very small groups that resembled tutoring. To Slavin, tutoring is the secret sauce that made both summer school programs work, echoing the strong, more definitive evidence seen in the research literature on tutoring.
Unfortunately, even Slavin points out the gains for students in the Los Angeles summer school experiment faded out by spring; the students who had attended the summer program were no better off than similar students who hadn’t. No one followed up with the students in the Pacific Northwest study to see how long those gains lasted. And no one has studied whether Slavin’s summer idea of alternating soccer practice, say, with math and reading tutoring would actually be effective.
What all this shows is that helping kids catch up after the pandemic won’t be easy. And pushing students into summer school isn’t the answer — as appealing as it might sound.
Photo: Depositphotos
Read this and other stories at The Hechinger Report