An online experiment found that information on how much kids learned at school, not how high they scored on tests, nudged parents to more racially integrated schools
From The Hechinger Report
By Jill Barshay
June 28, 2021
Imagine you’re a parent relocating to a major metropolitan area. The quality of the schools will likely factor into your decision about where to live. Googling around, you might look at websites such as GreatSchools and Niche to compare academic ratings, demographics and family income among school districts.
The devilish details of this data matter. An intriguing online experiment suggests that displaying some types of academic information, and withholding others, might influence parents to make different decisions about where to live and send their kids to school.
Data on student achievement, such as average test scores, prompted a preference for school districts that are wealthier and more white, researchers found in the new online study. By contrast, data about how much kids learn at school each year — how fast test scores grow — nudged survey respondents to prefer school districts that are poorer and less white. That’s because children in affluent areas tend to test well and children in poorer districts score much lower, on average. But impressive test score growth can happen even at poor schools where students are low achieving.
“We know that growth is a better measure of educational quality than achievement,” said David Houston, an assistant professor of education at George Mason University, who conducted the online experiment together with Jeffrey Henig at Teachers College, Columbia University.
“It’s not all that shocking when you give people growth information, they tend to choose higher growth schools, which on average, not every time, aren’t always the whitest and wealthiest schools,” Houston said in an interview.
Of course, this was just a hypothetical online exercise and it’s not clear that anyone would really make these moving decisions. But it’s interesting how academic information can reinforce and possibly unwind entrenched divisions between race and class in our schools and neighborhoods. The study, “The Effects of Student Growth Data on School District Choice: Evidence from a Survey Experiment,” is slated to be published in the August 2021 issue of the American Journal of Education and was posted online in June 2021.
“We do *not* believe that a minor tweak like changing how we measure and report school & district performance is the secret key to ending decades of school segregation,” Houston tweeted about his study. “In a society with multiple layers of systemic racism, more dramatic reforms are necessary.” He added that “small improvements to the status quo are still worth pursuing.”
In the experiment, researchers surveyed 2,500 adults online and asked them to imagine choosing a school district for their child in five metropolitan areas: New York, Los Angeles, Chicago, Houston and Phoenix.
In the New York area, participants chose from New York City; Yonkers; Jersey City, New Jersey; Patterson, New Jersey, and Toms River Regional School District, also in New Jersey. Median household income and the percentage of students who qualify for free or reduced price lunch were listed, along with the racial and ethnic makeup of the students. In Toms River, an affluent suburban district, the median income exceeds $80,000 and 80 percent of the students are white. On the other end of the spectrum is Paterson, where median income falls below $32,000 and 5 percent of the students are white. (See image from the experiment below.)
The researchers randomly assigned respondents to view additional information for each district. One group was shown the average achievement level and national rank. A second group saw how many grade levels a typical child grows each year and the national rank of this growth score. A third group saw both achievement and growth. A fourth group, which saw only the demographic and income data, was used as a comparison.
Researchers purposely narrowed the choices to districts where academic achievement and growth don’t always move in tandem. In Toms River, students are among the top 20 percent highest achieving students in the nation but the average student grows 0.93 grade levels each year, learning less than the majority of U.S. children in a school year. In Paterson, the academic achievement of students is in the bottom 10 percent in the nation but students grow more than a whole grade level each year.
By contrast, participants who saw growth data, but not academic achievement, tended to choose districts with about $2,900 lower household incomes, 2 percentage points more students eligible for free and reduced-price lunch, and about 4 percentage points fewer white students.
When both student growth and achievement were displayed, they seemed to cancel each other out. Respondents picked the same district that they would have picked if they hadn’t had any academic information and all they knew was the racial composition of the students and family incomes.
This study was inspired by earlier Stanford University research that measured school effectiveness by teasing out what’s actually happening in schools from the socioeconomics of the students in the schools. The school system that rose to the top, according to this measure, was Chicago, where the majority of students are poor enough to qualify for free or reduced price lunch. Students typically entered third grade well below grade level in reading and math but almost caught up to the national average by eighth grade. Academic growth like this was not as strong in many wealthier school districts.
Houston wondered what parents might choose if they were told how effective schools are, using this Stanford yardstick. Would they still pick the highest achieving school? Or might they pick the school where kids are learning more?
What makes this analysis complicated for parents is that student achievement is important too. Students don’t only learn from their teachers and their lesson plans but also from their peers. An extensive body of research shows the power of peer effects. That’s why high achieving schools can often be high growth schools too. Princeton, New Jersey, is a good example of a school district where students test well and show above average academic growth each year.
Data on student academic growth is reported by 42 states and the District of Columbia on their annual school report cards, according to the Data Quality Campaign. However, it’s often hard for parents to see this growth information. Sometimes it’s buried in Excel spreadsheets. Another six states measure growth but don’t report it publicly. California and Kansas are the only states that don’t measure it. GreatSchools includes academic growth measures on its website and it recently started weighing student growth more heavily in the overall score it places on each school.
Houston is currently repeating this online experiment, asking people to choose schools instead of districts to see if his results replicate. That could help improve school choice systems where families can choose a school within a school district.
My take is from this experiment that parents might choose a more diverse school if they are told about student growth but not test scores. It’s not practical to hide those. You need to know annual test scores first in order to calculate how much they have grown.
What’s clearer to me is that public reporting of academic achievement for every school has exacerbated the residential and educational divisions in our society. The 2001 No Child Left Behind Act that required annual testing was intended to help spotlight and improve schools where student performance lagged. The unintended consequence was that this data also put a spotlight on the highest achieving schools. Families wanted to send their children to them and that increased demand for the houses in those neighborhoods. As housing prices soared, those neighborhoods became less affordable to middle class families. The neighborhoods with high achieving schools became ever pricier and higher achieving in a self-perpetuating cycle of wealth and privilege. Ignorance may have been better for our society.
Photo: Anne Arundel Community College
Read this and other stories at The Hechinger Report