Whether or not Chicago Public Schools students are gaming a standardized test system was the topic of debate during Wednesday’s CPS board meeting.
The district’s outgoing inspector general says his investigation shows a fraction of students are taking twice as long to complete the Northwest Evaluation Association (NWEA) test – and sometimes longer – than the national average.
The untimed, standardized test is given to third- through eighth-grade students.
Inspector General Nicholas Schuler says his research shows a “concerning level” of unusually high test durations – students taking a couple of hours to take the test when the national average is 45 minutes to just over an hour.
Schuler found this to be the case for more than 50,000 students who took the test. He says it’s been going on since 2016, and the times have been getting longer every year.
His research shows (below) the gap between the amount of time it takes a CPS student to complete the test, compared to the national norm:
Schuler also noted that 4% of tests had a high number of pauses. Usually, a pause indicates a student’s trip to the bathroom, or a problem with the technology.
But if a student doesn’t know the answer to a question, he or she can just wait it out, or ask to pause the test, and after 25 minutes, the computerized test introduces a new question &dnash; and the student might have a better chance of answering this one correctly.
In his report, Schuler stops short of saying there’s evidence of outright cheating, but he says most of these higher instances of pauses were clustered in the same few schools, which he doesn’t name.
He says the concern here is that there are high stakes attached to these tests.
“This is a major concern, given the many ways CPS uses these results,” Schuler said in his prepared statement to the board Wednesday. “However, an untimed test that carries stakes for students, teachers, principals and schools, high durations and pauses could also be indicative of improper attempts to win higher scores or gains. OIG interviews with a small sample of 20 students and 10 teachers suggested this might be the case in certain instances.”
Among those stakes, the NWEA test results are used to promote students to the next grade, or for admission into selective enrollment high schools. They’re also used for teacher evaluations for math and reading and for the School Quality Rating Policy (SQRP), which is the district’s system for rating schools.
Even the Chicago Teachers Union argues that while teachers support assessments, attaching them to such high stakes is dangerous.
“We think the SQRP and the schools’ ratings policies put too much pressure on these tests, which in turn drive a number of other adult consequences and the logical outcome of that is that people game tests,” said CTU President Jesse Sharkey. “And if we want to stop gaming tests, we need to look at the SQRP and underlying policies.”
Schuler’s recommendations include hiring a test security expert, which the district says it has already done; setting time limits on tests for general education students; and keeping math and reading teachers – whose evaluations are tied to student performance – from proctoring their students’ tests.
The district says Schuler’s research did not find any correlation between test duration and high academic growth: just because the students took longer to take the test doesn’t mean they were able to successfully game the test and get a higher score.
“The headline is, there in the IG slides is that, there is no causal relationship,” said Jeff Broom, CPS director of school quality measurement and research. “But just to, again, help our stakeholders interpret this data, we don’t want to draw conclusions that aren’t warranted or substantiated by any evidence.”
The district also cites other research they say further proves the academic growth CPS students have been making – that does not include the NWEA test results.
“The NWEA test was intentionally designed to have flexibility as it is an untimed, adaptive assessment. The OIG performance review has revealed that the NWEA assessment flexibility has resulted in long durations and inconsistent pause usage,” said Chief Education Officer LaTanya McDade. “These findings illustrated the need for additional guidance clearer policies, protocols and strengthened training.”
The report is among the last for Schuler, who gave his resignation to Mayor Lori Lightfoot last month. He was asked to resign amid the results of an investigation conducted by a private law firm, at the request of the board, into allegations that he had created a toxic work environment for his staff in the inspector general’s office.
Schuler claims that while some concerns were founded, many were exaggerated and he says he’s proud of the work his office has done.
Schuler’s last day is Friday.
Follow Brandis Friedman on Twitter @BrandisFriedman