(WTTW News)
Illinois signed a contract for $640,000 for a report that could help legislators determine the value of the state’s recently defunct private school scholarship program, but a lack of data makes its findings “inconclusive.”
“What we can learn from this report is limited and inconclusive due to the absence of demographic data for the scholarship recipients and the lack of apples-to-apples comparisons,” Illinois State Board of Education spokesperson Jackie Matthews said in a statement about the just-published analysis on Invest in Kids.
Invest in Kids allows Illinois residents and businesses to receive a tax credit worth 75% of their contributions to certified scholarship granting organizations that fund scholarships for needy students to attend qualified private and technical schools.
The program is on its last legs.
Despite an active campaign by beneficiaries of the program, lawmakers last spring chose to not renew it for a seventh school year — a win for teachers unions that view the program as a back door voucher program that siphoned dollars from public schools, and a loss for parochial school networks that say the program’s end leaves low-income families without education options.
Illinois stopped awarding tax credits to Invest in Kids donors for contributions made after Dec. 31.
Students who go to school based on the tax-credit supported scholarships will no longer be able to rely on them starting the 2024-25 school year.
Backers of the phased-out program are hoping for a revival, but the 14-month analysis they’re working to retract isn’t likely to help their cause.
The report found that public school students out-performed Invest in Kids scholarship recipients in standardized tests.
For example, an analysis of the 2021-22 school year found that 20.8% of third-to-eighth grade Invest in Kids students met or exceeded English and language arts expectations on the Illinois Assessment of Readiness standardized test, compared with 30.1% of all public school students.
“Among grade 3-8 scholarship recipients, less than one-fifth achieved proficiency in Math in 2022, compared to about 1 in 4 public school students,” the report reads.
Invest in Kids champions said those and other similar findings should be considered “meaningless” and “unfair” because it compared scholarship students with all public school students.
The appropriate comparison would have been between Invest in Kids students with their low-income peers attending public schools, said Bobby Sylvester with Empower Illinois, the largest scholarship granting organization.
“They compared low-income scholarship students to the average public school student, not the low-income public school students that they are required to,” Sylvester said. “The number one predictor of student test scores is going to be their socio-economic background … the more low-income you are, the more likely you are to perform worse on the scores.”
The report, by California-based research company WestEd, reached such a conclusion in its examination of student performance at private schools it surveyed.
The analysis found that students attending “economically advantaged” and “predominantly white” private schools performed better than those in “economically disadvantaged schools” and “predominantly minority” private schools.
“This finding is consistent with other research studies that have shown test proficiency is positively correlated with income and parents’ education level and income is also correlated with race/ethnicity,” the report reads.
Sylvester characterized sampling scholarship students with the general student population as making an “apples to zebras” comparison.
WestEd didn’t go that far, but acknowledged limitations.
“The research team was unable to conduct a true apples-to-apples comparison because we did not have enough information to do so,” one of WestEd’s authors said in an emailed response to questions from WTTW News.
Why wasn’t there an apples to apples comparison?
WestEd said “the research team was constrained by the data that ISBE shared with us.”
ISBE provided individual-level data on third-to-eighth grade students who took the Illinois Assessment of Readiness and the high school students who took the SAT at private schools involved with Invest in Kids.
The data did not contain demographic information about individual students, including race/ethnicity, English learner status or level of economic disadvantage, WestEd said.
The only information the research team had about Invest in Kids families’ economic status was the parameters of the program — that to qualify, a student’s family income could not exceed 300% of the Federal Poverty Level (FPL), and that after securing a scholarship, a student could retain it as long as their family’s income didn’t exceed 400% of the FPL.
“There was no specific information about income levels,” provided by ISBE, WestEd said.
Sylvester suggested that WestEd would have been better served to survey public school students who qualify for the Free and Reduced Lunch program, which serves students whose families are at or under 185% of the FPL.
“That is nearly 70% of the students on these scholarships,” Sylvester said.
WestEd said it was a “possible criterion,” but the research team didn’t know how many Invest in Kids students were eligible for Free and Reduced Lunch “so it was not possible to compare them with similarly situated public school students.”
Sylvester also said Illinois Department of Revenue data collected on Invest in Kids students’ race and family income levels could have been used.
“The purpose of the program is about providing scholarships for low income and working class families,” he said. “That information was out there.”
WestEd said that its team is aware of the Department of Revenue data.
“However, at the time of analysis, this data was not available to us,” WestEd said. “Our charge, as requested by ISBE, was to conduct an analysis of the average performance levels of students receiving the scholarships, based on the data that ISBE provided to us.”
WestEd said it used the average achievement levels of public school students across the state for its analysis “under the guidance and approval of ISBE.”
For its part, ISBE points out the state law requiring the report on Invest in Kids mandated that it be conducted by an independent research organization.
“ISBE has not conducted any validation of the results, so we cannot comment on the study except to echo WestEd in urging caution in interpreting the results, since the data limitations prevented any apples-to-apples comparisons,” Matthews said. “The independent research organization stated in its report that it was unable to collect demographic data on scholarship recipients, so it could only compare averages of all scholarship recipients to averages of all public school students. Without demographic data, we do not know if any of these comparisons are valid.
While WestEd answered specific questions poised by WTTW News about its analysis, the company had identified limitations upfront in its report, which was made available to lawmakers on Friday.
“Given the limited number of testing years and the data limitations outlined in the report, WestEd urges the findings to be interpreted with caution,” the report’s summary reads. “If the IIKA (Invest in Kids Act) scholarship program continues, additional information about scholarship recipients would allow the state to conduct a more rigorous analysis of the effectiveness of the program and the equity of impacts on students with different backgrounds and needs. The state may want to consider asking scholarship-receiving schools to include student demographic information, including special education and English language learning status, in all IIKA scholarship recipients’ student assessment data.”
Empower Illinois and other members of the Secure Our Scholarships coalition said the report doesn’t comply with requirements in state law (35 ILCS 40/1) which called for a report that must, “to the extent possible” compare learning gains of Invest in Kids recipients with learning gains of public school students “with socioeconomic backgrounds similar to those of students receiving scholarships.”
Sylvester said the coalition of Invest in Kids backers have reached out to ISBE, and want the study retracted.
The Secure Our Scholarships coalition nonetheless said the report has its merits.
Researchers surveyed principals, teachers, parents and students at private schools where Invest in Kids students were enrolled.
Sylvester said that demonstrated “overwhelming support” by parents and children for the education Invest in Kids students were receiving.
The site visits and interviews, however, also raised areas of concern particularly for smaller private schools, such as a lack of support for special education students and students with behavioral issues, and a lack of resources for counseling, after-school programs and other supports.
“Participants at schools WestEd visited expressed concern about their ability to support students needing additional help with language, behavior, and physical or mental challenges,” the report concluded. “If the IIKA program continues and the scholarships are meant to help students regardless of their additional needs, the state may want to assess the need for additional support for scholarship recipients with language, behavior, physical, or mental challenges.”
WestEd cautioned that these were “anecdotal” findings, as there is no “systemic data” for whether these are issues at schools the research team didn’t visit.
Secure Our Scholarships is lobbying the General Assembly to renew the program.
The coalition in the spring promoted a compromise (HB4194) that would have reduced the tax credit amount available, while also giving donors a larger credit for funding a scholarship awarded to a student from an underserved community — a means of increasing the number of students of color partaking in Invest in Kids.
The measure never came up for a vote.
Gov. J.B. Pritzker took a hands off approach to Invest in Kids in the spring, and is not working to bring it back.
On Friday, ISBE said it had paid $394,972 toward the $640,000 contract. The 2024 fiscal year invoice is for an additional $122,652, bringing the total to $517,624.
Note: This article was updated to reflect how much ISBE has paid as of March 1 toward the $640,000 contract.
Contact Amanda Vinicky: @AmandaVinicky | [email protected]
