[A] company is halting its work on a grant-funded effort to help Vermont educators better use data to improve student performance and is blaming the Agency of Education for failing to provide the needed test results.
Last fall, a web-based tool called Polarys was made available to educators for free to help them draw useful information from test data. But the creators of the tool are leaving a two-year private grant unfulfilled after what they say were repeated attempts to get test scores from the state.
The agency has cited student privacy concerns as one reason for its refusal to provide the data breakdown the company is seeking.
Mary Westervelt, director of operations at ReContext Data Solutions, which developed Polarys, said the company had joined with the Vermont Rural Education Collaborative to offer temporary free use of the web program to educators.
“We feel like kids take these tests, we spend millions of dollars implementing them, and then we hear from people that it is hard to access the (Smarter Balanced Assessment Consortium) data and they don’t know what to do with the data,” she said.
Polarys would be the solution, according to Westervelt, if the company had a little cooperation from the Education Agency. “It’s frustrating to feel like we have something people could make use of and we are 90 percent of the way there and we hit a roadblock and can’t take it the last 10 percent of the way.”
The Smarter Balanced test is aligned to the Common Core and replaced the New England Common Assessment Program.
The agency told VTDigger that the SBAC information has been released. But to perform its analysis, Polarys needed information that was not publicly available.
“It is not usable because they redacted everything you could use to make comparisons,” said Polarys developer David Adler, who is also the director of curriculum at Orleans Southwest Supervisory Union.
He was looking for data broken down by demographic factors.
“By controlling for factors that affect state exam results but are outside the school’s control, we are able to demonstrate the changes in outcome at the district, school and classroom level,” according to the ReContext website.
Polarys analyzes test scores along with other factors, such as programming, spending, student demographics, teacher characteristics and school structures. It also groups similar schools, so administrators can find those that are doing better on an indicator and learn about improvements they can make.
In January, Westervelt and Adler went through a formal research application process as a final attempt to get access to the testing data they wanted. They requested test scores from each Vermont school with more than 11 test-takers; the state identified 11 as the smallest number allowed before violating federal privacy laws. The company asked that the data be broken down by grade level, those who are on free and reduced-price lunch, and students in special education and nonnative English speakers.
“Our work will have the greatest degree of statistical validity when we are able to do analysis at the grade level. It is important to note that our presentation of the results will be available to the public only when re-aggregated to the school level,” the application says.
The company received a final rejection in March. Wendy Geller, director of analysis and data management, wrote that the agency was concerned that ReContext Data Solutions is a for-profit company and therefore might fall afoul of the Family Educational Rights and Privacy Act, known as FERPA.
“We wouldn’t be violating FERPA because we wouldn’t be publishing anything AOE doesn’t want published,” said Westervelt.
An agency spokeswoman also said officials felt uncomfortable with Polarys being owned by a private business that has no formal links to the agency.
“While we value schools working to make better use of their data in an effort to improve learning opportunities for all students, Polarys is a private business and not directly associated with the Agency of Education,” said the agency’s Stephanie Brackin.
Westervelt acknowledged it’s a for-profit company but said it partnered with the Vermont Rural Education Collaborative to make the tool available free of charge for two years.
Adler said the company didn’t intend for Polarys to be free forever but said he would try to get grants to extend the free period.
The agency also didn’t like that the researchers wanted to use certain school-level test scores.
“Releasing that score would certainly confuse the public, because it is nuanced,” Adler said. But that is the point of the Polarys algorithm, according to Adler: It takes many factors into account and norms the data.
Finally, Geller said the agency doesn’t have enough staff to regularly provide the data to the company.
Margaret Maclean, president of the VREC, said she was disappointed the state denied the data requests. “The process that ReContext was using with teachers was an excellent inquiry process to support teachers looking at data collaboratively,” she said.
Still, the developers say they aren’t abandoning Polarys. It will be available online with past data for the foreseeable future. The company hopes that after the state has accumulated several years of SBAC data, it will release the requested information. If that is the case, the company intends to revive Polarys with new infusions of information.
“We really believed that the data would be available. It had been historically, and we anticipated that it would continue to be,” Westervelt said.
In the past, Adler has mined the Education Data Warehouse to feed his program, but the state took it down in conjunction with the new SBAC test and has yet to replace it, he said.
In 2012, Vermont was awarded a $4.95 million grant from the federal government to build a statewide data system covering kindergarten through 12th grade. The project was supposed to be done by June 30, 2015, but a one-year extension was granted, according to the latest report on it.
“There is a really powerful effort to shut down quantitative analysis in the education space, which does a disservice to those trying to use data responsibly to serve kids and improve schools,” said Westervelt.