The federal No Child Left Behind Act, which required the nation’s schools to make “adequate yearly progress” by meeting proficiency standards on statewide tests, has been distressing Vermont schools for more than a decade.
As of last year, the 2001 law expected every public school across the country to reach 100 percent proficiency, but nearly every school in Vermont failed to reach the goal, leaving 290 schools with a black mark. Like more than half the states, Vermont has a waiver – a two-year reprieve from using the test scores for accountability purposes while it transitions to the Smarter Balance Assessment Consortium (SBAC) test.
A recently released Web-based tool called Polarys promises to help educators look at the information gleaned from the tests in a new way, with a “much better performance indicator” that levels the playing field and allows administrators to use data to improve their school, according to its creators at ReContext Data Solutions.
“By controlling for factors that affect state exam results but are outside the school’s control, we are able to demonstrate the changes in outcome at the district, school and classroom level,” according to the ReContext website.
The point of testing students annually is to make sure that the most vulnerable students are not falling behind. The idea was to provide educators with the kind of classroom level information that would help them help their students. Instead, a more cynical culture emerged, according to Polarys architect David Adler.
“Part of the problem with standardized exams is that educators believe the data is useless and rigged,” said Adler, who is also the curriculum director for the Orleans Southwest Supervisory Union.
The Annual Yearly Progress indicators are based on high stakes tests that can be misleading, Adler said. Schools that score well on the exams can miss opportunities to improve and a “bad” school may not have too far to go to raise performance criteria, Adler said.
Teachers, principals and administrators have access to a lot of data these days. Adler designed Polarys to turn it into actionable information using much more than just test scores. While improved student achievement is the ultimate goal there are many variables that can impact the outcomes.
“Factors like programming, school structures, spending, student demographics and teacher characteristics all impact outcomes. Figuring out what changes to make, if any, can seem like a Herculean feat,” according to Adler.
For example, the state average for proficiency might be 87 percent while a school is scoring at 60 percent, says Adler. It might look like the school needs to make up 27 percentage points, but when applying Polarys algorithms it turns out the gap between where the school is and where it should be is really only about 8 percent. At this point, an administrator can compare performance with similar schools and find out what they are doing to close the gap.
Put another way, Polarys works by normalizing NECAP results for factors that are beyond the school’s control, such as poverty or special education.
“We can norm the data and come up with a number that represents school performance,” said Mary Westervelt, director of operations for ReContext.
The free program also creates a snapshot by grouping schools with others that have similar characteristics. This feature allows administrators to compare performance in an “apples to apples way” and collaborate on solutions.
Richard Boltax, the school improvement coordinator at the Agency of Education, says Polarys is a “practical and useful tool.”
Boltax’s job is to help schools develop improvement plans and he finds that the principals and superintendents that he works with like Polarys because it is simple to use, automatically generates graphs and presents information in a way that is understandable.
All of the same raw data is available on the AOE’s website. “You have to know how to work it and comb through the data,” said Boltax. Adler’s program helps sort it and package it in a straightforward manner.
Adler has been designing the tool for the better part of a decade with the most significant adjustments coming in the past three years. “It started out as an Excel spreadsheet,” he said. He used publicly available information on school performance, demographic and structural data from the state. Polarys will soon integrate the SBAC results into the matrix.
About 50 school leaders in Vermont have been working with the Excel-based product since 2013. But this summer, Polarys was launched as a Web-based tool and is being offered in partnership with the Vermont Rural Education Collaborative free to Vermont educators for the next 18 months through an anonymous grant.
“Small schools provide many personalized learning opportunities for students, it will be helpful to have data that shows the impact of these opportunities on students success,” said Margaret Maclean, president of the board of directors for VREC and director of the Vermont Rural Partnership. The VREC feels it is particularly important for small rural schools to have tools like Polarys that sift through a wide range of data, Maclean said.
“What I really like about the program,” said Boltax, “is how it immediately pulls up the demographically similar schools so you can compare from school-to-school. It is a practical quick tool to paint a good picture based on accountability.”
Michael Hock, director of assessment at the Agency of Education said that American Institutes for Research, Vermont’s test contractor, provides tools and trainings on test interpretation and data usage. The agency welcomes anything that helps schools make better use of their data.
Vermont is a very collaborative environment, Adler said before adding that the purpose of Polarys is to provide meaningful analysis, meaningful feedback and spark meaningful conversations that will lead to our goal of educating students well.
