State announces ‘open data’ pilot project

As Vermont’s state government takes its first baby step into the giant world of open data, the state’s civic hackers are lining up to help.

Harry Bell of Vermont’s Department of Information and Innovation announced Tuesday that the state would be stepping out of its website shell and into the “open data” movement — a growing international trend toward making government data more available to the public.

WHAT’S A CIVIC HACKER?

Civic hackers use computer skills and public data to solve civic problems. Often the computer programs they invent are open-source, meaning they can be freely used and adapted by others.

In line with principles of “proactive disclosure,” which makes public information freely available before anyone has to ask for it, open data is the product of a systematic methodology that seeks to catalog, structure, unify and publish as much government data as possible.

At an Open Data Consortium in Montpelier Tuesday afternoon, coordinated by the Vermont Center for Geographic Information, Bell described his job as using the Internet to facilitate communication between the state and the rest of the world. To date, that communication has consisted of a hodgepodge system of websites, most cut off from each other by separate back-ends and proprietary formats.

“The rest of the state’s interaction with open data kind of starts with last week on Tuesday,” Bell said.

And he wasn’t kidding.

The state signed a contract the first week of October to publish 10 data sets on a new Web-based portal. The nature of the contract — tiny compared to full-fledged open data endeavors — marks a substantial progression beyond the state’s scattered and hesitant forays into making public records available online.

To the civic hackers, software developers, website designers and state government workers attending the consortium, the news resonated like a pending sea change in the way the government and people of Vermont communicate and work together for public good.

Open data vision

“Open data should be the mantra” of state government, one attendee suggested.

His comments helped kick off a wide-ranging discussion that followed three hours of presentations by a diverse lineup of panelists from the government, corporate and civic hacker communities.

State officials, civic hackers and techies put their heads together Tuesday at an Open Data Consortium in Montpelier. Photo courtesy Nick Floersch.

State officials, civic hackers and techies put their heads together Tuesday at an Open Data Consortium in Montpelier. Photo courtesy Nick Floersch.

“But they have finite money and time,” countered keynote speaker John Cohn, an IBM fellow in corporate technical strategy and a self-confessed newcomer to open data. “Can we give some input to the state?”

The cost of open data is not just temporal and financial, but also political, an employee from the Agency of Agriculture added. She said her office faces a strong constituency of people who do not want the agency posting its data online.

“There’s a strong privacy lobby in this state,” she warned.

VCGI’s outreach coordinator, Leslie Pelch, who organized the consortium, observed a generational gap in some of the polar opposite approaches being voiced.

“We can’t dismiss that,” she said. “We are going to have to deal with the fact that there is this expectation” of total openness among the younger generation.

Within a half-hour, the roughly two dozen people remaining from the afternoon established that long-term cultural and operational change in the government’s approach to data would require incremental progress and a collaborative attitude on the part of the public.

And patience. And prioritization. And their help.

“If you can show a demand and anticipate value,” Cohn said. With a gray shock of frizzy hair on a balding head, he looks something like a mad scientist.

“If we could say, here’s three kinds of data we’d really like to see and here’s how we’re going to use it,” he continued, warming up. “Especially if we could offer the tech support to do it!” The idea caught fire.

Another presenter, Jim Duncan of the Vermont Monitoring Cooperative, said Massachusetts estimates it saves $2 million on Freedom of Information requests state employees no longer have to respond to: The information is proactively “open” and the public can peruse a catalog of data on its own. Duncan qualified, however, that he didn’t know the methodology by which the state arrived at that figure.

Code for BTV’s Bradley Holt chimed in. “It’s not just interesting things we’re going to do,” he contributed to the hypothetical conversation now taking place between the hackers and agency officials. “But what we’re going to do furthers the mission of your agency!”

Pelch kept the idea rolling. “That would be an interesting hack-a-thon,” she said with a smile. “Pair the state agency people with hackers.”

WHAT’S A HACK-A-THON?

Hack-a-thons bring together a group of people (often civic hackers) for a day or more of collaborative computer programming. Hackers often work in teams, and many times try to solve a central problem for a community.

Within minutes, the next open data event was conceived, before the group had even decided whether they should create an email listserv or a Web-based forum to keep their conversation going.

Opening the state’s data

Excitement aside, Bell emphasized in his talk that several barriers exist before the state can commit fully to open data.

One challenge that may be more significant for Vermont than other states, he said, is the “legacy” information systems currently employed. In other words, the antiquated state of Vermont’s information technology architecture.

“Most of Vermont’s systems were built well before (data sharing) was even a concept,” Bell said. Much of the state’s data is still “locked up” in mainframe computers, a legacy that will take significant time and resources to overcome.

“There would have to be considerable modernization in various systems to make that even possible,” Bell cautioned.

Privacy concerns will be tricky to navigate, too. Trevor Lewis, a records analyst from the Secretary of State’s archives and records administration, said public records law has not kept pace with the notion of “practical obscurity.”

For a great many records, privacy arguments are a moot point because the records themselves are so obscure that they’re never made public. But digitization and data access unlock a “mosaic effect” that brings together previously unseen or disconnected pieces of information. When viewed or analyzed in relation to one another, new information can be gleaned about the potentially private matters of the people at the heart of those data.

Fear of misuse is another worry, Duncan said. Open data means that information is free to be used in any way, even if its analysis is either unintentionally or nefariously misleading.

The demand for proven, quantifiable benefits to justify the cost of opening data is another common barrier governments must overcome, he said.

Additionally, the concept of open data — predicated as it is on electronic government operations — is distinct from the aim of establishing “e-goverment” systems. “They overlap but they’re not the same thing, and mixing those two can sink an open data effort,” Duncan emphasized.

Bell said the pilot project is intended to be just a proof of the concept to “get things rolling.” If successful, the goal simply will be to continue moving forward. Success will be measured, in part, by how well the state’s geographic information systems and financial data can interface with the open data platform.

The company Socrata holds the one-year contract, worth about $12,000, Bell said.

Three of the first 10 data sets to be “opened” have already been chosen from the existing Spotlight on Fiscal Transparency website, Bell said. They show expenditures to various vendors and data about state salaries, he said.

And he’s taking nominations for the remaining data sets to be used as guinea pigs.

“If you’re aware of state data sets you’d like to see exposed, feel free to share your ideas,” he said. Bell’s email address is [email protected]

“Before the year is up, we’ll make a decision about how to proceed, assuming the pilot is successful.”

Correction: This article was corrected on Friday, Oct. 11, 2013 to correct the spelling of Leslie Pelch’s name and reflect that some comments came from an employee at the Agency of Agriculture. 

Follow Hilary on Twitter @nilesmedia

Comments

  1. timothy price :

    This is exactly what is needed to evolve a shared goal into a happy future for Vermont, or anyone else, who wants to enjoy the benefits in a new world where the “streets are made of glass”. :-) Great idea.

  2. Arthur Hamlin :

    This seems like nothing more than a fad with no real purpose. No one has proven how open data is better or saves money but every day people’s lives are ruined by government and corporations playing fast and loose with their personal information. The state should spend some money fixing its own antiquated computers and not throw more money down the black hole of outside IT contractors.

  3. David Usher :

    This is a useful step in the right direction. It’s important to clearly define the data sets to be made available and how they will be kept current. Old data is helpful, but new data is essential to monitor today in addition to yesterday.

    Parameters must be established for the accuracy and completeness of the data sets.

    Unlocking the legacy data is a costly job, but Vermont needs a workable plan to accomplish it.

  4. frank seawright :

    I hope that the folks involved in this are alert enough to ensure that people using Linux operating system can use the data. It is not open if people are forced to buy and use microsoft windows so they can run plugins lik microsoft’s Silverlight – the Vermont Center for Geographical information is Windows heavy and silverlight is needed to use their data and the same is true for the ANR’s biofinder. And the foxpro based NEMRC for property tax determination and collection.

    Here’s a link to the VT Accessibility Policy :
    http://www.vermont.gov/portal/policies/accessibility.php

  5. Just to clarify Frank Seawright’s comments: Silverlight is indeed needed to use VCGI’s map viewer (an application, not data). We anticipate that the next version of that viewer will NOT require a plugin. Our data is in a format that does require special software, but it can be used by open source GIS software like QGIS. Great to highlight these issues, thanks.

    • frank seawright :

      “We anticipate that the next version of that viewer will NOT require a plugin.” Excellent.

      I should make myself a bit more clear about the NEMRC software; Silverlight is not required for it, but Microsoft’s FoxPro is. That software should be written in Python or similar so that towns are not forced to use Microsoft’s operating system and applications.

  6. Keith Vance :

    Without a funded mandate to open data, the agencies will not be able to do this on a grand scale.

    It’s good to Vermont moving in this direction. Openness should be a goal of every level of government.

  7. Keith Vance :

    Also, it’s odd to make fun of how one of the speakers looks.

Comments

*

Comment policy Privacy policy
Thanks for reporting an error with the story, "State announces ‘open data’ pilot project"