Officials debuted Vermont’s map, which sets parameters for out-of-staters traveling to the state, on June 6. Since then, the catchment area has expanded. Now millions of people in the region can come to Vermont without having to quarantine.
It’s based on a single, simple metric: whether the county they come from has fewer than 400 active Covid-19 cases per 1 million residents.
But one expert has raised concerns about how the state calculates those numbers. And he believes the standard that Vermont has set is both dangerously arbitrary and unfairly restrictive.
“Unfortunately, the fact is that the state simply can’t do this. The state cannot calculate these rates with the claimed degree of precision,” said Dr. Remington Nevin, a medical doctor and epidemiologist based in White River Junction. “In fact, they can’t calculate these rates with anywhere close to that degree.”
Vermont was the first state to allow cross-state travel on a county-by-county basis. (In-state travel is not restricted.) Michael Pieciak, commissioner of the Department of Financial Regulation, says the state was forced to set its own standard. Pieciak has served as the lead in modeling in the Scott administration’s response to the coronavirus crisis. The Vermont Department of Health has consulted with DFR on Covid-19 data.
“We’re really the first state in the country, certainly, pretty much the first jurisdiction in the world that contemplated this, and it’s the first time we’ve had a pandemic of this level in 100 years,” he said. “So there really wasn’t scientific evidence or scientific literature that we could rely on.”
Nevin said the methodology is questionable.
“What they’re attempting to do is entirely novel, and without precedent, and there’s a reason for that,” he said. “No one else is doing this because the data simply don’t support it.”
After VTDigger asked DFR officials about Nevin’s concerns, a member of the state team contacted Nevin with more details about their process for calculating the standard. They have promised to take a deeper look at some of his questions and concerns.
Precisely how many people are taking advantage of the county by county travel guidance is uncertain. About 1,700 travelers have signed up since July 10 to be monitored with the state’s Sara Alert tracker, a system that reminds people to check for symptoms. Compliance is voluntary.
The state’s ability to permit travel while preventing the spread of Covid-19 has major implications for the tourism industry, second homeowners and temporary students and workers.
Issues with underlying data
The problem with Vermont’s county-by-county active case rate starts with questions about the underlying data, Nevin said.
He’s not the first to raise concerns about the quality of Covid-19 data nationwide. Advocates and researchers say there are reporting delays associated with the data and each state has different standards for reporting testing and hospitalizations.
There is also no single, official data source for up-to-date case numbers, leading volunteer efforts like the Covid Tracking Project to fill in the gaps. In this case, Vermont chose to use the Johns Hopkins University dataset, a well-regarded project that nevertheless still can’t make up for the poor quality of the underlying state data, Nevin said.
“It’s a great little table. It’s a great little tool. But I guarantee you, the people that maintain that table would absolutely not support its use by Vermont in this manner. That’s not what the data is meant to be used for,” he said. (Nevin earned two degrees at Johns Hopkins and has taught there.)
When asked about Vermont’s use of data, a spokesperson for Johns Hopkins said their dashboard is intended to help policymakers with decisions but did not comment on the specific model Vermont uses.
Even in states that issue timely and accurate numbers, there is a good chance those numbers don’t reflect the true case load of residents because of low testing rates, Nevin said.
Several states within Vermont’s travel range aren’t meeting testing goals established by the Harvard Global Health Institute to contain their own outbreaks — much less provide a standard for travel.
County level reporting isn’t available for most states, so it’s hard to say whether a low case count is due to a low case rate or low testing rate.
In small populations, like counties, whether they meet the state’s standard may be due to random chance, Nevin said.
“If you’re only testing a very small percentage of residents in a particular jurisdiction, you can’t be sure if that very small group is representative of the entire population,” Nevin said.
Nevin gave the hypothetical example of a county nearby that has a population of 600,000.
If that county actually has about 400 active cases per 1 million people and tested about 1% of its population per week (in line with Vermont’s testing rate), it could have anywhere from 100 to 1,200 positivity rate per million — a pretty broad margin of error.
“Simply by chance, such a jurisdiction would commonly measure above the threshold by fairly large margins,” he said via email.
Nevin modeled one particular county in the data, Franklin city county in Virginia, and found that a basic version of the state’s model shows that the county could have anywhere from half to 3-4 times the reported case rate, because of the margin of error. (His methodology is at the end of the article.)
Nevin reached out to Vermont’s state epidemiologist, Patsy Kelso, at the beginning of the pandemic to offer his skills as a consultant, but never got a response.
More recently, he has been going back and forth with state officials to get more details about the state’s methodology. He hopes to be able to calculate each county’s margin of error himself to see how far-reaching a concern it is.
State assumptions
Once the state gets the case data, it uses its own, newly created method to estimate whether a case is “active,” meaning that the person still has symptoms or is potentially infectious.
Scientists are still uncertain the degree to which asymptomatic cases are responsible for the spread of the disease, or how long a person can be infectious after they’ve contracted it. Pieciak said the state developed an internal approach to active cases that they are comfortable with.
The state model assumes that people are most infectious at the time they’re symptomatic, and uses the date they tested positive as a proxy for when they’re symptomatic, according to their published methodology.
Nevin disagrees with that model. Someone could get Covid, develop symptoms for the first 7 to 10 days, and then get tested on day 21 when they’re no longer symptomatic — yet still test positive because of the small amount of virus left in their system.
“The date of a positive PCR [polymerase chain reaction] test is almost meaningless in the scheme of truly identifying risk, because it has to be interpreted, ideally in the context of your symptoms and exposure to other people that you may have contracted it from,” he said.
After calculating an estimate of active cases, the state determines whether that county meets the standard of 400 active cases per million people. But there’s no hard evidence that number will prevent any cases spreading to Vermont.
Pieciak said the state came up with 400 as a “reasonable” number, and pointed to states like New York and New Jersey that have imposed similar standards.
But he said the state’s permission doesn’t mean these are “safe” counties. “There’s always an element of risk,” he said.
That element of risk applies to Vermont, too. Several Vermont counties have at times had more than 400 case rate — meaning that if they were to be included under the travel policy, they’d be restricted, too.
The restrictions require travelers from regions above 400 to quarantine for 14 days — or 7 days with a negative test — either in their home state or at a Vermont lodging property.
In response to questions about Vermont’s “failing” counties on June 12, Pieciak said the state only needed the 400 metric for other states because they lacked the granularity of data available in Vermont.
“We know that increase in Chittenden County from 159 to 814 is related to that outbreak in Winooski, and not necessarily, at this time, broader community transmission,” he said. “So that’s why we’re not necessarily applying that same metric to us.”
The state map doesn’t use the same color scheme for Vermont counties as it does for other states, instead opting to make those above-400 counties a deeper shade of blue.
VTDigger asked two other experts to review the state’s methodology. Easton White, a researcher at the University of Vermont, said that while the calculations were prone to error, that “doesn’t mean their approach is a poor one.”
“If 400 active cases per million provides enough of a buffer to account for these types of errors, I think it is fine to use,” he said via email.
But ultimately, the county-by-county method is a “pretty crude measure,” and testing new arrivals would be a better approach. “They could push for more testing for those that arrive as opposed to an automatic quarantine,” he said.
Dr. Ubydul Haque, a biostatistician and epidemiologist at the University of North Texas, said he agreed with Nevin’s criticisms.
Asked what system the state should use to allow for travel, he said, “I will not encourage any travel.”
“However, if it is essential, yes, less risky areas (where active cases are very low), some travel can be allowed,” he said.
Is the state’s system fair?
Nevin says the state is also unfairly penalizing people based on where they live, independent of their individual risk of having the virus.
“The bigger picture here is that Vermonters are at risk of becoming comfortable with the idea that we should judge people, and judge the risk that they pose, based on some characteristic of theirs that they have no control over,” he said.
Place of residence is not a protected class in the United States, unlike race, sex or age. But in a country with a long history of systemic racism in housing and economic opportunities, identity can play a big role.
According to VTDigger’s analysis of the cross-state travel data provided by ACCD, a white, non-Hispanic person who lives in one of the states on the travel map is three times more likely to be allowed to travel to Vermont than a Black person.
That’s because white people are more likely to live in a county with a low enough case rate to be included on the list, meaning that Black workers, tourists and second homeowners are being excluded from the opportunity to come to Vermont.
The disparity is even more stark for people living in majority-Black counties: Every single majority-Black county within the 13 states and District of Columbia included in the map is restricted in traveling to Vermont.
Nevin said he believes the disparities from the policy to be “coincidental.” The algorithm the state uses doesn’t factor in race as part of its calculations.
However, research has shown that historic housing discrimination and racism has translated to modern-day segregation, meaning that white and Black people tend to live in different areas of the country.
Combined with data across the country showing that Black people are at higher risk of contracting and dying of Covid-19, it’s possible that systemic racism has played a role in the results, even without any intention from the Vermont government.
Despite the racial disparities, Meg Mott, a constitutional law expert at Marlboro College, doesn’t think a legal case against the state is likely, since any case against it would need to prove intentional discrimination.
“You do have a 14th Amendment argument for lack of equal protection under the law,” she said. “But that’s going to be a hard one to prove, because all the state needs to say is that it’s trying to protect against physical pestilence.”
The governor has pretty broad power to limit people’s movements for public health reasons, she said. “They just need a rational basis for why they’re doing what they’re doing.”
The governor extended the state of emergency on Tuesday to Aug. 15.
Nevin said the state’s decision echoes the injustice of racism and other forms of discrimination.
“We are strongly criticizing police for judging the risk of a driver who might happen to be Black, for example, and taking action based on that characteristic — we recognize that is fundamentally wrong,” he said. “And yet, at the same time, [we’re] supporting judging the risk of that same driver as elevated, simply because they happen to hail from a particular county.”
Asked about the disparities, Pieciak said the racial discrepancy in Covid-19 cases is troubling to everyone, but pointed out that travelers can still come to the state as long as they quarantine for the recommended time period.
In response to questions about the state’s methodology, Gov. Phil Scott said he had faith in his employees, including the state epidemiologist. He said the new policy has been in place for weeks.
“We know they’re here anecdotally, and we’re seeing the plates and so forth, and we haven’t seen a spike in terms of the number of active cases,” he said.
He asked if Nevin had reviewed the standards of other states, to see if any of them have similar concerns or a better method of figuring out who can come.
“It’d be great to find out because if somebody else is doing it better than we are,” Scott said, “Maybe we can learn from them.”
Nevin said in response that other states aren’t attempting to do what Vermont’s doing: allowing travel on such a detailed, county-by-county basis.
“EU nations, for example, have made travel prohibition decisions based on a comparable threshold, but on a country-by-country basis, where the precision of their estimates can be much greater,” he said via email.
“While New York, New Jersey, and Connecticut have likewise made similar decisions on a state-by-state basis, where the populations involved typically number in the millions.”
Even if Vermont were to adopt a different standard, there’s one other concern: Compliance. Vermonters have reported seeing out-of-staters who are not following the guidelines in swimming holes, downtowns and their local grocery stores.
Nevin is familiar with that concern, too: He’s seen second homeowners from Boston or New York in his own hometown who didn’t seem to be following the rules.
“The folks who own second homes here in large measure are not abiding by the governor’s quarantine orders,” he said. “Because they know full well it’s not going to be enforced.”
Part of that is honest people from red zones aren’t visible because they aren’t here. He said those well-intentioned people are being unfairly punished by this “unscientific” policy.
“If you outlaw travel from other states, the only people from out of state you’ll see will be the outlaws,” he said.
A note on methodology:
The state’s methodology and the data it compiles are both available on the ACCD website. To find out how the policy impacts Black Americans, VTDigger used Census 2019 population estimates for Black residents and white, non-Hispanic residents of all states within Vermont’s travel map. We used an odds ratio to compare the difference between Black and white people’s likelihood of being able to travel to the state.
Dr. Remington Nevin reviewed the ACCD methodology and consulted with DFR to recreate some elements of the state’s methods and find out the unpublished margin of error for a single county, as a way of illustrating the potential problems with the map. He used Johns Hopkins University case numbers to get the reported recent cases in a county, then used two modeling methods — a nonparametric approach, and a Poisson distribution — to get the estimated variability of the actual case numbers in that county. The margin of error under the state’s model might vary from his calculations due to differences in how the state accounts for changes in the data on a day-by-day and week-by-week basis.
Clarification: This article has been updated to include more specific information about conditions under which travelers from regions with higher incidences of Covid-19 cases can visit Vermont.
