Source – https://knowledge.wharton.upenn.edu/
A new machine learning approach to COVID-19 testing has produced encouraging results in Greece. The technology, named Eva, dynamically used recent testing results collected at the Greek border to detect and limit the importation of asymptomatic COVID-19 cases among arriving international passengers between August and November 2020, which helped contain the number of cases and deaths in the country.
The findings of the project are explained in a paper titled “Deploying an Artificial Intelligence System for COVID-19 Testing at the Greek Border,” authored by Hamsa Bastani, a Wharton professor of operations, information and decisions and affiliated faculty at Analytics at Wharton; Kimon Drakopoulos and Vishal Gupta from the University of Southern California; Jon Vlachogiannis from investment advisory firm Agent Risk; Christos Hadjicristodoulou from the University of Thessaly; and Pagona Lagiou, Gkikas Magiorkinis, Dimitrios Paraskevis and Sotirios Tsiodras from the University of Athens.
The analysis showed that Eva on average identified 1.85 times more asymptomatic, infected travelers than what conventional, random surveillance testing would have achieved. During the peak travel season of August and September, the detection of infection rates was up to two to four times higher than random testing.
“Our work paves the way for leveraging [artificial intelligence] and real-time data for public health goals, such as border control during a pandemic,” the paper stated. With the rapid spread of a new coronavirus strain, Eva also holds the promise of maximizing the already overburdened testing infrastructure in most countries.
“The main issue was, given the fixed budget for tests, whether we could conduct the tests in a smarter way with dynamic surveillance to identify more infected travelers,” said Bastani. One of the biggest challenges governments face in dealing with COVID-19 is the inability of the testing infrastructure at their national borders to realistically check every arriving passenger. Such comprehensive testing would be both costly and time-consuming, which is why most countries screen either arriving passengers from specific countries or conduct random testing for COVID-19.
“The main issue was, given the fixed budget for tests, whether we could conduct the tests in a smarter way with dynamic surveillance to identify more infected travelers.”–Hamsa Bastani
Eva also allowed Greece to identify when a country was exhibiting a spike in COVID-19 infections a median of nine days earlier than what would have been possible with machine learning-based algorithms using only publicly available data.
The underlying technology of Eva is a “contextual bandit algorithm,” a machine-learning framework built for “sequential decision-making,” taking into account various practical challenges like time-varying information and port-specific testing budgets, Bastani explained. The algorithm balances the need to maintain high-quality surveillance estimates of COVID-19 prevalence across countries and the allocation of limited testing results to catch likely infected travelers. Eva is the first instance of that technology being applied to address a public health challenge, although such algorithms have found use in online advertising and A/B testing, she added.
Overcoming Data Challenges
Eva is an advancement over conventional border control policies because it does not rely on publicly reported data, which has a number of issues.
Publicly reported data is of “poor quality” chiefly because different countries follow different reporting protocols and testing strategies. It is common to focus testing resources on symptomatic patients, but the resulting prevalence rate may not be reflective of the asymptomatic population that is likely to travel. There is often also a reporting delay due to poor infrastructure, said Bastani. “We can tell, based on the data we’re actively collecting at borders, that a country’s COVID cases are spiking typically nine days before you will see that reflected in the public data.”
“Testing is usually targeted towards symptomatic individuals rather than asymptomatic individuals,” Bastani said in an interview with the Wharton Business Daily radio show on SiriusXM last July, as the Greek deployment was getting underway. (Listen to the podcast from that episode above). “You can imagine tourists who are coming in are probably asymptomatic.” That underscores the criticality of not relying on publicly reported data, but using data that accurately reflects the prevalence of asymptomatic COVID-19 travelers across countries.
Eva’s algorithm overcomes the poor quality of public data by dynamically collecting testing results at the Greek border, thereby maintaining high-quality surveillance estimates of the prevalence in each country. “By adaptively adjusting border policies nine days earlier, Eva prevented additional infected travelers from arriving,” the paper noted, referring to the Greece deployment. “That is a long period of time in which a lot of high-risk people would probably have come in and infected other citizens,” said Bastani.
It is common for border control policies to use publicly reported data, but such data are often unreliable and inconsistent across countries, said Bastani. The inconsistencies arise from censorship of testing data by some countries, and even varying definitions of a COVID-19 death, she added. She pointed to the recent discovery of undercounting of COVID-19 deaths in nursing homes in New York City as an example of flawed data. “That issue is exacerbated when you compare death counts in different countries because in some places they’re accounting very accurately and in other places they’re not.”
Greece is the first country to design border controls based on the dynamic random surveillance testing approach that Eva uses. The model specifies the infrastructure required to collect COVID-19 test results, using those to form estimates and to inform future testing decisions in a dynamic feedback loop.
“No country should just be relying on public data; they should be actively monitoring who is coming to their borders, testing at least a subset of them, and using that to make informed decisions about border control.”–Hamsa Bastani
In using the Eva model, Greece required every individual or family planning to enter the country to fill out 24 hours before arrival a digitized “Passenger Locator Form,” where they provided some basic information about themselves such as other countries they have visited in the past year. All those who submitted those forms received a QR code that allowed tracking. Eva’s algorithm processes the information in the forms to identify those who need to get tested for COVID-19. Greece’s border control authorities processed an average of 38,500 forms each day; some 18% of those who submitted the forms did not eventually show up.
Keeping COVID-19 at Bay
Eva’s targeted testing that allowed for adaptive border control policies helped Greece keep its case count “very low pretty much all of the summer” said Bastani. The country was able to maintain some economic activity, unlike many others that had to completely shut down, she noted. Greece imposed a second lockdown and travel restrictions in November after a spike in COVID-19 cases.
The Greek government acknowledged Eva’s accomplishments in a press conference last July. “The AI system developed by Bastani, Drakopoulos, Gupta, and Vlachogiannis has been an asset both for preparing the opening of the country to visitors from all over the world, as well as for allowing flexibility in decision-making regarding our COVID-19 strategy,” said Nikos Hardalias, Greece’s civil protection and deputy minister for crisis management, who heads the COVID-19 Response Taskforce for the country.
Free-to-use Technology
Eva is an open-source technology, which means Bastani and her team will provide it free of cost to any country that might want it. They have made presentations to COVID task forces in several countries in the European Union. Adapting it to other countries would involve designing passenger locator forms that are appropriate for different immigration processes and dovetailing back-end resources such as testing labs.
Bastani made a strong pitch for governments to capture private data such as that generated by the passenger locator forms used in the Greece deployment, and customize them to suit their specific situations. “No country should just be relying on public data; they should be actively monitoring who is coming to their borders, testing at least a subset of them, and using that to make informed decisions about border control,” she said. “That said, if a country doesn’t have the resources to do that, it’s probably better to use a policy that mimics another country that is doing that rather than relying only on public data.”
Bastani and her colleagues are working on refining Eva to incorporate more passenger-specific information than they used in the Greece deployment. Europe’s General Data Protection Regulation limited the scope of data they could use with Eva; they used only anonymized and aggregated data with limited demographic information. Other countries with less stringent data protection regulations could gather a wider range of data, such as on occupation, Bastani said. “We know that certain occupations carry a much higher COVID-19 risk than others.”
Eva could also be trained to incorporate pooling to mitigate constraints faced by testing labs, she added. Overloaded labs could share their samples with other labs that may have spare capacity at any given point in time, she explained. In much the same way, Eva could also use dynamic data to help determine optimal staffing levels at labs and other locations in the testing infrastructure, she added.