What is the Most Important Thing We Can Learn from this Pandemic?
“But, Lord! How sad a sight it is to see the streets empty of people, and very few upon the ‘Change. Jealous of every door that one sees shut up, lest it should be the plague; and about us two shops in three, if not more, generally shut up.”
In 1944, President Roosevelt sent a letter requesting Vannevar Bush, Director of the Office of Scientific Research and Development, to propose a plan for applying similar research principles used in the recent war (WWII) and apply them to the war on disease. The Bush report, “Science, The Endless Frontier,” led to the development of the National Science Foundation (NSF) and then the National Institute of Health (NIH). The model that was applied is termed the ‘Linear Model of Innovation’ and its focus is on basic science using reductionist tools, like controlled research trials.
One example of this reductionist model for innovation is the Norden bombsight. An engineer, Carl Norden, wanted to build a bombsight that improved accuracy to allow bombers to fly at a safer height. He developed a complicated machine (about 2,000 parts) based on his expertise in gyro stabilization. In controlled testing, the bombsight performed brilliantly. The CEP (a circle into which 50% of bombs would fall) was 75 feet, a fantastic performance for that time. Without testing in the real-world, the Norden bombsight was mass produced and tens of thousands were installed in planes for the US Navy and Air Force at a cost of nearly $10,000 each.
When the bombsight was eventually used in real-world conditions, the CEP increased to 1,200 feet, a decay of accuracy of over 90% – about the level of decay in accuracy for our current airport screening process during real world conditions versus controlled testing. The controlled environment did not account for several factors that might affect accuracy, like weather conditions, mechanical issues, and that the person operating the bombsight might be getting shot at. The performance was so bad its use was abandoned soon after it was introduced into combat.
A different model for innovation comes from the principles of systems and data science in real-world conditions, utilizing analytics and feedback loops providing iterative improvement. One example of this is the Wright brothers’ successful achievement of flight. They used real-world conditions, collecting and analyzing data with feedback loops to improve their designs. At one point, they tested dozens of wing surfaces over two months, analyzing the data as it was generated. After their success, one of the Wright brothers commented, “It is doubtful if anyone would have ever developed a flyable wing without first developing this data.”
If we continue to use reductionist tools for the COVID-19 pandemic, we will continue to require the same management strategies we applied almost 400 years ago during the plague – shelter-in-place, social distancing, quarantines, etc. And our linear model for innovation, waiting for the results of controlled clinical trials, will result in lengthy delays and unintended waste and harm when treatments are introduced into real-world patient care.
Controlled studies attempt to determine if a treatment works or doesn’t work and generates recommendations for the average patient. This one-size-fits-all approach is not appropriate for a complex biologic system. With an appropriate data analytics infrastructure in place, we could identify different patient subpopulations and apply the optimal variety of treatments matched appropriately to these subpopulations. This is the type of analytics Netflix uses to present the optimal variety of movies and shows to different subpopulations of people (thank goodness for Netflix and their algorithms during this pandemic).
Many treatments are already being used in clinical settings to improve outcomes for patients with COVID-19, but without a data infrastructure in place that can measure outcomes and assess the value of any treatment in different subpopulations, we are flying blind. We could also use a data infrastructure to identify the factors that lead to a subpopulation that gets the virus and has minimal or no symptoms, one that has moderate symptoms, and one that develops a severe illness and is at risk of dying. The best we have now in the real world is doctors and nurses posting their observations on social media and other forms of communication. But without a data and analytics infrastructure, these are just anecdotes.
A real-world data infrastructure should include input from the front-line clinical team in each clinical environment. This would enable a human-computing symbiosis where the clinical team could identify the most important patient and treatment factors that would influence outcomes that measure the value of care. The dataset could then be uploaded into analytics and visualization tools and the clinical team could then interpret the results and apply insights to improve measurements and generate ideas to improve outcomes. Each clinical team could then be networked with other teams to share their learnings and improve algorithms to identify subpopulations for an optimal variety of treatment and preventative measures, termed the ‘Ensemble Model for Learning.’
Our healthcare system has spent billions of dollars for a data infrastructure designed for documentation, coding, and billing. We have fragmented electronic medical records, hundreds of revenue cycle management software products, and armies of coders and billers (guided by the system) in every clinical environment, all focused on getting paid. If we applied a fraction of those resources to a data infrastructure to support the measurement and improvement of value-based outcomes for real-world patient care management, we would have a sustainable healthcare system globally and this pandemic would likely be managed and resolved by now. I believe this type of data infrastructure is the most important lesson to learn and apply from the tragedy of this pandemic.
Recently during a Senate hearing about the lessons learned from our response to COVID-19, Senator Romney asked Dr. Robert Redfield, the CDC Director, “How is it possible in this day and age that the CDC has never established a real-time system with accurate data?” Dr. Redfield responded, “The reality is there is an archaic system… This nation needs a modern, highly capable data analytic system that can do predictive analysis. I think it’s one of the many shortcomings that have been identified as we went through this outbreak, and I couldn’t agree with you more, it’s time to get that corrected.” I agree.
(The content of this blog post will also be published in the June issue of General Surgery News)