On Modeling
It’s more difficult than it may appear. There’s tremendous pressure to get it right. And, it’s nearly impossible to predict whether it will be successful.
I’m not actually writing about fashion modeling, but scientific modeling. If your social media feed is at all like mine, many of your friends who were recently experts on impeachment have now become experts on epidemiological models. And, unfortunately, this pandemic has become politicized, like nearly everything in our country these days. Some left-leaning voters are blaming Republican governors for not ordering mandatory shelter-in-place, while some right-leaning voters have compared those governors (both Democratic and some Republican) who have issued such orders as fascist tyrants. And, both sides cite various scientific models and historical examples to bolster their claims. Can we know who is right?
All Models are Wrong, but Some are Useful
The title above is a fragment from several similar quotes by statistician George Box. It is true not only for statistical models, but also economic and scientific ones, to include models of our current pandemic.
Models are “wrong” in three fundamental ways:
First, all models are simplifications. They can never incorporate all the complexity inherent in any real-world problem. Some relevant data may be excluded because it is unknown, or because it is difficult to quantify and incorporate in a model.
Second, they may rely on inadequate or unreliable data. For example, in this pandemic, the number of known people who have been infected by SARS-CoV-2 is not known, due to inadequate testing.
Finally, all models depend upon the validity of the assumptions made in constructing them. If those assumptions are not sound, the model will be less accurate and helpful.
The experts doing these models understand all this. Most have already updated their models as new data was made available, and the serious researchers will continue to do so. These models should be considered as intuitions about the future, rather than predictions. They are estimates, not demonstrably factual. Or, to use a mildly pejorative term from my Army general surgery residency days, they are a “SWAG” (a scientific wild-ass guess).
Despite their limitations, these models are useful, and are the best tool we have available in the midst of a rapidly evolving situation.
Institute for Health Metrics and Evaluation (IHME) COVID-19 Projections Assuming Full Social Distancing through May 2020
An influential model, and the main one used by the U.S. administration, from the University of Washington. It has been updated multiple times.
Imperial College COVID-19 Response Team Reports
Another influential and well-known model from Imperial College London, which also has been updated multiple times.
Oxford Model: Coronavirus May Have Already Infected Half of U.K. Population
A more controversial model from Oxford University, which suggests that the coronavirus may have arrived months earlier than presumed. The link I had to the actual white paper is no longer valid, so I’m linking to a news story about the study.
Mathematics of Life and Death: How Disease Models Shape National Shutdowns and Other Pandemic Policies
A discussion about how pandemic models influence public policy.
A Call to Honesty in Pandemic Modeling
An interesting read, with a self-explanatory title.
I also bring up the issue of scientific models because there are two I would like to see, but have not found. One has to do with the “side effects” of our interventions to date, which Dustin and I discussed in my last post, Coronavirus Update. The second relates to how we hope to see this all end. Let’s take those one at a time.
A Model for Future Pandemics
Since a full analysis of the “side effects” of our current interventions will have to be done after the pandemic, it won’t benefit us now, but it should help us better plan for future pandemics. Again, that is not a case of if, but when.
There are some comparisons which have been drawn from the 1918-1919 influenza pandemic which have some value. For example, prior studies have shown that in those communities in which greater social interventions were taken, mortality from the flu was less. And, a recent study even found that a stronger pandemic response yielded a better economic recovery in those communities.
I would point out, however, that this is a different virus and a different time, so such comparisons may not be as valuable as they appear on the surface. A century later, our healthcare system is profoundly better, our society is much more interconnected, and our economy is more complex and integrated with the world economy. It would also be helpful to have a full analysis of more than one global pandemic in the modern era.
One analysis I have not found with regard to the 1918-1919 pandemic, and which I hope we will study with this one, is a comparison of all-cause mortality among different communities. So, focusing not only on deaths due to COVID-19, but all deaths; from heart attacks and cancer, to homicides and suicides, and everything in between.
Quartz: This Chart of the 1918 Spanish Flu Shows Why Social Distancing Works
MIT News: The Data Speak: Stronger Pandemic Response Yields Better Economic Recovery
According to the CDC, there are approximately 2.8 million deaths in the U.S. each year. Just over half of these deaths are due to cardiovascular disease (854,000) and various malignancies (599,000). Deaths from motor vehicle accidents (39,000) and falls (36,000) comprise a much smaller share. Drug overdoses (64,000), suicides (47,000), alcoholic liver disease (22,000), and homicides (20,000), what I will will call “society illnesses”, together comprise a still relatively small but larger segment (153,000 in total). In comparison, the current estimate of COVID-19 deaths from the University of Washington is 60,000 in the U.S. by August.
What do we know of the effects of our mitigation efforts to curb the pandemic on other causes of death? I don’t know, and as mentioned, I’ve seen no modeling done to even address that question. In particular, I’m concerned about the effects of our “social distancing”. (By the way, I’ve come to hate that phrase; we should practice “physical distancing”, but we need social ties now more than ever.)
In the past three weeks, we’ve lost more than 16 million jobs. That is approximately twice the number of people who lost their jobs over a two year span during the Great Recession, and in only three weeks! The unemployment rate, though not yet calculated to take into account those recent job losses, is likely over 10%; and some projections are that it might rise to 25% by the summer, which is comparable to the Great Depression. Hopefully, this economic downturn will be brief, but we don’t know if it will be.
The impacts of the virus, and our mitigation efforts against it, could cause a significant rise in deaths due to those “society illnesses” of drug overdoses, suicides, alcoholic disease, and homicides. Job loss and financial insecurity can lead to depression, substance abuse, crime, domestic violence, etc.
It could also result in an increase in deaths from other causes, to include cardiovascular disease and malignancies. We have put on-hold “elective procedures” for people with those ailments. If that lasts only for a month or two, that will probably not cause a significant rise in deaths, but what if longer? And, cumulative stress also increases the risk of developing some new diseases or exacerbating existing ones.
CDC: National Vital Statistics Reports: Deaths: Final Data for 2017
NYT: Jobless Claims Surpass 16 Million; Aid Package Stalls in Senate
There is a balance which needs to be found. To keep it simple, let’s focus on only three options, even while recognizing that there is actually a continuum of choices. For all three, we’re going to assume the same level of public health intervention and medical support; the only difference will be in how much we “socially distance”.
Option one would be no effort at distancing. This would clearly be a bad choice, resulting in periodic saturation of our healthcare system, resulting not only in more COVID-19 deaths, but also deaths from all causes.
Option two would be some intermediate effort at distancing. My guess is that it is somewhere within this broad range that balance can be struck.
Option three would be maximal distancing. At an extreme range, this would include mandatory stay-at-home orders with closure of all schools, churches, and all but a mere handful of businesses (grocery stores and pharmacies maybe being the only ones excepted).
My suspicion is, and I would like to see the research which would prove or disprove it, that the economic effects from maximal distancing will indeed reduce deaths from this pandemic, but might result in a more than commensurate increase in deaths from all other causes. To be most thorough, we would need to assess deaths from other causes not only for the duration of this pandemic, but for several years thereafter.
If I’m correct, and the research is done following this pandemic, we might be able to better tailor our responses in the future, to strike the proper balance. A more robust public health effort than we have experienced this pandemic is critical, to include prompt and widespread testing, contact tracking, and isolation of infected individuals.
Beyond that, we might find that we learn that we could practice “physical distancing” rather than “social distancing” to comparable effect. Wearing masks would be but one example. During a pandemic, maybe businesses could add “No Mask” to their “No Shirt, No Shoes, No Service” signs. If plentifully available, maybe businesses could even provide them to customers. Making hand sanitizer more readily available in commercial locations, as we are seeing now, might be useful, too. Already, on many cruise ships, one is required to use hand sanitizer before going into the restaurant.
Perhaps we could keep most businesses and schools open, people employed, and save lives by striking that balance. Wearing masks and using hand sanitizer on a regular basis during a pandemic would be different than what we are used to, certainly, but possibly more familiar than what we are experiencing now.
The Light at the End of the Tunnel?
Perhaps I’ve just missed them, but I haven’t come across any scientific modeling of a likely “exit plan” from this pandemic. There has been some speculation about how long we’ll need to keep up our “social distancing”, and some editorials addressing the same question I’m asking, but nothing that I would consider actual modeling of how this is likely to play out. Again, such a model would be wrong, but it might be useful.
The closest I’ve yet found is a report from the American Enterprise Institute. It lays out some good benchmarks, I think, but no timeline. An article from the New York Times provides a good summary of this report, too. While I think it’s a good start, it is AEI, which is a somewhat right-leaning organization; I’ll be interested in any feedback from my left-leaning friends about it.
To my friend who first asked me last week about a timeline for the end of this pandemic, this is the best I could find, for now. If anyone has any other good sources to answer this question, please let me know!
