Christians Shouldn’t Be Dismissive of Scientific Modeling
Image
Projections from an IHME model.
Over the last several weeks I’ve encountered a range of negative views toward the models epidemiologists have been using in the struggle against COVID-19. Skepticism is a healthy thing. But rejecting models entirely isn’t skepticism. Latching onto fringe theories isn’t skepticism. Rejecting the flattening-the-curve strategy because it’s allegedly model-based isn’t skepticism either.
These responses are mostly misunderstandings of what models are and of how flattening-the-curve came to be.
I’m not claiming expertise in scientific modeling. Most of this is high school level science class stuff. But for a lot of us, high school science was a long time ago, or wasn’t very good—or we weren’t paying attention.
What do models really do?
Those tasked with explaining science to us non-scientists define and classify scientific models in a variety of ways.
The Stanford Encyclopedia of Philosophy, for example, describes at least 8 varieties of models, along with a good bit of historical and philosophical background. They’ve got about 18,000 words on it.
A much simpler summary comes from the Science Learning Hub, a Science-education project in New Zealand. Helpfully, SLH doesn’t assume readers have a lot of background.
In science, a model is a representation of an idea, an object or even a process or a system that is used to describe and explain phenomena that cannot be experienced directly. Models are central to what scientists do, both in their research as well as when communicating their explanations. (Scientific Modeling)
Noteworthy here: models are primarily descriptive, not predictive. Prediction based on a model is estimating how an observed pattern probably extends into what has not been observed, whether past, present, or future.
Encyclopedia Britannica classifies models as physical, conceptual, or mathematical. It’s the mathematical models that tend to stir up the most distrust and controversy, partly because the math is way beyond most of us. We don’t know what a “parametrized Gaussian error function” is (health service utilization forecasting team, p.4; see also Gaussian, Error and Complementary Error function).
But Christians should be the last people to categorically dismiss models. Any high school science teacher trained in a Christian university can tell you why. I’ve been reminded why most recently in books by Nancy Pearcy, Alvin Plantinga, William Lane Craig, and Samuel Gregg: Whether scientists acknowledge it or not, the work of science is only possible at all because God created an orderly world in which phenomena occur according to patterns in predictable ways. For Christians, scientific study—including the use of models to better understand the created order—is study of the glory of God through what He has made (Psalm 19:1).
Most of us aren’t scientists, but that’s no excuse for scoffing at one of the best tools we have for grasping the orderliness of creation.
Should we wreck our economy based on models?
The “models vs. the economy” take on our current situation doesn’t fit reality very well. Truth? The economy is also managed using models. A few examples:
- Calculating the unemployment rate
- Unemployment forecasting (also this)
- Business forecasting
- Cost Modeling
Beyond economics, modeling is used all the time for everything from air traffic predictions to vehicle fire research, to predictive policing (no, it isn’t like “Minority Report”).
Models are used extensively in all sorts of engineering. We probably don’t even get dressed in the morning without using products that are partly the result of modeling—even predictive modeling—in the design process.
Christians should view models as tools used by countless professionals—many of whom are believers—in order to try to make life better for people. Pastors have books and word processors. Plumbers have propane torches. Engineers and scientists have models. They’re all trying to help people and fulfill their vocations.
(An excellent use of predictive mathematical modeling…)
Why are models often “wrong”?
An aphorism about firearms says, “Guns don’t kill people; people kill people.” Implications aside, it’s a true statement. It’s also true that math is never wrong; people are wrong. Why? Math is just an aspect of reality. In response to mathematical reality, humans can misunderstand, miscalculate, and misuse, but reality continues to be what it is, regardless.
The fact that the area of a circle is always its radius squared times an irrational (unending) number we call “pi” (π) remains true, no matter how many times I misremember the formula, plug the wrong value in for π, botch the multiplication, or incorrectly measure the radius.
The point is that models, as complex representations of how variables relate to each other and to constants, are just math. In that sense, models are also never “wrong”—just badly executed or badly used by humans. That said, a model is usually developed for a particular purpose and can be useless or misleading for the intended purpose, so, in that sense, “wrong.”
When it comes to using models to find patterns and predict future events, much of the trouble comes from unrealistic expectations. It helps to keep these points in mind:
- Using models involves inductive reasoning: data from many individual observations is used in an effort to generalize.
- Inductive reasoning always results in probability, never certainty.
- The more data a model is fed, and the higher the quality of that data, the more probable its projections will be.
- When data is missing for parts of the model, assumptions have to be made.
- Changes in a model’s predictions are not really evidence of “failure.” As the quantity and quality of data changes, and assumptions are replaced with facts, good models change their predictions.
- True professionals, whether scientists or other kinds of analysts, know that models of complex data are only best guesses—and they don’t claim otherwise.
- The professionals that develop and use models in research are far more tentative and restrained in their conclusions than people who popularize the findings (e.g., the media).
In the case of COVID-19, one of the most influential models has been one of IHME’s (Institute for Health Metrics and Evaluation). Regarding that model, an excellent Kaiser Family Foundation article notes:
Models often present “best guess” or median forecasts/projections, along with a range of uncertainty. Sometimes, these uncertainty ranges can be very large. Looking at the IHME model again, on April 13, the model projected that there would be a 1,648 deaths from COVID-19 in the U.S. on April 20, but that the number of deaths could range from 362 to 4,989.
Poor design and misuse have done some damage to modeling’s reputation. Some famous global-warming scandals come to mind. But in the “Climategate” controversy, for example, raw data itself was apparently falsified. The infamous hockey stick graph appears to have involved both manipulated raw data and misrepresentation of what the model showed. Modeling itself was not the problem.
(XKD isn’t completely wrong … there is such a thing as “better garbage”)
Why bother with models?
Given the uncertainty built into predictive mathematical models, why bother to use them? Usually, the answer is “because we don’t have anything better.” Models are about providing decision-makers, who don’t have the luxury of waiting for certainty, with evidence so they don’t have to rely completely on gut instinct. It’s not evidence that stands alone. It’s not incontrovertible evidence. It’s an effort to use real-world data to detect patterns and anticipate what might happen next.
As for COVID-19, the idea that too many sick at once would overwhelm hospitals and ICUs, and that distancing can help slow the infection rate and avoid that disaster, isn’t a matter of inductive-reasoning from advanced statistical models. It’s mostly ordinary deduction (see LiveScience and U of M). If cars enter a parking lot much faster than other cars exit, you eventually get a nasty traffic jam. You don’t need a model to figure that out.
You do need one if you want to anticipate when a traffic jam will happen, how severe it might be, how long it might last, and the timing of steps that might help reduce or avoid it.
Leaders of cities, counties, states, and nations have to manage large quantities of resources and plan for future outcomes. To do that, they have to make educated guesses about what steps to take now to be ready for what might happen next week, next month, and next year. It’s models that make those guesses educated ones rather than random ones.
Highly technical work performed by exceptionally smart fellow human beings is a gift from God. Christians should recognize that. Because we’ve been blessed with these people and their abilities (and their models) COVID-19 isn’t killing us on anywhere near the scale that the Spanish Flu did in 1918 (Gottlieb is interesting on this). That’s divine mercy!
(Note to those hung up on the topic of “the mainstream media”: none of the sources I linked to here for support are “mainstream media.” Top image: IHME.)
Aaron Blumer 2016 Bio
Aaron Blumer is a Michigan native and graduate of Bob Jones University and Central Baptist Theological Seminary (Plymouth, MN). He and his family live in small-town western Wisconsin, not far from where he pastored for thirteen years. In his full time job, he is content manager for a law-enforcement digital library service. (Views expressed are the author's own and not his employer's, church's, etc.)
- 180 views
[Andrew R.]It’s ironic that those who deride “the models” are working from a mental picture which they believe to approximate reality—i.e., they’re working from a model. It’s often a singularly uninformed model, based on a few poorly understood data points—and much more handwaving than mathematics—but it’s a model.
For the record, I have a BS in Mathematics with 2 classes in modeling and 3 in statistics, and 3 graduate classes with “modeling” in the name. If I don’t know it, I ask my wife with a PhD in statistics who does modeling for a living… Am I an epidemiologist? No. Do I know modeling? Yes. I said what I said because there is a need for pure modeling. Unfortunately, when you are dealing with things like pandemics and climate change, politics and money get involved fast.
COVID-19 model skeptics are winning the debate in the comments of this post, and they are winning everywhere else - except in politics and with the so-called experts. Even the experts are beginning to recognize they are losing credibility because too many people are learning that their earlier models were as significant a failure as Trump’s initial response.
Ashamed of Jesus! of that Friend On whom for heaven my hopes depend! It must not be! be this my shame, That I no more revere His name. -Joseph Grigg (1720-1768)
[Mark_Smith]Yes, Dr. Birx herself said this from the White House podium. I am also told Medicare pays a certain amount for a death listed as pneumonia, but a significantly larger sum if the doctor notes COVID-19.
OK, first of all, Medicare does NOT pay more because the death certificate says COVID-19. It is fee for service. I don’t know who your sources are, Mark, but they are dead wrong, and anyone familiar with how Medicare works could tell you this.
Moreover, regarding the notion that doctors would lie on death certificates to inflate COVID death figures to “match” the estimates, a death certificate is an official government document, and to lie on one is perjury, a felony. A doctor found to have lied on a death certificate, especially in an effort to get more money for treating patients, is likely to lose his medical license.
Guys, this is the kind of nonsense you’re listening to, and it does the cause of Christ no credit to spread this kind of claims around. Reality is that in most cases—not just COVID—getting to near 100% certainty in a diagnosis will explode medical costs. That’s why, when you’ve got pain in the chest, neck, or shoulder, you generally talk with a triage nurse before they send you in for an EKG and a consultation with a cardiologist (something I’ve experienced twice, thankfully with zero heart attacks). The reason New York added something like 3700 deaths to the official list is because the symptoms on their medical charts were consistent with COVID-19, and not consistent with other ailments of the lungs. Yes, a specific test for the virus/antibodies might be good, but sometimes, you’ve got high confidence of the cause and no particular need to administer a test.
While there is probably some error rate in these estimates, it is, again, par for the course. Quite frankly, it scares me the kind of things that are being said here. Brothers, there are consequences to belittling the efforts of those doing these models, specifically that the end result is to reduce vigilance versus a disease that could indeed kill hundreds of thousands to millions if we give it the chance. It’s the kind of thinking that would, if placed back in London in 1854, complain bitterly about walking a few blocks more for water while ignoring the fact that the cholera epidemic had abated.
Let’s get some perspective here; of course there are uncertainties in the models, and of course they change while people change their behavior. From the rhetoric here, you would think this was a conspiracy of people who did not pass 7th grade health class.
Aspiring to be a stick in the mud.
[Bert Perry]OK first of all, Medicare does NOT pay more because the death certificate says COVID-19. It is fee for service. I don’t know who your sources are, Mark, but they are dead wrong, and anyone familiar with how Medicare works could tell you this.
[snip] Guys, this is the kind of nonsense you’re listening to, and it does the cause of Christ no credit to spread this kind of claims around.
WRONG. Per the recently passed CARES act, hospitals do indeed receive higher payments for COVID patients (whether they die or not). From the liberal leaning factcheck.org:
“It is true, however, that the government will pay more to hospitals for COVID-19 cases in two senses: By paying an additional 20% on top of traditional Medicare rates for COVID-19 patients during the public health emergency, and by reimbursing hospitals for treating the uninsured patients with the disease (at that enhanced Medicare rate). Both of those provisions stem from the Coronavirus Aid, Relief, and Economic Security Act, or CARES Act.”
Your factual inaccuracies are the nonsense, and you are making false claims about those of us who are correctly and accurately pointing out the flaws in current models and reporting. It is NOT spreading false claims around or discrediting the cause of Christ to speak the truth (even if it makes you unhappy) and you should be ashamed of libeling fellow believers You owe Mark a sincere apology..
Do you watch the daily presidential press conference? Dr Birx SAID, I heard it with my own ears, that doctors were to write COVID on the death certificate if a person died of pneumonia, if they had “symptoms” of COVID 19, even if they did not have a confirmed test. That’s is my source for that little piece of information.
My source for the information about reimbursement is numerous stories I’ve read over the last weeks in the media. And for the record, I tend to read liberal or left-leaning media that comes up on my iphones News app. Places like NYT, LA Times, the Atlantic. These articles are on my daily default by the way, news feed.
No conspiracy brother.
Everybody’s not following the same rules for how they count a death as a COVID death or not a COVID death. As for the particular example you mentioned, Mark, if they made the decision to count that way, it would be worth finding out why. It is not automatically clear that this would result in overcounting. The concern is to avoid undercounting given the shortage of available tests. If you don’t have enough tests to test the already dead, you have to decide if it’s likely they died of COVID-19.
Mostly, I’m seeing both penumonia and COVID counts and then decision-makers have to decide how to interpret and use that info.
Note CDC’s actual counting process…. emphasis added:
The provisional data presented on this page include the weekly provisional count of deaths in the United States due to COVID-19, deaths from all causes and percent of expected deaths (i.e., number of deaths received over number of deaths expected based on data from previous years), pneumonia deaths (excluding pneumonia deaths involving influenza), pneumonia deaths involving COVID-19, and influenza deaths; (a) by week ending date, (b) by age at death, (c) by sex, (d) by place of death, and (e) by specific jurisdictions. Future updates to this release may include additional detail such as demographic characteristics, additional causes of death (e.g., acute respiratory distress syndrome or other comorbidities), or estimates based on models that account for reporting delays to generate more accurate predicted provisional counts. https://www.cdc.gov/nchs/nvss/vsrr/covid19/index.htm
This kind of info is very easy to find.
On the constitutionality of state measures question: this story is one example of a genuinely unconstitutional overreach by a governor:
https://www.christianpost.com/news/federal-judge-says-kansas-churches-c…
What happened here is that the executive order singled out churches and religious gatherings in ways that didn’t apply to other gatherings.
There have not been very many of these… and so far, judges seem to be doing a good job of reining them in. (Though in the Kansas case, it looks like they shot the rule down on a technicality rather than addressing the constitutionality question)
Views expressed are always my own and not my employer's, my church's, my family's, my neighbors', or my pets'. The house plants have authorized me to speak for them, however, and they always agree with me.
Thought I’d pass this one on. For those who feel a strong need to be contrarian, this piece argues that stay-at-home orders don’t work, though he thinks quite a few other measures do.
https://www.thepublicdiscourse.com/2020/04/62572/
He’s no epidemiologist, but has some overlapping expertise, and some background in working with models.
Views expressed are always my own and not my employer's, my church's, my family's, my neighbors', or my pets'. The house plants have authorized me to speak for them, however, and they always agree with me.
OK, yes, I’ll concede there is a surcharge. Note, however, that factcheck notes that there is no evidence of widespread mis-identification of symptoms as COVID. Here are, again, some reasons why:
1. The surcharge is not a cash cow for hospitals or providers. It is a way of keeping vulnerable hospitals open. A 15% addition on Medicare on 198 diagnosed patients in Olmsted County does not cover the 60% drop in filled beds for the 2000 beds at St. Mary’s and Methodist. Not even close. The numbers at your local hospital may not be as stark, but this is hammering hospitals.
(at Mayo, even some doctors working with COVID just got a pay cut, and a nurse I know who was working with COVID patients just got temporarily furloughed…it’s not like this is a cash cow for providers here)
2. Doctors know that misleading diagnoses and death certificates will get them expelled from the profession for life.
3. Quite frankly, doctors want to get back to work and the office, and that’s not possible (again) if the hospitals and clinics need to remain closed due to COVID.
4. COVID symptoms are actually pretty distinctive vs. those of the ordinary flu. (smell and taste, lung imaging, virulence, who is affected, etc..) Mark would like to make a huge deal out of whether lab tests (virus, antigen) were performed, but reality is that performing lab tests (which have false positives and negatives, too) only incrementally improves the diagnosis. It’s not like the difference between flipping a coin and drawing a straight flush in 5 card stud.
5. Even for those doctors and nurses (etc..) who remain employed, dealing with COVID is a pain, and trust me, as soon as they can get out of enhanced PPE, they will. My daughters, CNAs at a local nursing home, come home sweaty and exhausted because PPE + 75 degrees is like a sauna.
Again, this is not a conspiracy where doctors are going to be falsifying diagnoses to “match the models”. Let’s repent of this slander.
Aspiring to be a stick in the mud.
Andrew R. wrote:
It’s ironic that those who deride “the models” are working from a mental picture which they believe to approximate reality—i.e., they’re working from a model. It’s often a singularly uninformed model, based on a few poorly understood data points—and much more handwaving than mathematics—but it’s a model.
For the record, I have a BS in Mathematics with 2 classes in modeling and 3 in statistics, and 3 graduate classes with “modeling” in the name. If I don’t know it, I ask my wife with a PhD in statistics who does modeling for a living… Am I an epidemiologist? No. Do I know modeling? Yes. I said what I said because there is a need for pure modeling. Unfortunately, when you are dealing with things like pandemics and climate change, politics and money get involved fast.
I agree that politics and money are factors to watch out for; my statement was more about those who dismiss conclusions because those conclusions are “based on models.”
Incidentally, I have a Ph.D. in mathematics and am (Lord willing) 3 months away from completing a second Ph.D. in astronomy—so I can say I that I too know modeling :-)
What we’ve learned here is that models are never wrong, they’re just less right…
[Barry L.]What we’ve learned here is that models are never wrong, they’re just less right…
Actually, you could come up with any number of models that would be dead wrong for an epidemic. Not a “Megamind” thing at all. For epidemiology, the classic model (which you probably learned in high school algebra or pre-calc) is a geometric series—OK, you modify it when you get a high prevalence, but that’s the basic form. So if you’re trying to model the progress of an epidemic of an infectious disease, any model that can’t easily describe both exponential growth and contraction is by definition wrong.
Aspiring to be a stick in the mud.
The model is only wrong if it doesn’t have exponential growth and then contraction. Spoken like this is just math.
No, its life. Its money. Its jobs. Its families. Its businesses. Its churches. Someone put together a model that showed this disease killing 2.2 million Americans. Why? To get the reaction they wanted. THEY KNEW 2.2 million Americans were not going to die from this… Whether it has exponential growth and contraction is not the point! This was pure manipulation.
That is what makes it wrong.
[Mark_Smith]The model is only wrong if it doesn’t have exponential growth and then contraction. Spoken like this is just math.
No, its life. Its money. Its jobs. Its families. Its businesses. Its churches. Someone put together a model that showed this disease killing 2.2 million Americans. Why? To get the reaction they wanted. THEY KNEW 2.2 million Americans were not going to die from this… Whether it has exponential growth and contraction is not the point! This was pure manipulation.
That is what makes it wrong.
I agree the model seems to be wrong, but I know nothing about statistics or modeling so my opinion doesn’t count for much. One question however: Isn’t it possible that this was not manipulation but was an early model based on the information available at the time? Would the person who put together this model come to the same conclusions with the information available now?
[Mark_Smith]The model is only wrong if it doesn’t have exponential growth and then contraction. Spoken like this is just math.
No, its life. Its money. Its jobs. Its families. Its businesses. Its churches. Someone put together a model that showed this disease killing 2.2 million Americans. Why? To get the reaction they wanted. THEY KNEW 2.2 million Americans were not going to die from this… Whether it has exponential growth and contraction is not the point! This was pure manipulation.
That is what makes it wrong.
Mark, you keep making claims about motivations. Where is your evidence that “someone put together… to get the reaction…”? It’s uncharitable in the extreme to attribute sinister motives to people… even more so without evidence.
Second, this is a fact, not an opinion: there is no single model responsible for the decisions that have lead leaders to take distancing and stay at home decisions in their jurisdictions. I’ve linked to at least three different models in this thread. This one article alone consults 5.
Another fact: models are almost never created by an individual, and even when they are, they rely on years of work by other poeple.
Another fact: most of these models have been around for years and have proved effective in comabatting other disease outbreaks. The IHME model, for example… do you know how long this model has been use? Do you think they dreamed it up in Feburary of 2020?
All I’m saying folks is let’s be factual about this. If you want to believe the use of models in this effort is all a sinister plot to wreck the economy, fine, back it with facts… and sound reasoning. Passion is no substitute for thinking straight.
Views expressed are always my own and not my employer's, my church's, my family's, my neighbors', or my pets'. The house plants have authorized me to speak for them, however, and they always agree with me.
[Mark_Smith]The model is only wrong if it doesn’t have exponential growth and then contraction. Spoken like this is just math.
No, its life. Its money. Its jobs. Its families. Its businesses. Its churches. Someone put together a model that showed this disease killing 2.2 million Americans. Why? To get the reaction they wanted. THEY KNEW 2.2 million Americans were not going to die from this… Whether it has exponential growth and contraction is not the point! This was pure manipulation.
That is what makes it wrong.
Yes, it’s life, jobs, and all that, but you cannot calculate that without a representative model of the disease. When people (like you) do foolish things like comparing the death toll from an epidemic in progress to other epidemics that have already run their course, then they fail that most basic test.
And regarding the initial numbers, it is absolutely true that if human behavior had not changed, we would have (and this was the claim of that model) seen that kind of horrendous numbers. When ill-advised choir practices and youth events get a few dozen people infected, let’s imagine what happens when a few people get off their cruise and go to some spring training games. Let’s imagine what happens when they attend NBA, NHL, and NCAA games. The likely fallout, say, from Twins opening day could overwhelm Minnesota’s entire supply of ICU beds.
I’m open to ways of doing these things better, to be sure, to get a bit more time for people to cope. Sort out which social distancing measures really help vs. those that are just there for the ride? You bet. Perhaps develop a system of more isolated hospitals for dealing with epidemics, instead of housing them at places like St. Mary’s in Rochester? Absolutely. Shout down the guys doing the modeling because we don’t like the implications?
No. Please, no.
Aspiring to be a stick in the mud.
Discussion