A recent study in the British Medical Journal Open that looked at the effect of abortion laws on maternal mortality in Mexico is egregiously flawed and biased. Written by ten anti-choice authors, it poses an unacceptable risk to public health because it could be used to advocate the criminalization of necessary health care for women.
The study purports to show that Mexican states with more restrictive abortion laws have lower maternal mortality rates than states with more permissive laws. Although the authors refrain from hypothesizing a causal link between criminalized abortion and better maternal health outcomes, that preposterous implication comes through nonetheless. It’s even clearer in the authors’ press release about the study.
BMJ Open is an open access journal that offers easier and quicker publication of studies, but its website also states (emphasis added): “Our aim is to provide a home for all properly conducted medical research to be fully reported, after a rigorous and transparent peer review process.”
So what happened? Why did BMJ Open accept this study without subjecting it to greater scrutiny? Some anti-abortion language even slipped through—on page three of the study, the authors characterize the Mexican states’ constitutional amendments protecting the “unborn” from conception as “progressive changes.” The study was peer-reviewed by two medical doctors, but neither appears to have expertise in abortion research or Mexico. One of the reviewers refuses to prescribe birth control to his patients, while the other specializes in safe driving education and injuries from agricultural accidents.
Vote for Rewire!
Rewire is competing for a CREDO grant this month and we need your vote. A few clicks is all it takes for you to help support evidence-based journalism on health, rights, and justice. Vote now to help us speak truth to power, as a matter of fact.
Credibility of Authors in Question
Four of the ten co-authors of the BMJ Open study were named as “False Witnesses” in an investigative series carried out last year by Rewire. The investigation found that those four researchers “pushed false information designed to mislead the public, lawmakers, and the courts about abortion” in their previous research or public statements. Those co-authors are:
- Elard Koch (lead author of the BMJ Open study)
- John Thorp (final author, which usually means the principal investigator or main writer)
- Monique Chireau (second author)
- Byron Calhoun (sixth author)
Five of the six other authors have published previously with lead author Koch on abortion-related research. The remaining author, Joseph Stanford, signed the Dublin Declaration that denies the need for legal abortion even to save a woman’s life. In fact, all of the other co-authors signed this anti-choice declaration as well, with the sole exception of Fernando Pliego.
Lead author Koch and co-authors Chireau and Calhoun are members of the group We Care, a group of anti-abortion researchers and doctors that formed around 2011 to publish their own research in mainstream venues, in an apparent effort to put a gloss of scientific respectability on their anti-abortion stance. Indeed, the current BMJ Open study suffers from repeated citations of their own past writings on abortion (over a dozen different citations from Koch and various co-authors), as if their work is equally legitimate to mainstream research. The effect is to create a false picture of scientific confusion and conflicting data in the abortion field.
The methodology of previous Koch-led studies related to abortion (in Chile, Colombia, and Mexico) has been shown to be seriously flawed and biased, rendering their conclusions invalid. My blog has a compendium of rebuttals of Koch et al.’s previous work from both professional and lay sources.
For example, in 2012, Ipas-Mexico published an analysis of maternal and abortion-related mortality in Mexico from 1998 to 2008, showing that one in 13 maternal deaths were from abortion. Koch and some co-authors then published a rebuttal to this, and Ipas responded with a statement that referred to the Guttmacher Institute’s previous rebuttals to Koch et al.’s work. Guttmacher had explained and defended widely accepted scientific methodologies for estimating numbers of illegal abortions and resulting maternal mortality rates—which Koch et al. had grossly underestimated.
Fatal Flaw #1: Data Dredging to Find an Association
The BMJ Open study concludes that “maternal and abortion-related mortality ratios were lower in states with less permissive abortion legislation compared with states with more permissive legislation.” However, this is based on arbitrarily dividing Mexican states into two categories using a variable with little significance for maternal mortality: those that ban abortion for reasons of “serious genetic or congenital conditions” in the fetus, and those that don’t. It’s worth quoting the authors in full to expose the nature of their error:
In exploratory analyses, segregating states by the number of exemptions provided in criminal codes did not result in detectable differences in sensitivity analyses, with the exception of abortion allowed by genetic or congenital fetal malformations. The remaining seven exemptions were distributed differentially in almost every state or very few states, thus offering no discrimination potential. Therefore, to differentiate between states with more or less permissive abortion legislation in subsequent statistical analyses, states allowing pregnancy termination due to serious genetic or congenital conditions were considered more permissive (14 states) while the remaining states were considered less permissive (18 states).
In other words, the authors subjected each legal exemption to a “sensitivity analysis”—which checks all the data to look for patterns among variables—until they found a variable that happened to show a detectable difference in maternal mortality. They then presented this as the primary finding. This error is called “data dredging” because chance associations between just about any two things are easy to find if you crunch enough numbers. To make their random association sound more meaningful, the authors manipulated the definition of “states with less permissive legislation” into something it didn’t actually mean at all.
Twelve Mexican states actually have a more liberal exemption allowing abortion in cases of serious risk to the woman’s health, yet nine of those states ended up in the study’s category of “less permissive.” This demonstrates the arbitrariness of the chosen exemption factor of fetal anomaly, and indicates that the study could well have come to the opposite conclusion if the health exemption had been chosen as the dividing criterion instead. (All Mexican states allow abortion in cases of rape. Otherwise, abortion is mostly or completely illegal in all states except Yucatán, where it is allowed for economic or social reasons, and the Federal District of Mexico City, which allows abortion on request up to 12 weeks.)
The selected exemption for fetal anomaly cannot possibly by itself show any trends or differences in abortion mortality rates between states. That’s because abortions due to fetal abnormality are always a tiny minority of abortions in any country. In Britain for example, only 1 percent of abortions are carried out for reasons of fetal anomaly, and the numbers are similarly tiny for other countries. Further, almost all abortions for fetal abnormality occur later in pregnancy because the anomaly cannot usually be detected until then.
In settings like Mexico, the numbers of abortions for fetal anomaly will likely be far smaller than 1 percent regardless of legality, because of stigma and other obstacles. Indeed, here’s a study showing that most Mexican geneticists advise against abortion when the fetus has a genetic or chromosomal disorder. And there’s no reason to assume that the average Mexican woman would even know that abortion might be legally available for reasons of fetal abnormality, let alone that she would have the means, resources, or courage to pursue that option. In other words, abortions for reasons of fetal abnormality must be very rare throughout Mexico, and cannot possibly serve as a proxy for tracking trends in maternal mortality due to abortion.
This fatal flaw renders the study meaningless and the conclusion invalid, because it relies solely on a rarely occurring variable that would not have any noticeable statistical effect on maternal mortality.
Fatal Flaw #2: Mistaking Legal Abortion for Accessible Abortion
Similarly, the study assumes that because about half of Mexican states tightened their already strict laws against abortion after 2007—by passing a constitutional amendment protecting “the unborn” from conception—this would have a measurable effect on death from unsafe abortion. There is absolutely no basis for this assumption. Criminal laws restricting most abortions were already in effect everywhere outside Mexico City, so women would not be driven to unsafe abortion in any greater discernible numbers.
This helps expose the second major flaw in the study: the authors’ assumption that abortion law accurately predicts abortion practice. In reality, few Mexican women actually obtain abortions under the legal exemptions due to fear and stigma, lack of resources or knowledge, and refusals by anti-abortion doctors.
One study (in Spanish) by the Mexico City-based Group on Reproductive Choice (GIRE) showed that between 2007 and 2012, only 39 women in Mexico actually got a legal abortion under the country-wide rape exemption, out of a total of 120 who had applied for one. Why would so few apply? First, most states lack sufficient administrative mechanisms for seeking out a legal abortion under any of the exemptions, which means there’s simply no way to even apply for an abortion. Second, abortion is highly stigmatized in Mexico, and it takes courage to apply for one—or seek medical attention after an illegal abortion. At least 679 women in Mexico were reported or sentenced for having an illegal abortion between 2009 and 2011. Mexico is one of at least seven countries in the world that imprisons women for having illegal abortions. From 2007 to 2012, 127 women were put on trial for abortion in Mexico, and in one particularly conservative state, Guanajuato, dozens of women have been prosecuted for abortion since 2000, with some of them receiving sentences of up to 30 years in prison.
Oddly, Koch et al. never mention such shocking facts, nor do they mention the deeply rooted stigma and shame surrounding abortion in Mexico, the judgmental attitudes of many health-care workers, or indeed any of the social, economic, or logistical difficulties that may inhibit women from even attempting to exercise their legal right to abortion in Mexico. Instead, the study’s methodology and conclusion depend on the unspoken assumption that legal exemptions for abortion mean that all or most of those exempted abortions are actually taking place as needed. Nothing could be further from the truth.
Failing to Account for Underreporting and the Impact of Safer Medical Abortion
Koch et al. fail to acknowledge that “do-it-yourself” medical abortions have increased substantially over the last decade or more, with pills to end pregnancy now widely available in Mexico and most other Latin American countries. In fact, the words mifepristone and misoprostol never appear once in the BMJ Open study, which is a serious oversight. Clandestine use of the drug misoprostol is generally accepted as being much safer than traditional and more dangerous methods (for example, significantly reducing the rate of infection), even when women misuse it or misinterpret its effects because they don’t have instructions on how to use it or what to expect.
In one of Koch’s 2012 rebuttals to the Guttmacher Institute, he asserted that “no study currently exists to date that seriously supports a decline in maternal mortality associated with the use of abortifacient drugs such as misoprostol in Chile.” But he ignored several studies from countries such as Brazil and Mexico that showed significant declines in the severity and number of abortion-related complications and sometimes mortality over the same periods in which misoprostol use has grown.
In the BMJ Open study, Koch and his co-authors follow the same pattern as in a previous Koch-led study on Chile: They underestimate the number of abortions and associated maternal mortality by relying only on official statistical sources, while failing to consider that large numbers of illegal abortions are not accounted for in these sources, and that related complications and deaths may often be misclassified. In a criminalized and stigmatized environment, many women will not admit to having an abortion, and many health professionals will not officially report complications or deaths as caused by abortion, either through ignorance of the real cause, or out of compassion for women and their families.
Koch et al. claim there is no reason for health-care professionals in Mexico to “misreport deaths from a suspected illegal abortion” due to the existence of separate reporting codes for various types of abortions, including for an unknown cause. This overlooks the fact that issues with miscoding have become more common with misoprostol-related complications. For example, it can be challenging for doctors to distinguish medical abortion from miscarriage or other obstetrical complications. Further, Koch et al.’s analysis ignores the effects of fear and abortion stigma on how abortion occurs in illegal settings and whether complications or deaths resulting from them are reported as such.
Using Reduced Maternal Mortality to Mask Abortion Deaths
It’s already well established—practically self-evident—that maternal mortality can be significantly reduced by educating women, upgrading health systems, and improving access to contraception, skilled birth attendants, clean water, sanitation, and so on. Yet, this study and previous Koch-led studies seem to treat such factors like their own new discovery that obviates any need to reform abortion laws.
Unsafe abortion is just one of many factors that affect maternal mortality rates, though it’s among the top five causes. An estimated 13 percent of maternal mortality globally is due to unsafe abortion. It is simply not possible to try to take into account a lot of contributing factors to maternal mortality and conclude that restrictive abortion laws have little or no effect, because the other factors can easily swamp the effect of unsafe abortion on maternal mortality rates. Is it possible that the anti-abortion authors of the BMJ Open study are using such factors as a smokescreen to cover up the effect of unsafe abortion on maternal mortality?
Mexico still has a relatively high maternal mortality rate compared to other countries—about 45 per 100,000 live births, compared to 28 for the United States, 13 for Canada, and four for Sweden. In Latin America, where abortion is mostly illegal, it’s 22 for Chile and 69 for both Brazil and Argentina (2013 data). It’s likely that the declines Mexico has been seeing in maternal mortality would be even steeper if abortion was safe, legal, and accessible, and the same goes for Chile.
Estimating the incidence of illegal, unsafe abortion as well as the resulting deaths and complications is of course a challenging task. Such abortions are unreported and usually never come to the attention of authorities, so vital statistics can only provide a fragment of the evidence-based picture. A variety of methods must be used to carefully piece together a picture that is as reliable as possible. These include, for example, surveys of women, surveys of specific health-care facilities, and interviews with knowledgeable health-care workers.
Such methodologies are embodied in the Abortion Incidence Complications Method (AICM), which was developed about 20 years ago. The AICM has been widely used in studies appearing in peer-reviewed journals, and is recognized by experts around the globe, including the World Health Organization. Despite this, Koch has simply tossed out the AICM on the basis that it uses “imaginary numbers.” Not only is this dismissal disingenuous and unwarranted, it amounts to a gratuitous slur against the hundreds of reputable scientists and researchers who spend large amounts of time carefully gathering, comparing, and adjusting abortion-related data under challenging circumstances.
Real World Absent From Study
The BMJ Open study has an important focus on maternal mortality, but unfortunately that focus tends to disguise certain facts that never see the light of day in the study:
- More than a million (1,026,000) abortions take place in Mexico each year, the large majority of them illegal.
- About 159,000 women were treated at public hospitals for abortion complications in 2009.
- An estimated 36 percent of all women who have illegal abortions develop complications that need medical treatment.
- One-quarter of those do not seek treatment, putting them at risk of lasting negative health consequences.
The question that Koch et al. need to answer is this: Even if the study did demonstrate that restrictive abortion laws are associated with lower maternal mortality, does that make it acceptable to let a million desperate Mexican women, year after year, suffer the distress and trauma associated with risking their lives, health, and freedom to obtain an illegal abortion?
Koch et al.’s studies, including the current one in BMJ Open, are promoted widely on the Internet by anti-abortion groups and individuals. Because the studies appear professional and are published in reputable journals, there is a real danger that they can be used to influence policy decisions of governments. For example, they may play a role in decisions to decrease or cut funding for reproductive health programs in developing countries—such as what occurred in Canada in 2010—or to further restrict abortion, despite current laws that still kill 47,000 women a year and injure over eight million.
By rendering those women invisible, such studies become dangerous weapons that threaten to slow down the global decrease in maternal mortality and continue allowing women to suffer and die unnecessarily. The BMJ Open study is the latest contribution to this ideological battle disguised as science, one that poses a grave public health risk to women.
Author’s note: I would like to thank the Guttmacher Institute for its past work, cited in this article, exposing the serious methodological flaws in Elard Koch’s work and debunking his false claims.