By Susan Hodges

Carolyn Keefe has pointed out the importance of getting your information from authoritative sources. In addition, we also need to develop the ability to assess the validity and meaning of studies and information even when they are published in respected journals.

In the Summer 2002 Citizens for Midwifery News we mentioned publicity about a study using Washington State birth statistics that purported to show that babies born at home are more likely to die than if they are born in the hospital. At the time we could only guess about flaws in the methodology.

The study was published in the August 2002 issue of the Journal of Obstetrics & Gynecology: "Outcomes of Planned Home Births in Washington State: 1989-1996" by Pang, Heffelfinger, Huang, Benedetti & Weiss. You can read a press release by the American College of Obstetricians and Gynecologists (ACOG) on their website: Home Births Double Risk of Newborn Death. For a copy of the original article, click on "August Issue of Obstetrics & Gynecology" in the text; those words link to a PDF file of the paper.

Now CfM and several other organizations and individuals have critiqued the studies (see end of article for links). This study provides a good example of the necessity for reading original papers, and not just accepting what is said in news coverage. It is also necessary to read any "scientific" paper critically, because medical and scientific journals do sometimes publish papers of dubious quality.

When I finally got my hands on a copy of the Pang study, I read it with pencil in hand to note all my questions. I kept in mind what I know about home birth, and especially aspects of home birth about which the researchers (all of whom are medical doctors) were likely to be ignorant and which might affect the data and conclusions. I looked for statements that were based on assumptions, rather than data, and I questioned the assumptions. More difficult, but also important, is looking for what is NOT included in the study. It is more important to ask the questions than to answer them; asking questions will help you think about the study and what it actually shows and doesn't show.

Here are some of the kinds of questions one can ask when reading this type of paper:

• Did the authors refer to and discuss all or most of the published articles on the same or related topic?
  
• What is the stated objective of the study? Does the study or experiment actually answer this question?
  
• Read the materials and methods section carefully. What is the source of the data and is it of good quality? (Has it been verified? How inaccurate can it be?) Are the data categories adequately defined so you can tell exactly what is included and what is not? How have the authors manipulated the data? Have they made assumptions? Have they justified their assumptions and shown that they are reasonable to make? Do you think their assumptions are reasonable and valid for this study? Why or why not?
  
• When data from two or more groups are compared, have the authors demonstrated that the groups, and the data from each, are essentially the same except for the aspects being compared?
  
• Do the results (what was found out) make sense? Are the numbers large enough so that inaccuracies in the data collection will not affect the statistical significance of the results?
  
• The discussion (or conclusions) should include the authors' interpretations of their results. What do the results mean? What can and cannot be concluded? What are the strengths and shortcomings of the methodology? What problems occurred and could those problems affect the results? How do the results compare with the results of other related or similar studies? How are similarities or differences in results explained?

This study on Washington State home births has many flaws and problems, many of which become obvious when read critically (i.e., questioning everything). Without going into great detail, here is an example. The title reads: "Outcomes of Planned Home Births in Washington State: 1989-1996," but the authors write in the Materials and Methods section that "Because Washington State birth certificates do not identify which home births were planned, we defined planned home births." This sentence alerts us to the fact that the authors were using inadequate data, and we had better look very carefully at the assumptions they used to decide which home births were "planned." Here is what they tell us they included:

• "singleton newborns" - okay, since this helps to make the home and hospital groups equivalent.
  
• "of at least 34 weeks' gestation - not okay since babies born "at home" at 34-37 weeks are likely to be unplanned, precipitous births, at higher risk due to prematurity. Experienced home birth midwives would not plan to attend at home at less than 37 weeks so these should not be included (although later in the paper they claim that omitting births prior to 37 weeks did not affect the relative rates of neonatal mortality).
  

• "who were delivered at home - maybe OK, but we do not know how this is designated on the birth certificates, and whether or not it includes unplanned out-of-hospital births (in the car, on the way, etc.).
  
• "and who had a midwife, nurse or physician listed as either the birth attendant or certifier on the birth certificate (if an attendant is not listed on the birth certificate, then the person listed as the certifier attended the delivery)" - we know that those babies born "on the way" would then go to the hospital where someone would sign the birth certificate - NOT a planned home birth.

We don't even have to go any further in the study to be certain that at least some unplanned, unattended, even premature births will have been incorrectly included in the cohort of "planned home births." Any neonatal deaths, postpartum bleeding, need for resuscitation for more than 30 minutes (all outcomes included in the study) associated with these high risk births would be wrongly attributed to the "planned home birth." Even the authors acknowledge later in the paper that this is a problem that could skew the results to make "planned home birth" appear more dangerous.

The authors chose to look primarily at neonatal deaths, even though the rate is very low for healthy normal pregnant women regardless of where they give birth. The study reports neonatal mortality rates of 1.5/1000 (hospital) vs. 3.5/1000 (home). The study offers no basis for knowing the magnitude of misclassifications of births in the "planned home birth" category; it is possible that eliminating wrongly classified births would diminish the difference to the point of statistical insignificance.

What about information and data that are not included but should be? For example, there is no mention made of unpreventable neonatal deaths, i.e., when the cause of death was unrelated to the intended or actual site of birth (congenital anomalies incompatible with life, for example). In addition, we cannot know from this paper how many of the neonatal deaths occurred in the unknown number of unplanned/unattended "home births" that were wrongly included as "planned home births." Addressing another outcome, the authors claim that women "intending a home delivery were more likely to have prolonged labor." However, we know that there can be substantial differences between home and hospital regarding when the beginning of labor is counted, and that typical hospital protocols call for interventions to speed or augment labor, so one might anticipate that hospital births would be less likely to have "prolonged labor," yet this is not discussed. In fact, medical interventions that impact on outcomes for both mother and baby, such as episiotomy, epidural and other drugs, induction, augmentation of labor, forceps, vacuum extraction and c-section, are not even mentioned, let alone discussed in relation to this study.

Finally, even if the findings of the study were "true," not even the authors claim anything more than an observation of an association in this set of data shaped by their assumptions. In other words, this study in no way was designed to show, and does not show, a causal relationship. The results cannot legitimately be used to predict even the risk of neonatal death as a result of planning a home birth. One should also note that while the original study, with all its flaws, at least has a neutral title, the ACOG press release overstates the study's findings with an inflammatory and misleading headline, and fails to even mention the shortcomings and limitations of the study.

These are just a few of the flaws that can be found by simply reading the study with a critical eye. However, while the Pang study provides a good exercise for critical reading, you do not have to figure it all out for yourself! CfM has prepared two fliers, one with the key criticisms of the study and the other quoting the authors' own words, in the study, regarding its problems and limitations. In addition excellent and thorough critiques can be found on-line (see below).

If this study comes up in your newspaper, or among legislators, use these resources to show up the flaws of the study. The CfM fliers in particular are designed with press and state officials in mind.

===================================

On-Line critiques of the Washington State planned home birth study by Pang et al:

Citizens for Midwifery fact sheets cover the main points of criticism.

Henci Goer, author of The Thinking Woman's Guide to a Better Birth, has written a critique "Homebirth: Is it really a safe option?" that is posted on ParentsPlace.com (where she is the Birth Guru). Click on "Birth Guru" under Ask The Expert, scroll down to "articles by subject."

MAWS (Midwives Association of Washington State) has written a response. Their statement includes a good list of references.

Faith Gibson has also written a detailed commentary and critique.

===================================

For an excellent and more thorough look at the topic of home birth safety than the Pang et al. study, using much higher quality and more detailed data, more births, and logistic regression analyses to show interactions of risk factors, see Peter Schlenzka's dissertation The Safety of Alternative Approaches to Childbirth - located toward the bottom of the page.

Reprinted from Citizens for Midwifery News, Fall 2002. Permission to reprint with attribution.