1. Archive

Flaws plagued huge health study

When Bernadine Healy announced plans for the Women's Health Initiative in 1991, the then-director of the National Institutes of Health promised it would be one of the most definitive studies of women's health ever undertaken.

Fifteen years and $725-million later, the project has upended the thinking on important health issues many experts thought were settled. The NIH has announced findings that menopause hormones are risky and don't protect the heart, that low-fat diets don't fight breast cancer or heart disease and that taking calcium and vitamin D doesn't protect bones or prevent colorectal cancer.

The conclusions of this high-profile study have drawn enormous publicity and had wide impact. The hormone result, for instance, so swayed the thinking of women and doctors that use of menopausal hormones has plunged by half.

But a close look at the Women's Health Initiative raises questions about its central conclusions. Design problems in all of the trials mean the results don't really answer the questions they were supposed to address. And a flawed communications effort led to widespread misinterpretation of results by the news media and public.

Even critics of the project agree it generated a wealth of valuable data, and this week the NIH is asking academic researchers around the country for proposals on how best to continue mining the information. Of particular interest are hundreds of freezers in Rockwell, Md., filled with tens of thousands of blood, urine and other samples taken from the 161,000 women studied.

Officials at the NIH reject criticism that they mishandled the study, saying people are upset because it upset accepted notions. "The strength of the reaction has been commensurate with the strength of the dogma it overturned," says Jacques Rossouw, the Initiative's project officer for the NIH.




Many doctors believe the public health messages that have come out of the Initiative in the past four years are simply a wrong interpretation of the data.

The nutrition study didn't reveal the full effect of a low-fat diet, they say, in part because most of the women didn't stick to one. The bid to measure the effect of taking calcium and vitamin D suffered because a majority of test subjects, including those given a placebo, were allowed to take calcium supplements on the side. And the hormone study was heavily weighted to older women long past menopause, giving the findings little meaning for the typical user of hormone drugs.

In many cases, the Initiative data paint a very different picture from the widespread public perceptions of the study. The data strongly suggest that women who stick to an eating plan that cuts dietary fat and who increase calcium intake probably do boost their health. They also suggest that the benefits of menopause hormones probably outweigh the risks for the vast majority of women who use them - women in their late 40s and early 50s who take them to treat hot flashes and other menopause symptoms.

Even some of the most staunch supporters of the Initiative are disappointed that both the findings and the limitations of the trials haven't been better communicated. "Unfortunately, science never works in sound bites," says Marcia Stefanick, a Stanford University School of Medicine professor who heads the Initiative steering committee.

Elizabeth G. Nabel, who has just become director of the National Heart, Lung and Blood Institute, which oversees the Initiative, says, "I'm not convinced anything really went terribly wrong here. I think this was a complex study with findings that did not turn out as many people hoped or expected, but when you drill down to the details the findings are very consistent with current health guidelines."

The Women's Health Initiative story began in 1991, when Healy took over NIH as its first female chief. A critic of the level of funding for research on women's health, she announced plans for a huge research project: three major clinical trials looking at hormones taken for menopause, at low-fat diets and at calcium and vitamin D supplements, as well as an "observational" study on tens of thousands more women.

In the diet study, one problem was that women recruited had lower-fat diets to begin with than the investigators had planned for. The design assumed they would be getting about 38 percent of calories from fat. It was actually about 35 percent. It may seem like a small matter, but in the tricky world of statistics, it had ramifications.

The study was meant primarily to test whether cutting fat from a woman's diet helps protect her from breast cancer. Women in one group were told about basic dietary guidelines but left to eat whatever they wanted. The others got intensive nutrition counseling in an effort to cut fat calories to 20 percent of the total eaten. Researchers knew many women would have a tough time sticking to that, and many did. By the sixth year of the study, only 14 percent were meeting the goal.




The Initiative's ambitious scope complicated matters. If its three clinical trials - of hormones, diet and calcium - were done alone, they would require enrolling 112,400 test subjects. Instead, the NIH recruited many women to take part in more than one trial at a time. There was so much overlap that only 68,132 women were involved in the clinical trials. More than half took part in at least two, and more than 5,000 were in all three.

Among problems this posed was simple burnout. Clinical-trial participation entails lots of office visits, lab tests and paperwork. The diet trial was especially time-consuming, requiring women to keep track of what they ate and to attend 18 lengthy nutrition counseling sessions the first year and quarterly sessions after that. Height, weight and other data were collected every six months.

The hassles of being in multiple trials may have contributed to compliance problems that plagued all three and hurt the reliability of their results. By the end of the two hormone studies, nearly half of women had stopped taking their pills. In the fat study, 70 percent weren't able to stick to the difficult eating plan. In the trial of calcium and vitamin D, 41 percent didn't take the pills consistently.

The trial of calcium and vitamin D sought to test the theory that regular use boosts bone density and lowers risk of fractures and colorectal cancer. However, the Initiative let women in the placebo group - those not given supplements - take calcium and vitamin supplements on their own. At the end of the study, 69 percent of trial subjects were taking calcium on the side. The result: Even among those in the placebo group, calcium intake averaged about 1,100 milligrams a day from a combination of supplements and diet.

Normally, such a study would discourage participants from taking calcium on their own. But all 36,282 women in the calcium study were also taking part in other Initiative trials. Investigators say they let subjects take supplements out of concern that some might drop out of one or all of the trials if told they couldn't take calcium. They also say it would have been unethical to tell women they couldn't take a supplement that the government already recommended. (Its guideline is about 1,200 mg a day.)




The goal of the hormone trials was to study the risks and benefits of menopause hormones, especially whether they helped protect women's hearts. A major worry was how to keep the study "blinded" so that women (and researchers) didn't know whether participants were getting hormones or a placebo. The problem was that women with symptoms such as hot flashes and night sweats would quickly realize they were in the placebo group and might drop out. Yet researchers needed to keep the trial going for years to get enough data on cancer and heart disease to be meaningful.

The Initiative's solution was to recruit older women, long past menopause. They would be less likely to get hot flashes, and they would be more likely to face the heart attacks and other problems the study hoped to measure.

The striking result was that the biggest study ever done of menopause hormones didn't study many women who were suffering menopausal symptoms.

Researchers had expected a positive finding: The strong thinking at the time was that hormones could help menopausal or postmenopausal women of any age to ward off heart disease. So recruiting predominantly older women didn't seem like a design problem at the time.

Age breakdowns of the data in months after the study's 2002 release show that younger women had a far different response to the hormones than older women. Because the study had so few women who still were young enough to have menopausal symptoms, no reliable conclusions about the risks and benefits of menopause hormones for those typical users can be drawn.

The hormone study was in two parts. The first looked at the standard combination of estrogen and progestin. The second, involving women who had had hysterectomies, studied estrogen only.

The NIH halted the first part in 2002, about four years earlier than planned, saying women given the combined hormones showed an increased risk of both heart attack and breast cancer.

It has become apparent since then that those risks probably apply only to older women many years past menopause. Subsequent analyses have shown that heart risk, in particular, is probably much lower or nonexistent for younger, menopausal women.

The agency didn't factor into the decision some of the most striking data emerging from the study. The research had begun to suggest that use of estrogen alone might actually lower breast-cancer risk, provide some protection against heart disease and lessen the risk of hip fractures. No definitive conclusions could be drawn because the data suggesting such benefits hadn't reached statistical significance. Had the study been allowed to continue, statisticians and researchers say, much of these data likely would have crossed the statistical threshold.


The National Institutes of Health established the Women's Health Initiative in 1991 to address the most common causes of death, disability and impaired quality of life in postmenopausal women. The agency addressed cardiovascular disease, cancer and osteoporosis. It was a 15-year, multimillion dollar endeavor, and one of the largest U.S. prevention studies of its kind.

Source: NIH


Problems with the three-part Women's Health Initiative study tend to undermine messages the public received:


Message: Supplements don't protect bones or cut risk of colorectal cancer.

Problem: Those in placebo group also took supplements in many cases.


Message: Doesn't cut risk of breast cancer.


+ Few met the fat goal.

+ A 22 percent drop in risk for women who cut fat the most got little emphasis.


Message: No benefit, possible increased cancer and heart risk.

Problem: Most in study were too old for this to apply to menopausal women