Claude Gilois, FIMBS, MBA, DipWSET.

Or why the majority of studies on this subject are of no real value


For the last thirty years, we have been assailed by scientific studies (widely relayed by the general press) on the health benefits and harm of an ever-increasing number of substances. These studies may say something one day and then say the opposite, sometimes in the same month or even in the same week.  Alcohol, which is the most studied subject in scientific and medical research, appears on the Olympic podium with metronomic regularity. How could we have arrived at this absurd situation which creates an anxiety-provoking climate in the population who no longer knows which saints to devote themselves to and who to trust. This grotesque situation led an eminent epidemiologist, Dimitrios Trichopoulos, director of the epidemiology department at the Harvard School of Public Health, to declare that: ‘epidemiologists have become a nuisance to the population’.   On particularly sensitive subjects with moralising connotations such as alcohol, propagandists of all sides appropriate the results of these studies and group themselves into two camps (for or against) for a trench war with more data releases, more or less well interpreted or assimilated, because the studies are often very difficult to understand due to their particularly sophisticated statistical treatment.


In medical and scientific research, it is usual to carry out trials where subjects are chosen at random for the study and for the control group. The Rolls Royce of this type of study is when the study subjects, as well as the control group subjects, are not known, either to the study subjects themselves nor to the observers. This type of study is called a double-blind randomised study.

But when we want to study the effects of alcohol on health, it is not possible to proceed in this way because ethics prohibit subjecting a healthy population to a substance that can be potentially dangerous. It is therefore necessary to proceed with indirect studies (control cases[2], cohorts[3], meta-analyses[4]) which are far less precise than direct studies.

The results of epidemiological studies are generally expressed using an index[5].  An index equal to 1 means the absence of risks. The higher the index, the more it expresses the strength of the relationship between cause and effect (for example, the relationship between alcohol consumption and cancer) and this is where the problem lies because the vast majority of epidemiological studies report indices which are between 1 and 2, far too low to overcome the difficulties inherent in the methodologies of epidemiological studies.

Many epidemiologists concede that their studies are riddled with biases, uncertainties and methodological weaknesses and that they are incapable of detecting weak associations. When Richard Doll highlighted the link between lung cancer and tobacco there was an increase in risk of 3000% so there was no doubt.

“We are constantly pushing the limits of epidemiology when we are not going beyond it” adds Dimitrios Trichopoulos, already cited, and these studies, he adds, “generate false positive or negative conclusions with disconcerting frequency”. “Bias and confounding factors[6] are the Achilles tendon of epidemiology,” says Philip Cole, professor of epidemiology at the University of Alabama. “Even statistical analysis techniques that have been available for 30 years of epidemiological research to calculate the effect of bias and to correct for the effects of confounding factors are not sufficient to compensate for data limitations,” says Norman Breslow, a biostatistician at the University of Washington in Seattle.

Sir Richard Doll (the discoverer of the link between tobacco and lung cancer) of Oxford University who was the co-author of a flawed study on a causal link between a blood pressure medication and breast cancer, suggests that no epidemiological study can be credible if the index is not greater than 3.  Dimitrios Trichopoulos suggests a risk greater than 4. Angell of the ‘New England Journal of Medicine’ declares « We need an index of at least 3 or more for us to agree to publish studies, particularly if the biological mechanism is improbable or if the discovery is new. Robert Temple of the Food and Drug Administration adds « if the index is not above 3 or 4 then forget the results. »

By the admission of the most eminent epidemiologists, the vast majority of studies are worthless because the index for the majority of them is between 1 and 3 and often between 1 and 2. We can therefore legitimately question why these studies are published. Scientists only exist through the quantity of publications they generate and, for around forty years, we have been witnessing a mad rush to publish, perfectly summed up by the well-known expression ‘publish or perish’.   Publishers are just as eager to publish.

Conflicts of interest are increasing because almost all scientific studies are financed, in part or in full, by the pharmaceutical industry. In 2006 the highly respected British newspaper ‘The Guardian’ revealed that Sir Richard Doll cited above, had for years been paid 1,500 US dollars (1200 Euros) per month by the well-known Monsanto firm (now Bayer) at the time for its involvement in polluting chemistry and today for its GMOs. This undoubtedly led him to minimise the environmental risks of cancers, which he estimated at 1%.

But aren’t meta-analyses (the repetition of a set of comparable studies) a way of circumventing the inherent difficulties and limitations of individual studies? Not really. There is an under-representation of negative studies in the scientific and medical literature, with scientists reluctant to submit negative results and scientific publications reluctant to publish them. Furthermore, ‘if the studies have the same methodological architecture and those studies have a bias, it doesn’t matter if they are replicable’ says David Sackett of the University of Oxford and adds ‘a bias is still a bias even if it is multiplied by 20 or 30’.

Little tips for intellectual self-defence to protect yourself from being cheated by epidemiological studies.

[1] Epidemiology is the study of factors influencing the health and disease of populations.

[2] Retrospective studies between two groups, one with disease (cases) and the other healthy (controls).

[3] Comparison between a group of subjects who are not sick but exposed to a risk and an unexposed group. These studies are generally more precise than case control studies but also more expensive.

[4] Resumption of a set of comparable studies and with a global analysis using adapted statistical tools and complex mathematical models.

[5] the RR (Relative Risk) or an equivalent OR index (Odds Ratio).

[6] Confounding factors are hidden variables in the populations studied that may generate a real association but which is not the one that epidemiologists think they have found.

[i] Main contributor: Taubes G.  Epidemiology Faces Its Limits. , 1995. Science. 269(5221):164-169.