Why Most Published Research Findings Are False

There is increasing concern that most current published research findings are false.

The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field.

In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance.

Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias. In this essay, I discuss the implications of these problems for the conduct and interpretation of research.

Published research findings are sometimes refuted by subsequent evidence, with ensuing confusion and disappointment.

There is increasing concern that in modern research, false findings may be the majority or even the vast majority of published research claims. However, this should not be surprising. It can be proven that most claimed research findings are false. Here I will examine the key factors that influence this problem and some corollaries thereof.

Source: http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124

By the same author:

Overall, not only are most research findings false, but, furthermore, most of the true findings are not useful.

Medical interventions should and can result in huge human benefit. It makes no sense to perform clinical research without ensuring clinical utility. Reform and improvement are overdue.



Why most clinical research is not useful

It makes no sense to perform clinical research that has no relevance to patient care, so why do we do it, and how can we stop? John Ioannidis ponders the problem and offers some suggestions.

Practicing doctors and other health care professionals will be familiar with how little of what they find in medical journals is useful.

The term ‘clinical research’ is meant to cover all types of investigation that address questions on the treatment, prevention, diagnosis/screening, or prognosis of disease or enhancement and maintenance of health.

Experimental intervention studies (clinical trials) are the major design intended to answer such questions, but observational studies may also offer relevant evidence.

‘Useful clinical research’ means that it can lead to a favorable change in decision making (when changes in benefits, harms, cost, and any other impact are considered) either by itself or when integrated with other studies and evidence in systematic reviews, meta-analyses, decision analyses, and guidelines.

There are many millions of papers of clinical research – approximately 1 million papers from clinical trials have been published to date, along with tens of thousands of systematic reviews – but most of them are not useful.

In order to be useful, clinical research should be true, but this is not sufficient.

Research inferences should be applicable to real-life circumstances. When the context of clinical research studies deviates from typical real-life circumstances, the question critical readers should ask is, to what extent do these differences invalidate the main conclusions of the study?

A common misconception is that a trial population should be fully representative of the general population of all patients (for treatment) or the entire community (for prevention) to be generalizable.

Randomized trials depend on consent; thus, no trial is a perfect random sample of the general population. However, treatment effects may be similar in nonparticipants, and capturing real-life circumstances is possible, regardless of the representativeness of the study sample, by utilizing pragmatic study designs.

Pragmatism has long been advocated in clinical research, but it is rare. Only nine industry-funded pragmatic comparative drug effectiveness trials were published between 1996 and 2010 according to a systematic review of the literature, while thousands of efficacy trials have been published that explore optimization of testing circumstances.

Studying treatment effects under idealized clinical trial conditions is attractive, but questions then remain over the generalizability of the findings to real-life circumstances.

Observational studies (performed in the thousands) are often precariously interpreted as able to answer questions about causal treatment effects. The use of routinely collected data is typically touted as being more representative of real life, but this is often not true. Most of the widely used observational studies deal with peculiar populations (e.g. nurses, physicians, or workers) and/or peculiar circumstances (e.g. patients managed in specialized health care systems or covered by specific insurance or fitting criteria for inclusion in a registry).

Eventually, observational studies often substantially overestimate treatment effects.

Patient centeredness

Useful research is patient centered. It is done to benefit patients or to preserve health and enhance wellness, not for the needs of physicians, investigators, or sponsors. Useful clinical research should be aligned with patient priorities, the utilities patients assign to different problems and outcomes, and how acceptable they find interventions over the period for which they are indicated. Value for money

Good value for money is an important consideration, especially in an era of limited resources.

Source: http://cancerworld.net/spotlight-on/why-most-clinical-research-is-not-useful/



Medical Misinformation Mess

Some scholars recently coined the expression “Medical Misinformation Mess” to describe current clinical medicine as a realm in which it is very difficult to evaluate both reliability and practical meaning of research studies.

…. most actors – patients, families, and also medical professionals – don’t even realize how hard is the challenge they are facing.

It’s a mess in which the different attempts at fine-tuning the use of statistics for biomedical studies ….

Currently, there are nearly approximately 17 million articles in PubMed tagged with “human(s)”, with more than 700 000 articles identified as “clinical trials”, and more than 1,8 million as “reviews” (approximately 160 000 as “systematic reviews”).

Nearly one million articles on humans are added each year writes biostatistician John Ioannidis in an analysis published on the January issue of the European Journal of Clinical Investigation, where he is Editor in Chief.

Popular media also abound with medical stories and advice for patients. Unfortunately, much of this information is unreliable or of uncertain reliability.

Most clinical trials results may be misleading or not useful for patients.

Most guidelines (which many clinicians rely on to guide treatment decisions) do not fully acknowledge the poor quality of the data on which they are based.

Most medical stories in mass media do not meet criteria for accuracy, and many stories exaggerate benefit and minimise harms.

Speaking in general, it is quite difficult to judge the quality of a researcher’s work: an in-depth evaluation requires knowledge of the specific field of research, methodological skills, access to the protocols and the raw data, and a lot of time.

Publication is just the first step in a long validation process that still requires replication …

Cancer research is hard to reproduce.

The strong alarm on the lack of reproducibility in preclinical cancer research went off in 2012, when Nature magazine published an article in which Glenn Begley and Lee Ellis reported the attempts by several researchers in companies like Amgen and Bayer at repeating crucial experiments in many landmark studies, mostly failed.

Read more: http://cancerworld.net/featured/navigating-uncertainty-in-the-era-of-mmm/

Financial Toxicity, A New Name for a Growing Problem

A growing body of literature has described the so called “financial toxicity” of cancer, a common consequence of treatments, that sometimes leads patients to bankruptcy, and affects quality of life and clinical outcome. It’s not just a matter of empty wallets: the lack of money and the loss of income is a cause of distress that interfere with the efficacy of treatments. Financial toxicity has been studied in the US and other countries with a private health care system.

You can read the article: http://cancerworld.net/spotlight-on/the-toxic-effects-of-hidden-costs/

“We don’t travel; we don’t do anything now because it’s a $100,000 illness. And it sucks.… What are you going to do? Caught between a rock and a hard spot.”

A 67-year-old, insured woman with metastatic breast cancer— let’s call her “Janet”— recently described how her life changed due to costs of cancer care. We all hear about the cost of care at every turn …..

….. Oncologists who treat patients daily listen to similar life-altering, cost-related complaints, along with reports of fatigue, nausea, and pain.

However, to date most of us have not considered financial distress in the same vein as chemotherapy-induced toxicity, nor do we have the training to alleviate that distress. In this, the first part of a two-part series, we will describe the patient-level impact of the cost of cancer care, what we call “financial toxicity.”

More: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4523887/






SMC reminds doctors of ethical obligations amid concerns of ‘over-servicing’

The government body that regulates medical practitioners here on Monday (Mar 19) reminded doctors of their “ethical obligation” to charge fair and reasonable fees for services rendered.

  • The government body that regulates medical practitioners here on Monday (Mar 19) reminded doctors of their “ethical obligation” to charge fair and reasonable fees for services rendered.
  • A doctor must always place his patient’s best interests above his personal interests and any business or financial considerations …

For example, a doctor must not subject a patient to unnecessary tests or procedures “simply because (he or she) stands to benefit from the fees”.

“In the same vein, doctors should not conduct tests or provide treatment merely upon a patient’s request unless there is a clinical basis for doing so.

  • Senior Minister of State for Health Chee Hong Tat cited concerns with “over-consumption, over-servicing and over-charging….”.
  • It reiterated that doctors have a “duty to provide competent, compassionate and appropriate care to patients, based on a balance of evidence and accepted good clinical practice”.
  • “This duty also entails not over-charging and over-servicing patients for financial benefit. As members of the medical profession, doctors are held in the highest esteem by the public, and much trust is reposed in them.
  • Doctors “should not need to be reminded how to carry out our work ethically, charge reasonably and make good medical judgements … It is disappointing that doctors have to be reminded to do the right thing.”

Source: https://www.todayonline.com/singapore/smc-reminds-doctors-ethical-obligations-amid-concerns-over-servicing




Eggs Connected to an Increase Risk of Prostate Cancer

In The Health Professionals Follow-Up Study (HPFS), a study observing over fifty thousand male health professionals since 1986 noticed that men who consumed ≥2.5 eggs per week had a 1.8-fold increased risk of developing lethal prostate cancer compared with men who consumed <0.5 eggs per week. However, this study found no association between consumption of eggs and risk of deadly prostate cancer after diagnosis.


If you have had prostate cancer (PC), or are at high risk of the disease, it would be prudent to stop eating more than an occasional egg, especially if you live in North America.




Highly processed foods tied to increased risk of certain cancers

The cancers that have the highest correlation to processed food consumption are breast, prostate, and colorectal cancer. Reuters  March 8, 2018 

Researchers examined data from dietary surveys completed by nearly 105,000 adults who didn’t have cancer. By the time half the participants had been in the study for at least five years, 2,228 cancer cases had been diagnosed, including 739 breast cancers, 281 prostate cancers, and 153 colorectal cancers.

Every 10% increase in the amount of heavily processed foods and drinks people consumed was associated with a 12% higher risk of developing all cancers and an 11% higher risk of developing breast cancer during the study, researchers report in The BMJ. 

“Ultra-processed foods and beverages contain some food additives for which carcinogenic effects are suspected such as titanium dioxide, a white food pigment which can be found in some confectionaries, chewing-gums, and biscuits,” Srour added. “Ultra-processed foods are also often packaged in plastic which might contain contact materials having controversial effects on health, such as bisphenol A.”

Read more: http://www.freemalaysiatoday.com/category/leisure/2018/03/08/highly-processed-foods-tied-to-increased-risk-of-certain-cancers/