A lot of what is published is incorrect … science has taken a turn towards darkness

Dr. Richard Horton (2015), Editor-in-Chief of The Lancet, wrote in 2015

“A lot of what is published is incorrect.” I’m not allowed to say who made this remark because we were asked to observe Chatham House rules.

The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness.

As one participant put it, “poor methods get results”.

The apparent endemicity of bad research behaviour is alarming. In their quest for telling a compelling story, scientists too often sculpt data to fit their preferred theory of the world. Or they retrofit hypotheses to fit their data.

Journal editors deserve their fair share of criticism too. We aid and abet the worst behaviours.

Our acquiescence to the impact factor fuels an unhealthy competition to win a place in a select few journals. Our love of “significance” pollutes the literature with many a statistical fairy-tale. We reject important confirmations.

Journals are not the only miscreants. Universities are in a perpetual struggle for money and talent, endpoints that foster reductive metrics, such as high-impact publication.

National assessment procedures, such as the Research Excellence Framework, incentivise bad practices. And individual scientists, including their most senior leaders, do little to alter a research culture that occasionally veers close to misconduct.

Can bad scientific practices be fixed? Part of the problem is that no-one is incentivised to be right. Instead, scientists are incentivised to be productive and innovative.

Would a Hippocratic Oath for science help? Certainly don’t add more layers of research red-tape.

But as to precisely what to do or how to do it, there were no firm answers. Those who have the power to act seem to think somebody else should act first. And every positive action (eg, funding well-powered replications) has a counterargument (science will become less creative). The good news is that science is beginning to take some of its worst failings very seriously. The bad news is that nobody is ready to take the first step to clean up the system.

Source: http://www.thelancet.com/journals/lancet/article/PIIS0140-6736%2815%2960696-1/fulltext?rss%3Dyes

 

 

Advertisements

Why Most Published Research Findings Are False

There is increasing concern that most current published research findings are false.

The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field.

In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance.

Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias. In this essay, I discuss the implications of these problems for the conduct and interpretation of research.

Published research findings are sometimes refuted by subsequent evidence, with ensuing confusion and disappointment.

There is increasing concern that in modern research, false findings may be the majority or even the vast majority of published research claims. However, this should not be surprising. It can be proven that most claimed research findings are false. Here I will examine the key factors that influence this problem and some corollaries thereof.

Source: http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124

By the same author:

Overall, not only are most research findings false, but, furthermore, most of the true findings are not useful.

Medical interventions should and can result in huge human benefit. It makes no sense to perform clinical research without ensuring clinical utility. Reform and improvement are overdue.

 

Why most clinical research is not useful

It makes no sense to perform clinical research that has no relevance to patient care, so why do we do it, and how can we stop? John Ioannidis ponders the problem and offers some suggestions.

Practicing doctors and other health care professionals will be familiar with how little of what they find in medical journals is useful.

The term ‘clinical research’ is meant to cover all types of investigation that address questions on the treatment, prevention, diagnosis/screening, or prognosis of disease or enhancement and maintenance of health.

Experimental intervention studies (clinical trials) are the major design intended to answer such questions, but observational studies may also offer relevant evidence.

‘Useful clinical research’ means that it can lead to a favorable change in decision making (when changes in benefits, harms, cost, and any other impact are considered) either by itself or when integrated with other studies and evidence in systematic reviews, meta-analyses, decision analyses, and guidelines.

There are many millions of papers of clinical research – approximately 1 million papers from clinical trials have been published to date, along with tens of thousands of systematic reviews – but most of them are not useful.

In order to be useful, clinical research should be true, but this is not sufficient.

Research inferences should be applicable to real-life circumstances. When the context of clinical research studies deviates from typical real-life circumstances, the question critical readers should ask is, to what extent do these differences invalidate the main conclusions of the study?

A common misconception is that a trial population should be fully representative of the general population of all patients (for treatment) or the entire community (for prevention) to be generalizable.

Randomized trials depend on consent; thus, no trial is a perfect random sample of the general population. However, treatment effects may be similar in nonparticipants, and capturing real-life circumstances is possible, regardless of the representativeness of the study sample, by utilizing pragmatic study designs.

Pragmatism has long been advocated in clinical research, but it is rare. Only nine industry-funded pragmatic comparative drug effectiveness trials were published between 1996 and 2010 according to a systematic review of the literature, while thousands of efficacy trials have been published that explore optimization of testing circumstances.

Studying treatment effects under idealized clinical trial conditions is attractive, but questions then remain over the generalizability of the findings to real-life circumstances.

Observational studies (performed in the thousands) are often precariously interpreted as able to answer questions about causal treatment effects. The use of routinely collected data is typically touted as being more representative of real life, but this is often not true. Most of the widely used observational studies deal with peculiar populations (e.g. nurses, physicians, or workers) and/or peculiar circumstances (e.g. patients managed in specialized health care systems or covered by specific insurance or fitting criteria for inclusion in a registry).

Eventually, observational studies often substantially overestimate treatment effects.

Patient centeredness

Useful research is patient centered. It is done to benefit patients or to preserve health and enhance wellness, not for the needs of physicians, investigators, or sponsors. Useful clinical research should be aligned with patient priorities, the utilities patients assign to different problems and outcomes, and how acceptable they find interventions over the period for which they are indicated. Value for money

Good value for money is an important consideration, especially in an era of limited resources.

Source: http://cancerworld.net/spotlight-on/why-most-clinical-research-is-not-useful/

 

 

Medical Misinformation Mess

Some scholars recently coined the expression “Medical Misinformation Mess” to describe current clinical medicine as a realm in which it is very difficult to evaluate both reliability and practical meaning of research studies.

…. most actors – patients, families, and also medical professionals – don’t even realize how hard is the challenge they are facing.

It’s a mess in which the different attempts at fine-tuning the use of statistics for biomedical studies ….

Currently, there are nearly approximately 17 million articles in PubMed tagged with “human(s)”, with more than 700 000 articles identified as “clinical trials”, and more than 1,8 million as “reviews” (approximately 160 000 as “systematic reviews”).

Nearly one million articles on humans are added each year writes biostatistician John Ioannidis in an analysis published on the January issue of the European Journal of Clinical Investigation, where he is Editor in Chief.

Popular media also abound with medical stories and advice for patients. Unfortunately, much of this information is unreliable or of uncertain reliability.

Most clinical trials results may be misleading or not useful for patients.

Most guidelines (which many clinicians rely on to guide treatment decisions) do not fully acknowledge the poor quality of the data on which they are based.

Most medical stories in mass media do not meet criteria for accuracy, and many stories exaggerate benefit and minimise harms.

Speaking in general, it is quite difficult to judge the quality of a researcher’s work: an in-depth evaluation requires knowledge of the specific field of research, methodological skills, access to the protocols and the raw data, and a lot of time.

Publication is just the first step in a long validation process that still requires replication …

Cancer research is hard to reproduce.

The strong alarm on the lack of reproducibility in preclinical cancer research went off in 2012, when Nature magazine published an article in which Glenn Begley and Lee Ellis reported the attempts by several researchers in companies like Amgen and Bayer at repeating crucial experiments in many landmark studies, mostly failed.

Read more: http://cancerworld.net/featured/navigating-uncertainty-in-the-era-of-mmm/

How to Counter the Circus of Pseudoscience

Lisa Pryor, a medical doctor, is the author, most recently, of “A Small Book About Drugs.” Lisa Pryor JAN. 5, 2018

One traditional view of the medical profession is that doctors are commanding and authoritarian, even arrogant. Though some individuals fit that description, in fact, the profession is built on doubt.

Most doctors, especially the good ones, are acutely aware of the limits of their knowledge. I have learned from those much more experienced and qualified than me that humility is something to be cultivated over time, not lost.

Our field is built around trying to prove ourselves wrong. In hospitals we hold morbidity and mortality meetings trying to show where we have failed, what we need to change, how we can do better. Our hospital work is audited to identify where we fell short of our ideals. Through scientific research we try to disprove the effectiveness of treatments. Our failings are exposed from the inside.

The nature of evidence-based health care is that practices change as new evidence emerges … This can be immensely frustrating for patients, even though it is what we must do to provide the best possible treatment.

It is a cognitive bias known in psychology as the Dunning-Kruger Effect. In short, the less you know, the less able you are to recognize how little you know, so the less likely you are to recognize your errors and shortcomings. For the highly skilled, like trained scientists, the opposite is true: The more you know, the more likely you are to see how little you know.

In the face of this circus, we doctors must hold tight to evidence. We must hold tight to our doubt, our knowledge of our fallibility as individuals and as a profession, knowing that humility is a strength, not a weakness.

But we must also as a profession engage in the public conversations about health, including on social media, along with our colleagues in allied health fields. If we do not, the discussion will be dominated by the passionately uninformed, who build trust only to sell false cures. And we must listen to patients, as we are taught to do, showing care and understanding. We must take on the difficult challenge of inspiring and motivating with the truth.

 

Our Medical Systems Are Broken

Our medical systems are broken. Doctors are capable of extraordinary (and expensive) treatments, but they are losing their core focus: actually treating people ~ Dr. Atul Gawande, surgeon and professor at Harvard Medical School.

 

We have now found treatments for nearly all of the tens of thousands of conditions that a human being can have. We can’t cure it all. We can’t guarantee that everybody will live a long and healthy life. But we can make it possible for most.

But what does it take? Well, we’ve now discovered 4,000 medical and surgical procedures. We’ve discovered 6,000 drugs that I’m now licensed to prescribe … And we’ve reached the point where we’ve realized, as doctors, we can’t know it all. We can’t do it all by ourselves.

We’re all specialists now, even the primary care physicians. Everyone just has a piece of the care. But holding onto that structure we built around the daring, independence, self-sufficiency of each of those people has become a disaster. We have trained, hired and rewarded people to be cowboys. But it’s pit crews that we need, pit crews for patients.

There’s evidence all around us: 40 percent of our coronary artery disease patients in our communities receive incomplete or inappropriate care. 60 percent of our asthma, stroke patients receive incomplete or inappropriate care. Two million people come into hospitals and pick up an infection they didn’t have because someone failed to follow the basic practices of hygiene.

There’s another sign … the unmanageable cost of our care. Now we in medicine, I think, are baffled by this question of cost. We want to say, “This is just the way it is. This is just what medicine requires.”

But I think we’re ignoring certain facts that tell us something about what we can do. As we’ve looked at the data about the results that have come as the complexity has increased, we found that the most expensive care is not necessarily the best care. And vice versa, the best care often turns out to be the least expensive — has fewer complications, the people get more efficient at what they do.

But when we look at the positive deviants — the ones who are getting the best results at the lowest costs — we find the ones that look the most like systems are the most successful. That is to say, they found ways to get all of the different pieces, all of the different components, to come together into a whole. Having great components is not enough, and yet we’ve been obsessed in medicine with components. We want the best drugs, the best technologies, the best specialists, but we don’t think too much about how it all comes together. It’s a terrible design strategy actually.

There’s a famous thought experiment that touches exactly on this that said, what if you built a car from the very best car parts? Well it would lead you to put in Porsche brakes, a Ferrari engine, a Volvo body, a BMW chassis. And you put it all together and what do you get? A very expensive pile of junk that does not go anywhere. And that is what medicine can feel like sometimes. It’s not a system.

Now a system, however, when things start to come together, you realize it has certain skills for acting and looking that way. Skill number one is the ability to recognize success and the ability to recognize failure. When you are a specialist, you can’t see the end result very well.

 

Hospital Error Death by Medicine is the third leading cause of death in the U.S.

Death my nedicine

Did you know that avoidable medical errors are the THIRD leading cause of death in America? Most of us already knew that the leading cause of death in America is Heart Disease. This is followed closely by the “Big C” (cancer). But most are shocked to learn the third leading cause of death…. Hospital Error. Death by Medicine has become the third leading cause of death in the U.S.

When you enter a hospital, you’re looking for help and expect to be safe, right? What you may not know is that you are literally putting your trust into a system that is broken.  Tragically thousands of people die every day from the “care” they receive … rather than from the emergency, disease or medical need which brought them to the hospital in the first place.

This information is strategically pushed “under the rug” and not openly discussed. Of course hospitals don’t want the public to know how statistically risky their hospital or healthcare facility may be.

Martin Makary, a professor of surgery at the Johns Hopkins University School of Medicine who led the research, said in an interview that the category includes everything from bad doctors to more systemic issues such as communication breakdowns when patients are handed off from one department to another.

“It boils down to people dying from the care that they receive rather than the disease for which they are seeking care,” Makary said. … His calculation of 251,000 deaths equates to nearly 700 deaths a day — about 9.5 percent of all deaths annually in the United States.

When there is an airplane crash and passengers are killed, there is a thorough investigation and, if need be, new laws are passed to help prevent the same problem from occurring in the future. The medical system is completely different. Hospitals habitually hide their “accidents” and the patient’s families don’t know where to turn for help.

Whenever possible: Stay out of the hospital. Remember… the people can be friendly and kind. It’s the system that is broken.

Paula

Read more:  http://www.paulablack.org/blog/2016/5/9/death-by-medicine-by-paula-black