Evolution of EBM. Part 3: Which guidelines can we trust?

Roman Jaeschke, MD: You alluded to trustworthy resources and to clinical practice guidelines. Yesterday [see MIRCIM 2018] we had examples of practice guidelines giving somehow conflicting advice including: Should you control the blood pressure to 140, 130, or 120 mm Hg? Should you fix all coronary vessels during some catastrophe or should you focus only on the culprit vessel? What kind of oxygen saturation should you be using? Is it 94% to 96%, 96% to 98%, or only over 90%? Those practice guidelines and recommendations differ. How could we increase the probability that we are listening to trustworthy guidelines?

Gordon Guyatt, MD: I thought you were going to ask me to explain the differences. You did not, but maybe we will come back to this.

What is a trustworthy guideline? A trustworthy guideline, as we have just said [see Evolution of EBM. Part 2. Systematic summaries of evidence as a milestone in the decision-making process], relies on well-done systematic reviews of the best evidence, and we have criteria for a well-done systematic review. It collects a panel. That is not as used to be the case; we used to call the previous approach GOBSAT, which stands for good old boys sitting around the table. Currently, we have those experts, although I will qualify that in a moment, but we also have methodologists who have a detailed understanding of dealing with the evidence and interpreting it properly. We have front-line clinicians to make sure that recommendations are practical for the people in the front lines. And we have patients. So we have a diverse team of panelists. In terms of the experts, we restrict experts to those without serious intellectual or financial conflict of interests.

Then we have a process of summarizing the evidence and moving from evidence to recommendations. There is a way of doing it called GRADE (Grading of Recommendations Assessment, Development, and Evaluation) that is now very strictly systematized with very detailed guidance that has been adopted by over 100 organizations worldwide, including very many prominent ones. That includes a rating of the quality of evidence, the strength of recommendations, and a particular approach to the evidence summaries that makes them transparent and easily understood by the clinician. All those things create the spectrum that means a trustworthy guideline.

Roman Jaeschke: Maybe I could ask you to explain the 2 concepts that you used. One is the quality of evidence and the other one is the strength of recommendations. They keep coming up in the practice guidelines.

Gordon Guyatt: Quality of evidence—there are different words that have been used—are confidence in the estimates, certainty in the estimates, and the trustworthiness of the evidence. That is one thing.

The next thing is how we are going from evidence to recommendations. I will give you an example. We may have high-quality evidence that a treatment reduces strokes by 1 in 100 or 10 in 1000 and we know pretty well that is what it does. It causes 3 serious gastrointestinal bleeds. In a course of a year, a person’s risk goes down by 1 in 100 for stroke but it goes up 3 in 100 for a bleed. And we have high-quality evidence in both. Does that mean we have a strong recommendation? No, it does not, the reason being there is a close trade-off. Some people would say, “Reducing stroke by 1 in 100 is worth the risk of increasing bleeds by 3 in 100.” Someone else will say, “These numbers are too small for me to worry about at all.” Somebody else would say, “I will stay away from that treatment, I am more worried about the bleeding than the stroke.” The bottom line is we have high-quality evidence about both, but it does not mean a strong recommendation because there is a trade-off.

There are some situations, for instance, you come to the hospital with an intracranial bleed. You were using warfarin and we want fresh-frozen plasma to prevent the progression of the bleed. We would all say yes. Somebody tells you, “There is only low-quality evidence supporting this.” However, it is such a catastrophic situation and the downsides are limited, everybody would say, “Yes, please, do whatever is possible, even if there is only low-quality evidence.”

The point is that in general high-quality evidence goes with stronger recommendations, low-quality evidence with weak recommendations, but there are exceptions. High-quality evidence closely balanced will be a weak recommendation. In some situations low-quality evidence can give strong recommendations, as in this catastrophic situation.

Roman Jaeschke: And a strong recommendation is something that we expect patients to choose and physicians to follow.

Gordon Guyatt: Exactly.

Roman Jaeschke: Thank you very much.