Understanding the evidence as a part of clinical reasoning

Understanding the evidence as a part of clinical reasoning

In the concept of teaching of all these skills—or bedside skills, or clinical skills—I would like to talk for a moment about the role of evidence in this process. The question is, we are always using a certain amount of language, and the language is our skill, and the skill is crucial for communication. It struck me when I was reading, after probably 2 years of not being on the internal medicine ward—I work in critical care—I read something like this: HPI: CAP; PMH: RecMJU, IVDU, and so forth.

I was trying to decipher what it means, and it actually means history of present illness: community-acquired pneumonia; past medical history: recreational marijuana use, intravenous drug use; CONS—I didn’t decipher this one, percutaneous nephrolithotripsy, acute kidney injury, ventilator-associated pneumonia, and so forth. I didn’t decipher CIH, so I actually went on Google and asked what it is, and I got certified industrial hygienist, complementary integrative care, and so forth.

So, what I am saying is I am confronted with a part of my daily medical practice and I have no clue what people are talking about. Now, when we want to introduce it… [My phones are acting up, I’m sorry.] When we want to introduce the discussion of evidence at the bedside, I will first use a similar statement of how probably not to do it. So, if we want to tell somebody that they should use a beta-blocker post myocardial infarction because there was an RCT with a concealed and stratified randomization and block size of 8, hazard ratio and 95% confidence generated with the use of a Cox proportional.... You get the point. I mean, you will lose absolutely everybody.

And this was noted by our guru, if I could use this term, which was Professor David Sackett, who years ago told us that after the 7th course of statistics, which would be required if you want to follow the previous page, a normal clinician knows as much statistics as before the first course and the reasons are that mathematical formulas are impossible to remember; because classic statistical techniques are totally isolated from patients’ care; and mastering them, even for a short time, requires sacrifice of family and social time, lowering your self-esteem and total loss of sense of humor. And it ties to your question… [inaudible]

So, what I would like to do at the moment is present the results of our experience from over 30+ cohorts of people whom we were trying to teach during evidence-based medicine (EBM) courses. And I could tell you that I will tell you the anecdotes or I could tell you that I am going to tell you the results of an observational study using a cohort of people who are interested in EBM. And we had both a derivation and a confirmation cohort. Maybe I will better tell you that it’s anecdotal, really.

So, the first thing, there are certain principles, and as you were talking about what the entrustable professional activities (EPAs) for the teaching of clinicians are and what the EPAs of the teaching of surgeons are, there are certain EPAs, which, I would like to claim, are crucial for people to have the knowledge and language of communication about the evidence. Because the first thing—and there are probably 3 or 4 I would like to take over with you—the first of them is that clinical reasoning, including evidence, leads to decisions, but the evidence never makes decisions for us.

I would like to present to you the results of what I learned from one of our colleagues, P. J. Devereaux—at the moment a cardiologist influencing perioperative care in cardiology, in surgery, but from the cardiac point of view—and I would like you to think about it for a moment. Paying attention, paying attention—I will not ask you individually, but please work out maybe with your colleagues, ask your colleagues as you are sitting, if you have a colleague sitting, what they would say.

And the story here was that in patients who had atrial fibrillation, as you know, there is a risk of stroke, and without treatment there will be 12 strokes and 3 gastrointestinal (GI) bleeds per year. Clear? Clear. Now, you go on anticoagulation and you cut it down by two-thirds. So, you will still suffer some stroke, but 4 instead of 12, and you will have 8 fewer strokes, including 4 major strokes. If you treat 100 people, you’ll have 4 fewer main strokes. Everybody clear?

The question now posed to people is, how many bleeds would you accept as a price for decreasing the amount of strokes by 4 major strokes? Think about it. I give you 15 seconds. You can prevent 4 strokes in 100 people. How many bleeds among those 100 people you would accept? Ready? I will not ask you, but I want you to know because I am going to present you the results.

There are 2 columns. One is in red, one is in blue. The blue represents [physicians]. As you could see, if you could see, there are some physicians who would accept only 1 bleed. There are some who would say, “Oh, I am ready to accept 1 to 1.” There are some physicians who would say, “Oh, I am ready to accept 8 bleeds for 4 fewer strokes” and so forth, right? And the red ones are patients. Patients ran out of space. These people were saying, “In order to prevent 4 strokes, I’m willing to accept 23 [bleeds],” and the only reason why they said 23—or 22—is that there are no more, okay?

So, the bottom line, these people—these physicians and these patients—will have completely different preferences when faced with the same evidence. So, the first thing is, the evidence doesn’t make decisions. And anytime you talk to people, it’s worthwhile to use it as a starting point. It may inform, but it will not make it for you, because what it makes is you obviously have to recognize the clinical situation. You have to know the evidence. So, 4 fewer [strokes] and a few more bleeds. And then, you have to explore the patient’s values and preferences. And only that way you could make the decision.

So, the first thing we try to teach while teaching EBM is that the evidence will never make a decision for you. You will have to make it with the patient. It’s not that the evidence is not relevant. It’s an interpretation of the evidence that differs. Then, the question is, are people capable of understanding the evidence and how well we understand that.

And again, I will present to you something that some of you had seen before, as I’ve been using this example for a long time. You’ve probably heard... Over the last 3 days, I heard that the American Radiology Association changed the recommendations for mammography. [To perform] the first mammography not from the age of 50, but from the age of 40 [years]. Have you heard it? Some of you did.

So, the way how this decision is made is on the basis of presentation, which I will go through with you. This was an experiment performed in Britain among people… You know, they have the system where you have the District Health Council, which consists of people who decide how their money is spent. So, those people were asked about buying a new mammography program and they could describe it on a scale from zero—I don’t support it— to 100—I support it 100%.

I will show you how they wanted to approve different mammography programs, but I will show you their enthusiasm. Even though there is not too many people here, I would like to calibrate your enthusiasm. Okay? So, what I will ask you for—and there will be no follow-up questions—and once I show you a certain statement, I will ask you how strongly you support it, the way how you wish: you can whistle, you can clap, you can say, whoo-hoo, or whatever.

So, the first one is, as a recognition of your participation in this Medical Education Forum, you will receive promotion to Associate Dean of Medical Education at your university. Do you like it? Well, that’s reasonable for this amount of people. But this position of Associate Dean is honorary and involves big responsibilities and no salary. Okay. Much less.

So now, I will show you those programs. The first program is Program A, which reduced the rate of deaths from breast cancer by third. So, two-thirds of people still die but one-third will not. Do you like it? Do you? Do you? Are you buying it? Hands up, who wants to buy this program?

Audience: Limited information.

Roman Jaeschke: Limited information.

Their willingness to pay was 79 out of 100. This program will be bought.

The second one, program D, means that 1592 women need to be screened yearly for seven years to prevent one death from breast cancer. Do you like it? Not that much. However, on this one, the willingness to pay was less.

Program C: increase the rate of patients not dying from breast cancer from 99.82% to 99.88%. Do you like it?

Well, probably some of you are noticing that these are the same programs, but depending on how you want to influence the decision, you will present it in a different way. And if you are not roughly skillful in this sort of mathematics, you’re lost. They will tell you, start at 40 years of age; a group of mammographers will tell you that everyone should have mammography. Well, probably they know the best. The reason is that if we ask ourselves how much we understand from this simplest mathematics, my statement is you cannot overestimate the lack of knowledge and lack of understanding of the things we assume everybody knows.

I was thinking, when you were talking, I remember looking at a 4. year resident who was putting the central line. And here is the patient and here is the resident, and I see this kind of movement, with fairly big rage. And then I ask him, “How are doing this?” And we started to talk and he said, “You know, I’m now a 4. year resident. Everybody assumes that I know how to do it and I was never told. I didn’t know.” Mind you, it was still before the time of Google, and maybe YouTube, and so forth. When we start to learn the insertion of a central line from YouTube, we’ll be out of work.

So, I tell you the trick, this is the second part: first, evidence doesn’t make the decision. Two, you cannot overestimate the lack of knowledge. If you tell sensitivity and specificity, I guarantee you that there is confusion. Everybody assumes that everybody knows what sensitivity and specificity are. They don’t, especially if you are dealing with people you are teaching. So, instead of talking about risks, please talk about money. Money usually interests people much more than risks and diseases, especially in medicine. Sorry, I didn’t say that. Please erase this part.

So, the way how to go is, rather than say something like, “this is risk, this is absolute risk, this is negative risk, this is relative risk reduction,” start to talk to them about money and you can figure out the examples. Before lunch, you have 4 coins. After lunch, you have 3 coins. How much money did you spend? Talk to your colleagues. How much did you spend? One. Right. Remember which finger to show.

Then, what is your absolute wealth difference? Absolute. I am saying that, you know, there are only 4 mathematical formulas. Forget about multiplication and addition. Talk about division or subtraction. So, how much you spent would be a proportion or a difference?

Audience: One coin.

Roman Jaeschke: One coin. Okay. And what is your absolute wealth reduction? Okay, so we know that. And now, we say, you had 4, you have 3. What is the proportion of wealth you still have? You have 3, you had 4. How much do you still have. Huh? Three quarters, 75%. Okay, good. What is the proportion of wealth you lost? Twenty-five. The funny thing is, what you still have and what you lost usually adds to 1. So, that’s important to remember. What is your relative—you know, you keep introducing this one—what is your relative wealth now compared to before lunch? Seventy-five percent or three quarters. What is your relative wealth reduction? Okay.

Now, there is another trick that is important to understand. On the higher level, you have this, say 4 and 3, and here you have 40 and 30. So, I will ask you again, what is the proportion of wealth you still have, if you had 40 coins before and now you have 30? Seventy-five percent. You could spend 1 and still have 75%, or you could spend 10 and still have 75%. So, how much money have you spent in this example? One. In this? Ten. Okay. What is your absolute wealth difference? And here? Ten.

And once you go with all this money, even better, if you buy them lunch, you can substitute the word wealth, or money, or dollars, or Polish zloty, for risk. And now, we are talking about the risk difference, relative risk, relative risk reduction. And if they get confused, you immediately reverse to money. Much easier.

I was telling you about language, money, and wealth. I was talking about that. Now, this third thing that I would like to sell you is Matrix. And matrix is something that I am trying to force any clinical or research discussion into making sure that we will address 4 issues: What are we dealing with? Can we trust the evidence? What is the evidence and how will this apply to my patients? I will go only and mostly to the first one because usually when we disagree it’s because we have different questions in mind.

And the question is defined by person, population, in an individual situation it’s the person with individual characteristics; intervention that would be tested, or treatment; what are we comparing it with; and what is the outcome? Each of them could be subject to manipulation or misunderstanding. I’ll show you one of my favorite examples.

This was an example that changed the resuscitation in outpatient sudden cardiac death, where the company was saying, “If you get the medication, they will take you to hospital and if you don’t take this medication, they will take you the other way.” Now, the problem is that you will get more commonly to hospital here, but people don’t necessarily want to go to hospital. People would like to go home. So, the story is, more people go to hospital, that’s true, but in terms of how many of them go home in a reasonable shape down the road, it is absolutely the same.

The last, and a very common one, which I will still try to introduce in these 2 minutes, is the concept of... Before I even introduce the concept, I will try to avoid to name it. So, I had a colleague who was a surgeon, a thoracic surgeon, who wanted to try new laparoscopic resection of the esophagus. It was 15 years ago. It was one of the first operations performed. The patient died. The surgeon was really upset. He made a publication of the case report to make him feel a little better.

The question is, why did this patient die? What do you think? Give me one reason why this patient died. Natural history. So, maybe the cancer was so bloody advanced. Anything else? An incompetent surgeon or incompetent procedure. The story is that this person might have died for different reasons. What was the first patient this guy operated on? Who would be the chosen patient to operate? A first-time operation… I think there are two possibilities. One, extremely ill; and two, extremely healthy. Let’s hope… That’s how I would do it.

So, this patient of this surgeon… five patients were operated, 80% of them are dead, he’s really upset. He is worried, but he publishes it as a case series, so it’s a little better. Why did these patients die? They were extremely ill, they were extremely old. The problem is, and I will skip the next slide, there were a number of reasons why these patients were dying, other than the technique of operation.

And those things that influence the outcome but are not related to [what] we are asking for, now, you could tell them, you know what, in this stupid jargon we call them... How do we call them? I will figure it out. We are calling them confounders or confounding factors. You are old, you are sick. The suction machined broke during your operation, the lights went out. Whatever. This was not related to the operation itself.

So, we are trying to introduce those concepts after you describe the situation rather than saying, “Now we’ll talk about confounders” and everybody switches off. So, this is another thing which I would like to share with you. I will skip here because the next step would be how to get the comparable control group and so forth, and you can go into randomization and so forth.

I would like to remind you that we were talking that evidence doesn’t make decisions; that rather than talking about the risks, talk about money; always remember about the description of population, intervention, comparator, and outcome; and don’t introduce difficult concepts by saying, “Now, we will be talking about difficult concepts.” Try to use your own examples from your own life and then say “And we call this [then call it].”

What I was talking about is the language which we are using and the way of communication, because if we do not communicate closely, we could run into trouble.

[Video:] “Mayday, Mayday. Hello? Can you hear us? can you hear us? Over? We are sinking. We are sinking.

Hello. This is the German coast guard.

We are sinking. We are sinking.

What are you thinking about?”

It’s not only English, it’s also how we communicate with each other in our own languages. Sometimes we don’t understand. And in order to introduce the concept of evidence at the bedside, we have to learn these basic skills. Thank you.