Where The Walls Listen
Artificial Intelligence and Clinical Scribes
Notes from a Family Meeting is a newsletter where I hope to join the curious conversations that hang about the intersections of health and the human condition. Poems and medical journals alike will join us in our explorations.
For those of you just joining, consider starting here to trace how I’ve been thinking about medicine and technology, a conversation I’ve been returning to time and again.
“Exam room 42.”
The message scrolled across Winston Sánchez’s phone just three seconds before he arrived at the front door of the clinic building. Various people shuffled around the clinic, heads down following directions on their phones to where they were supposed to go. Occasionally Winston passed another clinician wearing blue scrubs, or a computer technician wearing yellow scrubs. He always expected to recognize one but he never did.
Winston’s assigned exam room for the day looked the same as any other. The floor, ceiling, and three walls were a dark, soothing blue. One wall was white and cast a soft light across the room. Three chairs were arranged in a circle next to an examination table. Everything else, ranging from robotic phlebotomy to minor radiologic equipment, hid in the walls.
Winston sat down and faced the white wall. He cleared his throat just a moment before a black box appeared. A bell chimed recurrently and pleasantly.
Winston answered the call. “Good morning, Dr. Paisley.” A woman’s face filled the box on the wall.
“Good morning, Dr. Sánchez! Getting ready for clinic I see? Well, hopefully this won’t take too long. Let’s see.” The wall populated with a several boxes of text, and various graphs and charts.
“Based on this quarter’s metrics, I’m really pleased to see how often you’ve been able to discuss your patients’ hemoglobin A1c. There’s blood pressure, cancer screening. Yup, and vaccines - your rates are on par. It seems like you’re able to cover a lot of ground in your encounters!” Winston spent the majority of his 7-10 minute visits with patients reading data and recommendations supplied to him by Meddi, the clinic’s artificially intelligent medical librarian. The patients would have had access to the same information at home, but Winston was available to provide human-level guidance and support for those who requested it. He also ultimately approved all the AI-recommended plans, though he rarely changed anything about them. If something did go wrong, though, he’d be the one to blame.
Winston was old enough to remember when clinicians literally typed their notes and orders into a computer during or after their encounters with patients. They generally relied on a wealth of minutiae they had memorized, or else they had to look things up manually in various resources online (or even in books!). About fifteen years ago, though, clinicians adopted ambient scribe technology. A few years later, clinics and hospitals dispensed with keyboards entirely. The scribes were a wild success. Clinicians and administrators alike celebrated their efficiency. The use of scribe technology became mandatory.
These quarterly performance reviews, gleaned from monitoring every one of Winston’s encounters, were intended to ensure he adhered to best practices. Everything Winston and his patients spoke in the clinical encounter was documented by the scribe. The encounters were also video recorded for a number of clinical and legal reasons. Everyone in his clinic affectionately called their automated scribe “Tabbi.”
Dr. Paisley’s brow furrowed. “I will say, it looks like you’re spending about an average of three minutes per encounter on non-clinical chatter. Hmm. The weather, patient vacations. Of course, of course we want our patients to feel welcome, but…” A pie chart grew on the wall. “If you reduce that to just one minute, you’ll be able to see seven more patients in a day! Goodness, look at that. Three minutes of non-clinical puts you in the bottom 10th percentile. Friendly, but inefficient!” An item popped up on Winston’s to do list at the bottom right hand corner of the wall: Complete efficiency module #151.
“One more point of feedback. It looks like for patients over the age of 65, when you start to ask about advance care planning, their body language suggests guardedness and frustration. Tabbi notes this happens at least once per day. Hmm. Based on those patients’ comorbidities, I see why you’d want to go off script, but maybe let the experts handle those conversations, eh, Dr. Sánchez?” Winston nodded. “It doesn’t look like those conversations are even impacting ACP outcomes anyway.” Dr. Paisley eyes flitted around as she evaluated the data on her own screen. “For the next two weeks, we’re activating a reminder for you. If Tabbi doesn’t like what she hears, she’ll ring a bell, like this,” An unpleasant, minor note sounded in the room. “Just a reminder, Dr. Sánchez.” Dr. Paisley smiled.
“Well, I think that just about does it. Let’s see here.” A number flashed on the screen and then floated into a piggy bank icon with a dollar sign on it. “Looks like today’s performance review netted you 34 clini-bucks! I hear the canteen’s ice cream is delicious, and only 40 clini-bucks on Fridays. A nice treat at the end the week, don’t you think? Well, Dr. Sánchez, I hope the day goes well. Keep up the good work.”
Dr. Paisley disappeared from the screen. Another chime issued from the door, and Winston’s first patient of the day entered. After greeting him, Winston kept the niceties to a minimum. They sat down and the wall populated with Winston’s first script of the day.
The electronic medical record (EMR) is a true pharmakon: its benefits are matched only by its drawbacks. I remember as a medical student hunting down paper charts, deciphering handwriting, navigating the idiosyncrasies of each attending’s chart organization methods as I rotated through their clinics. But now, with the EMR, we face bloated charts, distrust due to copy-and-paste and templated garbage, and a co-opting of the record for non-clinical purposes.
A fair trade? That’s the wrong question. It fails to appreciate how the EMR is the evolution of the paper record, not its competitor. Nothing changed about the interests parasitic to the clinical encounter, the reign of efficient thinking, or the irascible hope that technology can save us from our squishy subjectivity. The EMR didn’t usher those in; it magnified them.
Despite the amount of time they spend there, clinicians really do not like wading through the EMR. They (including I) hone ways to comb through records and dispatch notes as quickly as possible while trying to provide the best care possible. Some are more or less successful in fending off the EMR’s intrusion into the clinical encounter proper, that sacred space between a patient and clinician. Some have succumbed entirely, faces buried in a screen, typing away as a patient bemoans their symptoms to this clinician they’re unsure is listening.
Enter scribes. I’ve never had the pleasure of working with a scribe, but I’ve heard they’re great. You make an arrangements with someone to have them follow you around and write your notes for you. You care for the patients, the scribe cares for the note. Brilliant!
But scribes are human. They’re intrusive, I imagine, in their own way, if you’re having a sensitive conversation. They might miss something. They need to eat and sleep.
Enter the machine. Apps are blooming like springtime dandelions promising ambient transcription services. Set an app to listen to your conversation and, using artificial intelligence, it will produce a note for you. It’s early days yet, but the promises are so alluring. Clinicians are hopeful it will soothe burnout and shoulder the brunt of the burden imposed by all their documentation. It might reduce time spent documenting (although its debatable whether a minute per note is meaningful; other studies have gotten better results). Most concerns about the drawbacks are technical.
I suspect the industrious creators and refiners will iron out the quirks in the years to come. Eventually this technology may be able to perfectly replicate a conversation between multiple parties, identify them, translate non-native speech, and format it all into a useful note. Iterations on the technology might then offer suggestions for an assessment and plan based on what the software captured. This might revolutionize clinical documentation.
But insofar as this is yet another evolution of a lineage that began with paper records and yielded the EMR, I think there are concerns we continue to overlook. Leo Marx, considering the pairing of innovation with progress, asked these questions:
“Does improved technology mean progress? Yes, it certainly could mean just that. But only if we are willing and able to answer the next question: progress toward what? What is it that we want our new technologies to accomplish? What do we want beyond such immediate, limited goals as achieving efficiencies, decreasing financial costs, and eliminating the troubling human element from our workplaces? In the absence of answers to these questions, technological improvements may very well turn out to be incompatible with genuine, that is to say social, progress.”
In order to consider what kind of progress AI scribes offer, we should reflect on whether we’ve progressed so far. Contrasting our hope for a future with AI scribes with our current use of EMRs could yield some questions worth exploring.
We think by writing.
This isn’t true for all clinicians, but I know for many in the realm of internal medicine and psychiatry the process of writing a note in the chart helps forms one’s clinical thought process.1 Writing in the format of subjective, objective, assessment, and plan (the classic SOAP note) not only reflects thought, but forms it. The note is an argument with evidence supporting a course of action. It’s also a way of telling a story. Early on, medical students and interns struggle to write a useful note because they’re still learning to make these arguments and tell these stories. Writing notes helps them learn.
When trainees and clinicians no longer write, or have their writing substantially curtailed, because the machine does it for them, what will this do to their capacity to develop a cogent argument in favor of a plan, or to tell a story that both honors a patient and provides a diagnosis? We needn’t wait for AI scribes to answer this question. The EMR has already facilitated the creation of notes filled with information no one reads because it’s templated and can be found elsewhere, templated and patently not true (did you really check cranial nerves on every patient?), or it’s no longer true because it’s been copy-and-pasted for the past three months. This doesn’t mean less time is spent in the chart, ironically, but it seems to mean less time free-writing. The time is instead spent sifting through reams of data or manipulating the technological apparatus to get it to produce the type of note you want.
AI scribes would allow clinicians to dispense with writing altogether. An important tool for forming clinical thinking will have gone extinct, or rather, handed over to the machine entirely.
Surveillance.
I’m not sure anyone predicted the EMR would be so quickly and completely co-opted for non-clinical interests, chiefly to be used for billing and coding. After that there are medicolegal uses of documentation. Trailing, frail and almost forgotten, are the actual clinical uses of documentation.
The EMR is already used as a form of surveillance, though it is a blunt instrument as it depends on the clinicians’ honesty and thoroughness in being their own clerk. “If you don’t document it, it didn’t happen.” Understood in a certain way, that’s just not true: the patient really did receive a thoracentesis whether it was documented or not. You really did palpate their abdomen whether it was documented or not. But in another way, in the way that administrators, lawyers, and billers care about, it absolutely did not happen if it wasn’t documented. The note doesn’t just reflect reality. It is reality.
When EMRs were first being broadly deployed in the United States, clinicians were encouraged to adopt them and demonstrate “meaningful use” - e.g., writing electronic orders. There were some incentives that went along with this. Now, as far as I know, there aren’t any more carrots. There are only sticks, ranging from polite reminders to addend or update documentation to accord with coding rules all the way to temporary banishment from the EMR if you’re delinquent on certain forms of documentation.
Dashboards also collect vast arrays of data for population health. A primary care clinician can see at a glance how their own clinic population fares in terms of diabetes management compared to their colleagues’. That seems like good news for their patients. It might identify variations in this clinician’s practice worth addressing. But their administrators have access to the same dashboard and are prepared to offer incentives for good behavior, or worse if the outcomes don’t measure up. The clinicians themselves may have little influence over how the output of surveillance is used or what metrics are used to track their work.
The EMR is a tool for quality assurance but it’s also a tool of surveillance to influence the clinical encounter in ways suitable to the interests of those who may not have patients’ health as their deepest concern. Think how much more effective this surveillance would be if, like in the brief vignette I provided, AI scribes offered the necessary information with near-perfect fidelity? Clinicians would no longer need to type anything (a little sugar to help the medicine go down), and the bureaucracy can finally see the clinical encounter in a language it can understand.
Expansive memory.
There’s a tremendous amount of information in the clinical encounter that goes undocumented because the clinician either doesn’t notice or doesn’t care. I’ve never documented, “I was angry,” or “I was sad,” in a patient chart. Their chart isn’t about me. Likewise, I may sometimes document a patient’s affect, but sometimes I don’t. It’s a judgment call on my part.
An AI scribe need not make any judgment calls about what’s documented. It can document everything. This sounds like a terrifying prospect for reviewing the note later, but that’s a mere technical problem. Notes can be re-imagined. Clinicians could request a summary of the visit and receive a nice, concise SOAP note. They, or someone else, could also request a comprehensive transcript, complete with AI-generated inferences about whether a tremor in someone’s voice signals emotion or pathology. The AI may also make its own inferences about the clinician’s words and behavior.
This isn’t anything new, but an extension of thinking already inherent in documentation practices: the medical bureaucracy must be comprehensive. Jacques Ellul raised this concern: in order to be maximally efficient, technique must pull everything under its purview. This is what happens when we try to care for the “whole person” without actually appreciating the purpose of medicine. Paper charts began to do this, the EMR does it to a greater extent, and an AI scribe will do it even more thoroughly, all while leaving unaddressed deeper questions of what we’re actually doing.
Time crunch.
I’m amazed anyone could believe that once an AI scribe saves a clinician time, administrators will leave the reclaimed time untouched. If a clinician could see 14-16 patients in a day without an AI scribe (already a number too high for primary care and most sub-specialties), surely they could see 20-25 patients with an AI scribe. The encounters become shorter because what is expected isn’t human connection but a technical service rendered ever more efficiently by other tools in the AI suite. That reclaimed, unbillable time shouldn’t be given back to those 14-16 patients because presumably those encounters are already optimized. It should be given to 6-9 other patients who can be seen today instead of next week. This isn’t good for the clinicians but the encounter isn’t about them. They are parts in the machine, and may soon be expunged by AI anyway. Ironically, neither is it good for the patient who finds themselves treated as a bureaucratic client, a machine, or an animal, depending on the flavor of dehumanization their condition warrants.
Asynchronous hopes.
One driver of clinician frustration with the EMR is the growing burden of messages from patients (i.e., secure messages, in-basket messages). The expectation (see surveillance above) is that clinicians stay abreast of all these messages. Sometimes they have support staff to help sift through them, sometimes not. There’s no reason to believe anything can staunch the flow of these messages. They’re supremely convenient for patients, even if they’re overwhelming for merely human clinicians. As clinicians spend their days seeing patients, these messages are usually relegated to the hours after dinner, later at night, or in every free moment during the day.
AI scribes are a step toward realizing what might be totally asynchronous clinical encounters, particularly when paired with wearable health technology. Imagine receiving a package from your primary care clinician with some equipment that you might wear for a day or longer. Perhaps you’ll also prick your finger to collect some blood. All of that is either mailed back or assessed remotely. You speak conversationally to an AI chatbot (who empathizes with you better than your robot-like, burnt-out human clinician). The chatbot guides you through the use of the supplied equipment to collect data for the physical exam (“Place the stethoscope here…”). All of that is collated into not just a note, but an entire asynchronous encounter for a human clinician to later review, complete with suggestions, prompts, and nudges from the AI. If the human clinician has any additional questions (unlikely, given the thoroughness of the chatbot) they can be sent back to the patient to be reviewed at the patient’s convenience.
The idea of a discrete, time-bound encounter would fade away into an ever-present health monitoring service. Eventually the human clinician would also recede into a kind of functionary and scapegoat, a human who would bear responsibility if something went wrong, but really just necessary to approve the plans set forth by AI. The success of this approach to healthcare is somewhat an empirical question to be settled by research, but by the time all the requisite technology is adopted, it will be hard to turn back to the practice, and sheer convenience may outweigh any contrarian qualms.
All of this is already set in motion by the hope ignited by in-basket messages and informal health monitoring technologies (e.g., Apple Health on your watch). Why spend a few hours round trip on visiting a doctor for a discrete visit when you can have seamless, almost continuous access at your convenience? Never mind the clinician tending to these asynchronous visits at all hours of the night, after they’ve spent the day tending to the patients who do choose to show up in clinic. Never mind the totalitarian influence over everyone’s lives of constant health monitoring.
Shaping robots.
Most people who have visited a clinician recently have encountered someone turned away from them, typing away at the computer. Answers to your questions might be curt, followed by non-sequitur questions about a possible symptom, statements about a study result, or an inquiry about a vaccination. Clearly the clinician is following some format, some list of check boxes, provided to them by the EMR. “Hm, pain in your abdomen? Okay. How long?” Click, click, click. “Oh, did you get your flu shot this year? Uh-huh. Let’s see. No blood work today.”
The EMR has done its part in creating this semi-robotic mix of a bureaucrat and technician. Now imagine an encounter where the clinician knows the walls are listening. They’re not typing, but they modify their speech to meet the mark set for them. It might be as wooden as the typist-clinician behind the computer, but it might be even worse. “Alright, Mr. Jones. Based on these results, let’s start some metformin. I am now handing Mr. Jones a metformin information pamphlet. Mr. Jones, would you like me to read it aloud to you? Okay. I acknowledge Mr. Jones declines to have it read aloud, he will read it later. Here’s a diabetes management pamphlet. Yes, I know you got one last year, but we need to provide them every year. Okay. Mr. Jones declines to receive the diabetes management pamphlet.”
What an awkward conversation, as the clinician divides their speech between speaking directly with the patient and speaking to the spirits superintending the encounter. This is not how human conversation works in almost any context, but what happens to the human mind when clinicians spend years, even decades, participating in this performance?
Whatever it is, it was born with the creation of record-keeping within the clinical encounter. My concern isn’t with the tool itself (documentation actually has some good uses) but with how we use the tool, wisely or unwisely, well or poorly. This spirit of efficiency and reductionism reaches new strengths as we innovate on our technologies.
Is this progress?
The concerns I’ve listed here aren’t wildly fantastical. They’re extensions of the rationale that support current documentation practices within the EMR. AI scribes hope to further magnify those technical impulses. They also threaten to distill the EMR’s troubles and failures into something even more potent.
I suspect this is true of all non-surgical disciplines. It’s probably also true of surgical disciplines, insofar as operative notes are manually written and the repeated documentation of what happened in surgery helps to reinforce surgical practice. However, I’m not sure we’re at the point of considering ambient AI for surgical documentation! Though it could be on the horizon somewhere.