Find out: What is a centaur and what does it have to do with healthcare? What are the criteria for a good machine learning project? What is the role of a virtual health center with predictive models? And most importantly: What ukulele song goes with machine learning?
Here are the slides for my talk given at SMILE (Symposium for Machine learning, ImpLementation and Evaluation). The slides are mostly self-explanatory. You can also watch my talk at YouTube. Here is a PDF of the entire deck.
This is a mind-blowing read. What is a hyper-object? It is a somewhat disturbing concept of something bigger than an object, something that transcends our understanding as a human. The concept’s inventor, Morton, defines it as: “phenomena too vast or fundamentally weird for humans to wrap their minds around.”
Consider examples like “all the plastic in the world” or “climate change” or “a black hole” or “massive oil spills.”
Science fiction author Jeff VanderMeer has said “hyperobject” neatly describes the bizarre alien phenomenon he wrote about in Annihilation, his surreal novel turned 2018 movie.
OK, now that’s disturbing, as I read the book and felt chills during a summer evening. That guy can write. This is as close to a jump-scare that I’ve ever had, reading a book. Even had to put it down for awhile to calm down. =shiver=
Now, I’m thinking: the Epic electronic health record’s INBASKET is a hyperobject.
EHR Inbasket as Hyperobject?
Now, we’re talking. Something that is weird, difficult for humans to grasp, and alas, vast. To the uninitiated, the Epic EHR inbasket is a message center where much of our internal communication takes place: incoming phone calls from patients can be sent to inbasket for nurses, docs, assistants to help manage the request; incoming patient portal messages come here; prescription renewals from pharmacies, from patients; consultation reports from specialists, hospital discharge summaries; notifications that “you did not finish writing this note for this patient visit”; test results from blood tests, radiology studies, biopsy reports; nurse-doctor communications; provider-provider communications. Lots of things.
And, for our busy clinicians, some inbaskets have dozens, hundreds, and sometimes THOUSANDS of unread messages that can be weeks, months, YEARS old. Yikes.
Solving the Rubik’s (hyper)cube?
First, improve teamwork, huddles
Ok, but that is a future post. Aside from the idea that we need to improve our internal teamwork and fundamentally redesign how we use our internal tools, there are some simple changes we can start with.
Time to cut our hyperobject down to size. We know that incoming inbasket messages from patients has tripled (see previous post). We know that our healthcare professionals are suffering from burnout (see previous post). We (I) have been guilty of delivering automated messages to our docs that we originally thought were helpful. Maybe it is time for a re-think.
Our plan to re-size our Hyperobject:
Pick a date (in December 2021), a one time PURGE of all messages 6 months or older in our Inbaskets. This is 7 million messages. Seriously. Rationale: If the provider hasn’t handled this by now, either the patient has called or messaged again (a more recent message), already come for a visit, or perhaps even left the practice to go elsewhere; there is NO VALUE in keeping these.
In December, begin a 90-day expiration clock on all new incoming messages. If you haven’t addressed a concern or responded or read a message by 3 months, it will disappear. Yes, there are theoretical risks of deleting reminders to complete a task or respond to a patient. But, here we are in mid November; is it still relevant that a patient called for advice in August, before school started? Also it is theoretically possible that a provider will stop someday and spend a weekend reading and replying to thousands of messages, but this is not likely at all. We are aware of some inbaskets with messages that number in the thousands. Yes, we are not proud of it. Our current setting where we NEVER delete old messages, I consider a personal failure of bad EHR design and configuration on my part. Behold: the man who ruined healthcare. :(. But, we’ll fix it now.
We are discontinuing the delivery of automatic CC (carbon-copy) messages from consulting specialists back to the referring provider and PCP (primary care provider). In 2011, CT Lin and his merry band thought we were doing everyone a favor by CRUSHING the BLACK HOLE (a DIFFERENT HYPEROBJECT!) of the University docs who never remembered to send a consultation letter back to the referring doc. “Hey, we sent you a patient for this clinical question, and WE NEVER HEARD BACK FROM YOU.” We created a technical solution to AUTOMATICALLY send a specialist’s clinic note back to the referring doc AND the primary care doc (if different). In the beginning, this was a great idea! However, this rule now sends several HUNDRED THOUSAND messages a year to our 6000 internal and innumerable community providers. I am personally burying my colleagues. Asking one of my full time internal medicine colleagues, he tells me “I receive about 100 to 150 auto-CC notes per week. Every couple of weeks I take a Saturday and read through 200-300 messages of which about 5 are useful to me. But I can’t NOT read them, what if I miss something?” What an excellent, OCD (obsessive compulsive) physician. But also the way to burn out on patient care. You work 40-60 hours a week and then spend evenings and weekends “catching up” on the blizzard of messages and tasks in the EHR. We need to do Today’s work Today. By the way, specialists can always manually send an important note back to the referring doc or PCP “hey, calling your attention to this” with a single click. And, our specialists ALREADY do this, so I often receive the automated note AND a manually sent note for specific concerns.
Thus, I feel pretty good about stopping the automation. How often do you read that sentence from a CMIO?
Clearly insane CMIO
To be clear, we have “internal” providers who use our Epic EHR and can look up the consultant specialist’s notes the next time they see the patient. These are the folks who will benefit. ON THE OTHER HAND, we have “external” providers in our communities who do NOT have access to our Epic EHR, they use a different EHR or perhaps are still using paper: We plan to continue to eFax or otherwise deliver these notes UNCHANGED. Thus, still addressing the community need for information and stopping the internal clogging of our own pipes.
Whew! That was a lot more long-winded than I intended, but this is a big deal, a big movement, that has already generated a lot of heat, a lot of concern about “why are you moving my cheese?” So far, in our internal provider discussions, we are hearing 90% support and 10% anxiety from our colleagues. We plan on moving forward and creating innovative solutions for those who do not see this as an improvement.
CMIO’s take? Hang on everybody. This Inbasket Hyperobject is getting resized. We have lots more plans for reducing the burden of inbasket messaging, this is just phase 1 of 4 major phases to come. Stay tuned!
This data dilettante (see previous posts: dilettante #1, dilettante #2) has enjoyed armchair theorizing with all of you, my best (online) friends. Today we explore how our super-smart team scrambled our way to improving sepsis care with a predictive algorithm we built.
The old saying goes: the success of any major project in a large organization follows the 80:20 rule. 20% of the work is getting the technology right, and 80% is the socio-political skill of the people doing the work.
We all underappreciate this fact.
It turns out that we spent months building a sepsis alert predictive tool, based on various deterioration metrics, and a deep analysis of years of our EHR data across multiple hospitals. We designed it to alert providers and nurses up to 12 hours BEFORE clinicians would spot deterioration.
We patted ourselves on the back, deployed the predictive score in a flowsheet row, and in the patient lists and monitoring boards, with color coding and filters, and stepped back to revel in our glory.
Turns out that our doctors and nurses were ALREADY FULLY BUSY (even before the pandemic) taking are of critically ill patients. Adding YET ANOTHER alert, even with fancy colors, did NOT result in a major behavior shift to ordering IV fluids, blood cultures, or life-saving antibiotics any quicker.
See the fancy patient-wearable tech on the left (Visi from Sotera, in this case), and one of our hardworking nurses, with ALL of our current technology hanging off her jacket and stethoscope. She should be the visual encyclopedia entry for “alert fatigue.” 😦
Back to the drawing board
As result of our failure, we huddled to think about transforming the way we provided care. It was time to disrupt ourselves. We decided to implement a Virtual Health Center, mimicking what we had seen in a couple places around the country: we deployed 2 critical care physicians and about a half-dozen critical care nurses on rotation, off-site at an innovative, award-winning Virtual Health Center.
This second time around, we created a cockpit of EHR data and predictive alerts to the VHC clinicians, who were dedicated to watching for deterioration across ALL our hospitals, and responding quickly. This does several things:
Takes the load off busy front line clinicians
Creates a calm environment for focused, rapid response
Dramatically improves the signal-to-noise ratio coming from predictive alerts
This way, the VHC nurses view all the alerts, investigate the chart, and contact the bedside nurse when the suspicion is high for sepsis, and start the sepsis bundle immediately.
Soon, by tweaking the ways our teams worked together, we were able to reduce the burden on bedside nurses and physicians and simplify handoffs.
See chart above: Before the VHC, bedside nurses were responsible for detecting sepsis (infrequent, subtle signals during a busy shift with lots of loud alarms for other things), with many ‘grey box’ tasks, as well as ‘magenta box’ delays.
After implementing the VHC, the VHC nurses took over the majority of ‘green box’ tasks, reducing the bedside ‘grey box’ work and completely eliminating ‘magenta box’ delays.
As a result, we have dropped our “time to fluids” by over an hour, and “time to antibiotics” by 20 minutes, which we estimate has saved 77 more lives from sepsis each year.
CMIO’s take? Predictive analytics, data science, machine learning, call it what you like. This is a paradigm shift in thinking that requires disrupting “business as usual” and is hard, but rewarding work. I can’t wait to see what we all can achieve with these new tools.
The TL;DR? 15 seconds should be the length of your educational videos. Wanna know why? and how? read on.
I was a Late adopter of Facebook
I’ve been thinking about the evolution of social media. In early days, I was a late adopter of Facebook, not getting why it was any better than email. Now, I get it: saying something once allows your network to see it, from close friends, to casual acquaintances. Medical residents explained to me that photos and memories were easier to share more broadly. AND, an existing large network made participation more valuable (hey! look at all the people I already know on here!).
Just like in the old days, getting a telephone was INCREASINGLY useful if there were MORE people and stores you could call. The network effect.
That led me over the years to LinkedIn (mostly for work contacts and posting my CV and work products publicly) and Twitter (still figuring it out, but a good way to keep up with news if you curate your network carefully, and also a way to post blog content). Also, Twitter allows you to curate for yourself an international community with similar interests, like #medtwitter.
And, my brilliant younger sister taught me that Twitter could also be good for lecture commentary and discussion (she will give a talk on 2 screens: one with her slides and another with a live pre-filtered Twitter feed: how brave! and give out a custom hashtag, like #postitpearls_lecture, and ask the audience to submit questions this way: wow).
And, some of you know that I’ve dabbled in amateur song-parodies with EHR songs on my youtube channel.
Finally, I’ve figured out how to blog regularly and then use IFTTT to cross-post my content auto-magically to my other platforms (Facebook page, Twitter, LinkedIn) so that I can seem more connected and omni-present than I really am (Thanks for another great tip, Sis).
BUT! TikTok is another thing altogether. My colleague and her daughter suggested that I take my latest Hamilton parody song (that I had gamely posted to YouTube and here I am shamelessly showing it to you again)
#notthrowinawaymyshot and now post it on TikTok, a post-millenial social media platform restricted to 60 second videos. Leaving aside the recent kerfuffle about Chinese ownership and control, this is qualitatively a different animal: getting your thoughts across in 15 seconds (preferred duration, and the time restriction being a result of the music industry’s maximum replay length of a copyrighted song). It has since been extended to 60 second maximum if you have an original soundtrack on your video.
So, I dove in. Unlike my “dozens” of views on my YouTube channel (with which I was satisfied; my broadcast domain is, admittedly to a relatively small physician informatics audience), my TikToks quickly blossomed to nearly 1000 views in 2 days.
Wow! I thought. I am AMAZING on TikTok.
What I did not appreciate is the 15 to 60 second format is much more attuned to the rapid “swipe” of post-millenials, and EVERYONE racks up lots of views. And, ultra-short videos are so easy to consume one after the other. AND, TikTok doesn’t need you to establish your network before your video gets out there; it shows your video to a random selection of viewers, and then those who LIKE it or SUBSCRIBE to you trigger the algorithm to show it to more viewers. So, an easy way to game the system is to use trending (but highly inaccurate) hashtags, like #superbowl, etc. Sadly, this user does not seem to have understood, or be willing to follow, some of these informal rules.
Furthermore, if you read online chatter about TikTok views “500 views total, is pretty sad; what you want is 500 views per hour.” For example, Nathan Evans, of Sea Shanty fame? He went viral at about 250,000 views, and now he’s at 12.9 million. Oh, well. Here’s my paltry Covid Sea Shanty, currently at 62 views (not 62,000) and SIX LIKES.
In contrast, our Informatics team at UCHealth just retired/deleted a 17-minute video I made a 10 years ago for a full “walkthrough” of how to use the Electronic Health Record for our ambulatory clinic physicians. Whew, how out of touch was THIS guy? Here’s a one minute snippet of the kind of video I posted back then, when we were on Allscripts Touchworks. So young, so naive.
Our more recent training videos are more like 1-2 minutes and focused on ONE technique or tool. Now, I’m thinking, maybe we need to shoot for 15-30 seconds. The cool thing about TikToks is that you can trim seconds, speed things up, because those viewers who “get it” can be done watching in 15 seconds, but the video can be paused and also it automatically replays so the viewer can catch subtle details. Hmm, is this a paradigm shift? Should we embed TikTok length education videos into our EHR?
Put Road Signs On the Roadway
As we say internally, shouldn’t we put the Road Signs and Driving Directions (our tips and tricks) on the Roadway (where our users are actually using the EHR) and not in the Garage (our online reference library and training webinars)? Aren’t our users more likely to click on tips WHEN they’re doing work, rather than when “oh, I have some time, let me see what I can go learn.” (which is never)
Austin Chang is my hero
There clearly is an entire evolution of thinking needed to succeed in this TikTok medium. And I don’t have the savvy (yet), the luck, or the persistence to grind out the many tries needed to break through. However, there are medical professionals who have. For example, Austin Chang.
Austin is … well, just go watch him. In 15 seconds, with hilarious music over-dubs, he uses captions and terrible dancing while in scrubs (ok not so terrible), to get his medical facts out there.
I both bemoan the general public’s deterioration of attention span (15 seconds now? Really?) and his ability to fit his tiny education bites (bytes?) into this format. It works. Some of his TikToks are over 2 million views. On MEDICAL TOPICS. Nice. Here’s the NYTimes writing about him.
This reminds me of reading The Shallows, a book about what the Internet is doing to our brains. Are we losing the ability to read a book? I don’t know. I, for one, did not finish reading the book. Ironic.
CMIO’s take: Beat ’em or Join ’em? What are YOU doing about TikTok in your field?
Thank goodness for smart colleagues. Dr. Elizabeth Harry is first author on an important work that ties physician/provider task load to burnout. See link above.
Using the NASA task load index, and the Maslach burnout inventory, she was able to demonstrate a substantial correlation with an increased task load (mental, physical, and temporal demands, and perception of effort) and burnout.
Far from pointing the finger at EHR’s alone, task load generalizes across many industries, with electronic tools such as the EHR being a major negative or positive influence.
I can see a fruitful future line of investigation and collaboration with this measurement tool.
CMIO’s take? How are YOU measuring and tackling provider burnout?
This post is THREE THINGS. A personal origin story, a (brief) book review, and a connection to recent stories on Pfizer and Moderna Covid vaccines. And, when we’re done, it might even tie together!
Image above: Dr. NoFronta Lobe, Mad Scientist. No this is not me in the research lab; this is me, a kindergarten parent at Halloween
My Origin Story (I was a budding molecular biologist in 1985)
I was alone in the brightly-lit sterile-white research lab; having spent 20 hours on a long, multi-day experiment. It was nearly midnight on Saturday in 1985. I was a college junior majoring in molecular biology, with aspirations of a scientific research career. I was studying P4 bacteriophage, a virus that attacks E coli bacteria.
The work sequence, I could now perform by heart: inoculate, incubate, centrifuge, enzyme reaction, pipette (fancy eyedropper tool) into an Eppendorf tube (a tiny plastic tapered tube. From a Q-tip-loaded with a single bacterial colony, I had carefully grown a quart of bacterial culture, then sequentially purified my sample down to 20 drops of a pearlescent white DNA solution.
So: 20 hours for 20 precious drops.
Exhausted and looking forward to heading home, I was on my last steps before overnight refrigeration, so as I held the open Eppendorf in my left hand and my pipette in the right, I randomly thought: “What time is it? Am I going to miss the last Orange Line train going home?”
So, I moved to look at my watch…
And since my watch is on my left wrist, the Eppendorf tube in my left hand did a 180…
And I watched as all the liquid ran out … and onto the floor.
I looked at the upside down Eppendorf, and then down at the floor and the drops of liquid there, uncomprehending.
*How… what… nnnnnNNNNNOOOOOOOOOOO!!!*
My late-night-fogged brain finally registered SHOCK, DENIAL, ANGER, BARGAINING. The lab was deserted, I deemed it safe to express myself:
“F$*&@! S!#%! D&$%!” I said, eloquently.
Desperate, I dropped down and started using the pipette to suck up DIRTY droplets of DNA extract from the floor and replace it into the Eppendorf. After a few minutes I had about 1/3 of the liquid, now brown-tinged, back in the tube. Resigned, I put the tube in the fridge.
NO time to fret, no time to start over. Nothing else to do. I got on my jacket and faced the Boston winter, and jogged for the Orange Line stop.
Once on board that last train, I started to sob. There was no way that soiled sample would be any good. This COMPLETELY SUCKED.
And, I realized, I really did not want to be here. I realized: I could do the scientific work, but, unlike some colleagues who revelled in long hours in pursuit of new knowledge, I was despondent, not very good at this, and missed being around people.
That was the night I decided that bench research was not for me. I had thought my calling was in pure science, but this DNA catastrophe taught me where I didn’t want to be. I needed Humanism AND Science. So, medical school it was. I’ve never looked back.
Molecular Biology after 1985 (CRISPR!)
Thirty-five years later after my profanity-laced change of career, Walter Isaacson chronicles the recent successes of genetic research, including the discovery of CRISPR (Clustered Regularly Interspaced Short Palindromic Repeats) and CAS9 (CRISPR ASsociated protein #9).
Book review rating? 5/5 stars.
In a nutshell: Jennifer Doudna and Emmanuelle Charpentier, and many others raced to understand these strange “repeating sequences” in DNA and mRNA, realizing that they were bacterial defenses against “phage” viruses.
In this work, they discovered that CRISPR-Cas9, an mRNA plus protein complex could identify attacking virus mRNA and SLICE IT UP, deactivating it. They realized that this ancient protein complex could be taught to identify ANY RNA or DNA. Gene editing, invented by bacteria as a survival mechanism a millenia ago, co-opted by humans. Precise genetic scissors.
I enjoy Isaacson’s writing style. Not only does he clearly explain the adrenaline rush of scientific discovery (and the delicate dance between scientific sharing versus the race against other labs to publish and claim credit), but also the technical details of how CRISPR works.
Isaacson writes about Doudna and the response to Covid-19. What is even more astonishing about Dr. Doudna, the bench researcher and lab leader at Berkeley, is that she had the socio-political skills to bring together 40 leading geneticists across the Bay Area to successfully set up a brain trust to develop Covid-19 testing and vaccine development. This team lays much of the groundwork of the accomplishments of this past year.
Drs. Doudna and Charpentier were, deservedly, awarded the 2020 Nobel Prize in Biology “for the development of a method for genome editing.”
A personal note: my brief journey in molecular biology never quite crosses Dr. Doudna’s path, but I recognize the genetic tools mentioned, and studied the work of the luminaries in the field. I feel like a distant cousin to these scientists.
Highly recommended read, to understand the genetic foundation of our modern age.
Molecular Biology: the Covid fight
Here are 2 stories about Covid Vaccines, from the New York Times and WIRED.com, fascinating glimpses into the genomic-industrial complex. As of May 14th 2021, 36% of US adults are vaccinated against Covid-19. It is highly likely that these speedier and more effective mRNA-based detection tests and vaccines will forever be part of our lives. This could shorten development and improve accuracy of future vaccines.
CMIO’s take? Our modern world is built from advances in scientific method, computing and now genome editing. Despitemy early failure in the lab, I feel fortunate, in the field of medical informatics, to be close to all 3.
Why make folks go to training or read a tip sheet if you could guide them just-in-time as they do their work?
Here’s a lovely example of our Physician Informatics Group (Large PIG) evolving as we improve the “intelligence” that our Electronic Health Record (EHR) supplies to our hardworking providers (physicians and advanced practice providers).
In the past
clinicians might have pulled up an app on their smartphone dedicated to the ASCVD: atherosclerotic cardiovascular disease RISK CALCULATOR, punch in some numbers and get a result to type back into the EHR.
Later on, some well known national websites would do the calculation for you. Still requires finding the website and typing in numbers.
Even later, we would put hyperlinks within the EHR to link you automatically, but the typing was still required to get an answer.
we have built a smartphrase (while using the EHR, in any text field, type “.ASCVDRISK”, hit the RETURN key, and Voila, the answer above:
Risk calculated based on what the EHR knows about your patient: age, sex, diabetes, smoking, blood pressure, cholesterol. AND THE ANSWER: 9.1%. FURTHERMORE, disappearing help text guides you to use this information appropriately, and only saves the relevant info to keep in your progress note.
My colleague’s wonderful metaphor for Clinical Decision Support like this?
Put the road signs on the road, not in the garage.
It is already hard enough to use an EHR with patients. Don’t make me go looking for that training document from weeks or months ago, don’t make me think. Make it easy to do the right thing. It is a small celebration every time we can do this right.
CMIO’s take? Thanks to Rich Altman MD for a beautiful new tool in our system. What road signs can YOU take out of the garage and put on the road?
Because we laid all the groundwork and also felt it to be the RIGHT THING FOR PATIENTS, we went ahead and launched our changes in November, and SURPRISE: I still have a job, wasn’t fired. Thanks to our MMOLC sharing community.
A brief reminder
The rule stipulates that patients should be able to receive a copy of
Test Results (laboratory, pathology, radiology) and
Their Provider’s (physicians and advanced practice providers) Progress Notes
upon request. The interpretation was that if a patient had a Patient Portal account in the EHR (electronic health record) of the clinic or health system, this meant immediate release of this information. This is a major change for many health systems who had not yet adopted Open Notes (release of progress notes) or delayed the release of test results by several days.
Since our adoption of Immediate Release
of Progress Notes and Test Results since November of 2020, I have had exactly 2 complaints reach my desk, both from oncologists representing unhappy patients who read their CT imaging study on Friday evening and had difficulty reaching their doc until Monday. This is in the setting of about 1 million clinic visits, 800,000 radiology studies, 30,000 hospital stays, millions of lab and pathology results in these last 4 months.
So, not perfect, but not nearly the “sky is falling” result, as anticipated by many colleagues.
In case you’re still working through this, here are some documents (some updated) that we used. Some are actually named Information SHARING instead of BLOCKING, just a more hopeful attitude. I hope these might help you. Images here, links to full PDFs further down.
Even better, our colleagues at OpenNotes.org are doing some terrific office hours with experts online where you can hear the real-life experience of leaders ALREADY doing this work in various fields, successfully.
CMIO’s take? Are you ready? Do you have documents YOU can share? Lets do this!