I saw another exciting news story on a mobile health intervention the other day. I honestly don’t remember the company or product, but what stuck with me was the declaration of success based on 10 patients using the product for three months. Success was touted in terms of cost reduction and resource utilization reduction in a before/after analysis. This inspired me to collect some thoughts on some of the challenges around evaluating success in mHealth.
mHealth represents the collision of two interesting worlds — mobile, which changes on what seems to be a daily basis, and health care, which changes infrequently, only after significant deliberation and usually much empirical analysis. In the tech (mobile) world, companies are talking about creating a minimally viable product (MVP), getting it out in the market, assessing adoption through metrics such as downloads and customer feedback, and iterating accordingly. This would seem to make sense in the consumer world where the goal is to sell a game, an information app or productivity app. If people use it and are willing to pay, that proves its utility, right?
There is something to this line of thinking. Empiric market success is in some ways the ultimate success, at least for those who want to make a big difference in how humanity benefits from technology.
But does this work in health care? I’m not so sure. As clinicians, we’re trained to turn our noses up at this sort of measure of success. But maybe we’re the ones who are wrong. Let me use the 10-patients-for-three-months example to illustrate some issues.
- Selection bias. Virtually all pilots and trials of any sort suffer from this to some extent. These days, it seems that patient/consumer engagement is the holy grail and we all must realize that people who show up to enroll in any sort of study are already engaged to an extent. What about the people who are great candidates for an intervention (conventional wisdom says the disengaged are sicker and more costly) but are too unmotivated even to show up to enroll? Does anyone know how to handle this one?
- Regression to the mean. This is a pesky and annoying one — and a favorite of folks trained in public health — but unfortunately it is a real phenomenon. This is the stake in the heart of virtually all before/after studies. If you follow a group of people, particularly sick ones, a certain percentage of them will get better over time no matter what you do. The more sick the starting sample, the more dramatic the effect. This is why some sort of comparison group is so helpful and why before/after studies are weak.
- Small sample size bias. This one can go either way, meaning you can exaggerate an effect or miss one. If you want to run a proper study, find someone who has training in clinical trial design to estimate the size of the effect of your intervention, and thus the size of the sample you need, to show its efficacy. Lots of technical jargon here (power calculations, type I error, type II error, etc.), enough to make your head spin. But bottom line, you can’t really say much about the generalizability of data based on 10 patients.
- Novelty effect. I made that up, and there is probably a more acceptable scientific term for it. But what I’m referring to is, when you take that same group of people that was motivated enough to enroll in a study and apply an intervention to them, the newness will drive adoption for a while. We see this all of the time in our studies at the Center for Connected Health. The novelty always wears off over time. In fact, I’d say the state-of-the-art in understanding the impact of connected health is one of cautious optimism because we haven’t yet done long term studies to show if our interventions have lasting effects over time. There is room for argument here, I guess, but three months is awfully short.
Why is health care tech different than finding the MVP in the rapidly-changing, market-responsive world of mobile tech? One reason may be that we’re dealing with health and sickness which are qualitatively different than sending a friend the latest snapshot from vacation. It is cliché to say it, but lives are at stake. So we’re more careful and more demanding of evidence. Is this holding us up from the changes that need to occur in our broken health care non-system? Possibly.
It is true that a well designed trial with proper sample size is expensive and takes time. Technologies change faster than we can evaluate them.
One thing we’ve done at CCH is design studies that use a large matched data set from our electronic record as a comparator. This speeds things up a bit, eliminating the need to enroll, randomize and follow a control group. Results are acceptable to all but the most extreme purists.
What ideas do you have on this dilemma?
I’ve been toiling in the field of connected health for 20 years now, watching for signs of adoption that will move us into the steep part of the curve. I have to wonder, with announcements from several huge consumer companies recently, if that time is coming.
By now you’ve heard about Apple’s HealthKit announcement (my thoughts on this detailed here), which involved not only Apple, but Mayo Clinic and Epic.
Samsung is not sitting still, having released increasingly sophisticated versions of their S Health app.
Rumor has it that Samsung will also be coming out soon with the next version of their Galaxy Smartphone accompanied by a developers’ toolkit for health apps. Google will be launching Google Fit.
Some of this exuberance (is it rational?) also involves excitement around wearables, and the intersection of mHealth and wearables is an area of particular interest. To wit, Microsoft is rumored to be introducing a smartwatch this fall, amid lots of interest at Google, Apple and Samsung in the role of the smartwatch in mHealth.
OK, you get the picture. Any analyst worth her salt has got to be predicting a break-out year for mHealth. The mHealth market is said to have been $1.95B in 2012, growing to $49B by 2020; the wearables market is predicted to be $12B by 2018, of which 60% will be health tracking. We just coined a new term, the Internet of (Healthy) Things to describe this convergence.
What’s not to like here? Well, I’m not sure, but there are some reasons to be cautious. Better still, there are some things we must get right as we steward this amazing opportunity to harness a game-changing technology (mobile) and apply it to the laudable goal of improving the health of our citizens.
First the words of caution. My friend Nancy used to tell me she hated to be the skunk at the picnic, and those words ring true here. But allow me a couple of comments contrary to all of the enthusiasm. As noted previously, if HealthKit is just another place to store health-related data — moving from a web application to a mobile phone — do we really think consumers are going to jump for joy? Santayana said, “Those who do not learn from history are doomed to repeat it.” Did we learn the lessons from Google’s failed PHR? From Microsoft’s HealthVault? If you build it, they do not necessarily come. For health, it has to be more compelling than that.
Here are some facts that remind us of the challenge:
- Although one out of ten U.S. adults over the age of 18 owns an activity tracker, within six months, one-third stops using it.
- More than 80% of health apps (like Lose It! or MyFitnessPal) that are downloaded are abandoned within two weeks.
- Also of note, Aetna discontinued CarePass last week, seemingly because their members weren’t enthralled by it.
Most app development is based on the adage, “Give the people what they want.” Snapchat, Instagram, Tinder, etc., are all designed to meet a basic human need in a very simple way. The challenge in health care is that, though we know what patients/consumers need to do to improve their health, most of them don’t want to hear about it. That makes building ‘sticky’ health apps and devices much tougher than a messaging or photo sharing app.
Today, most health care app development is still confusing education with inspiration. They are not equivalent! I’ve told the story before about how we check our smartphones 150 times per day. But in a blood pressure monitoring study at the Center for Connected Health, we had difficulty getting patients to push one button once a day to participate in a program their doctor enthusiastically recommended for them.
What, then, do we do to take full advantage of the opportunity that Samsung, Apple, Google and others are providing us? The answer, simple and yet elusive, is: Focus on Engagement
Over these 20 years, I’ve seen technologies come and go, trends take hold and others fade away. We are learning a great deal about how to empower patients to self-manage their health, and what to do with all of this patient-generated data. The common denominator, the one critical element we must get right, is how to ‘sell’ health to consumers and keep them coming back for more. I say it’s got to be personal, motivational and ubiquitous. What do you say?
I heard the other day that by 2017, 50% of the pharmacy spend in the U.S. will be on specialty pharmacy. It seems this is driven by two phenomena. The first is the growing crop of new molecules that are in the class ‘biologics’ – developed via biotechnology and which are complex to manufacture, require special handling and care coordination. They are incredibly expensive but have given us new hope for such conditions as Hepatitis C and a variety of cancers. These are classified as specialty pharmacy drugs. The second phenomenon is that just about everything else will be generics. Specialty drugs in this one burgeoning expense class seem to be taking over the pharmaceutical industry, and bucking the trend in health care — to succeed by being more efficient. This brings to mind two opportunities for connected health. One is surrounding these expensive therapeutics with connected health applications in order to improve outcomes and reduce costs. The second is that connected health interventions, because of their demonstrated improvements in adherence, can improve the care experience, patient satisfaction and quality of life, and themselves prove to be therapeutic.
I am not going to speak to the first opportunity, but we are working on a real-life example of this at CCH now. We are under non-disclosure with the research sponsor, but I promise you it will be an exciting result when we can publicly discuss it.
The latter opportunity is intriguing and a bit of a sleeper. Traditionally, the introduction of new technologies into health care has been assumed by knee-jerk reaction to add costs. Yet, we’ve accumulated evidence to the contrary. I have two stories to demonstrate this.
The first example is a clinical research program we have under way with adolescents who have asthma. We’ve created a private Facebook group for them to be part of, and that’s about it really. No fancy bells and whistles. Just old-fashioned social networking. This is a study in progress, but to date we’ve already seen a positive effect, as measured by an instrument called the Asthma Control Test (ACT). Typically, the success rate of teenagers filling out this survey is 18%. Although the results are still preliminary, it appears that just putting kids in a Facebook group increased their participation to 80%. More importantly, the improvement in the score on the ACT measuring how well these teens are controlling their asthma also appears to be significantly improved, compared to the use of a new inhaler, it seems that Facebook can be more therapeutic than a drug. I’m being provocative here, but you get the idea.
The second example is in type II diabetes, using connected health to improve activity. We randomized patients with type II diabetes into two groups, one received an activity tracker and nothing more versus a second group that received a tracker plus were sent automated motivational messages every day.
The messages were algorithm-driven; they were not sent by a person. The algorithm took into account several variables, including self-reported information on how motivated the individual was to increase activity, data from the activity tracker, weather data, and some electronic records data. This intervention was conducted over six months. Interim data suggests that patients receiving the automated messages had a significant drop in HbA1c, more robust than the effects of Metformin, one of those generic drugs referenced above. Once again, we see the potential of connected health to be more therapeutic than a drug. This result is even more impressive when you take into account the fact that the messages were machine generated.
These data drive home the point that engagement is powerful and that engaged patients do better. In both cases, those patients who were engaged, measured by either participation in the Facebook group or frequency of opening messages, did even better than the intervention groups as a whole.
Here are two examples, then, where connected health competes with chemical therapeutics in terms of efficacy.
It suggests a future where connected health programs are widespread, either as adjuncts to or substitutes for chemical therapeutics. And, we haven’t yet discussed how connected health strategies can be integrated into clinical trials, or deliver value-added programs to build brand loyalty and patient engagement.
Of course, these days connected health programs are more costly than the chemical therapeutics (the generics anyway), but that cost will plummet over time.
How does this future look to you?
Apple has been making headlines again, which is not unusual for a company that sets trends and moves markets. A week or so ago they announced that iOS 8 would have a built-in app to collect all of your health info in one place. Then there was the announcement of the partnership with Mayo Clinic and Epic Systems. This generated even more headlines. I saw all kinds of opinion pieces published, ranging from predictions that this will be the ‘game changer’ that mHealth has been waiting for, to those who said, “Not so fast. What is really novel here?” I let the dust settle, processed all of this, and came up with the following reflections.
It’s impossible to give an educated opinion because all we’ve seen so far are some hints at what the software will do. We have no idea what the hardware play will be and we don’t yet know all of the planned software capabilities. However, there are some things I believe we can count on.
Apple has an amazing track record of creating superbly designed, intuitive software and beautiful, flawlessly integrated hardware. By contrast, virtually all software created for health care users (providers, patients and administrators) is poorly designed. If Apple offers some breathtaking software and very sexy hardware to help us stay healthier, it could make a difference, or at least point the way for others as the iPod and iPhone did.
Also, hats off to Epic and Mayo Clinic for showing us the way on the integration of patient-generated data into the electronic health record. If the platform gets traction with Mayo clinicians and/or if they can show patient-generated data being used to improve health outcomes or lower cost, this could be a very big deal.
I can’t help but be optimistic. But I also can’t resist playing armchair quarterback and offering some advice as the effort goes from screen shots to reality. With so much opportunity to be transformative, I hope they take up that mantle seriously and don’t repeat the mistakes of recent connected health history. Here is my wish list for Apple as they launch HealthKit.
- Please don’t think that a health data repository on a mobile device will be transformative. Google discontinued its health data repository and Microsoft’s has gotten very little use. We (society, health care experts, providers) have not given consumers/patients enough of a reason to make the effort to store all of their health data on one platform. People don’t feel compelled to take ownership of their health data. The only compelling use case is the traveler who gets ill and that just isn’t enough. Yes, HealthKit will make the collection of health data mobile and most likely passively captured. But overcoming these consumer barriers will not be enough to assure widespread adoption, I fear.
- Please don’t make it just about seeing your doctor’s notes/lab data, etc., on your mobile device. This would be under-imagining the potential of this type of platform. I’m assuming that with the wave of wearables and Apple’s interest in health tracking, we will see heavy integration of patient-generated data. But, assuming is dangerous, so this is my explicit plea to Apple to do lots with patient-generated data, both collected via a device and self-reported.
- Please do employ analytics on all of the data streams to feed insights to users in order to help us to improve our health. This seems obvious, but I worry. At the Center for Connected Health, we have accumulated lots of evidence that personalized engagement messaging is what will make something like HealthKit sticky over the long run. I don’t see evidence that Apple has done this.
Apple does OK at best, while companies like Google, Amazon, Netflix and Facebook live and die on their analytics ability. Other than iTunes, can you think of a software application that Apple created that is superior? Apple excels at many things, but any time they’ve been challenged to use analytics to target messaging and personalization, it has not gone so well. Just look at the example of Ping, Apple’s attempt at a social network, or how iTunes Radio stacks up to Pandora. We know there is no comparison to great machine learning as it pertains to keeping individuals engaged. I don’t see this as a core competency at Apple.
For me, it boils down to this: Will easy-to-use, intuitive, engaging software and beautifully designed hardware be enough to bring people into an environment like HealthKit and keep them there? Or will it take a killer app with analytics to drive personalization to keep people engaged (a la Netflix recommendations, Pandora’s predicting songs for you or Google knowing exactly what your searching for after only three keystrokes).
If you are in the former camp, you predict Apple will be remembered for changing the game in connected health. If you are in the latter camp, you may be seeing HealthKit go the way of Ping or Google’s health data repository.
What’s on your wish list for HealthKit?
As part of our mission at the Center for Connected Health, ‘to create and validate connected health solutions that empower patients and providers to transform care,’ we also engage in a number of activities to help move healthcare forward. Our management team is often found at the podium at industry conferences, roundtable discussions, government hearings, grand rounds and other venues sharing our experience and vision; we participate in webinars, tweetchats and media interviews to help inform and motivate people to embrace health technologies. We also host our annual Connected Health Symposium, which gathers over 1,200 industry, clinical, patient and government leaders from around the world to discuss, debate and ultimately integrate healthcare into the day-to-day lives of patients.
Next week, the Center is pleased to be participating in a first-of-its-kind online forum for the health technology ecosystem, the Innovation HealthJam, taking place June 17-19. The Innovation HealthJam is a virtual event that brings together a diverse and knowledgeable group of people from the healthcare and technology fields to brainstorm ideas, improvements and innovation in healthcare. We are hosting a track within the HealthJam, focusing on remote patient monitoring.
The Center is very proud to have gathered a highly-respected and experienced panel of experts to discuss the benefits, opportunities and applications of remote patient monitoring. Remote patient monitoring can deliver timely information to healthcare providers to increase quality of care, and decrease healthcare delivery costs. Our research and clinical programs in heart failure, diabetes and hypertension have proven this time and again.
I’m privileged to be moderating the discussion, with our panel, including Larry Brooks from Boehringer-Ingelheim, Chris Hendriksen from VRI, Vicki Smith from Qualcomm Life, Jasper zu Putlitz from ansacloud and Khinlei Myint-U from our team here at the Center. These speakers bring years of experience and unique perspective to address issues ranging from the applications of RPM for pharmaceutical manufacturers, scalability, technical connectivity and the implementation of remote patient monitoring programs.
HealthJam attendees are encouraged to join in this virtual discussion, share ideas, promote innovation and help us answer some important questions, including: Will patients feel empowered to better self-manage their health or will they perceive home monitoring as a ‘safety net’ instead of a preventative measure? How will healthcare providers be able to use all the transmitted data to diagnose quickly without being overloaded?
Remote patient monitoring can improve chronic disease management and provide important support for those newly discharged from the hospital. It can also allow clinicians to provide just-in-time care for some of our sickest patients, and help keep people healthy at home. Join us next week to share your ideas, successes and challenges. Together, we hope to find more answers, create more opportunities and help more providers and patients improve clinical outcomes and quality of life.
For more information or to register, please visit the Innovation HealthJam website.
Hope to Jam with you next week.
This post first appeared in Health Affairs:
Joseph Kvedar, For Telehealth Patient Safety Insists Upon An Evolution In Policy, Health Affairs Blog, May 29th 2014, http://healthaffairs.org/blog/2014/05/29/for-telehealth-patient-safety-insists-upon-an-evolution-in-policy/
Copyright ©2010 Health Affairs by Project HOPE – The People-to-People Health Foundation, Inc.
Editor’s note: For more on this topic, see the February issue of Health Affairs, which features a series of articles on connected health.
The nation’s ongoing battle to strike a delicate balance between increasing access to quality health care for all Americans and reducing overall health care spending just scored one of its most substantial victories. In late April, after several months of thoughtful and robust collaboration, the Federation of State Medical Boards (FSMB) ratified a new model national policy – the Appropriate Use of Telemedicine in the Practice of Medicine – at its annual meeting in Denver. This marks the first time the medical community has unilaterally acknowledged the impact technology has had on the practice of medicine, and the ability telemedicine — or connected health — has to facilitate and improve the delivery of health care.
Let us first put this in perspective. We all know health care is at a critical juncture. The implementation of the Affordable Care Act means millions of newly eligible Americans will seek access to an already over-burdened health care system. The nation faces a serious shortage of primary care providers, specialty care is becoming more diversified, and access to care in rural areas is an ongoing challenge. All of these issues are on the rise.
Enter technology-enabled care. Real-time video encounters between patients and providers reverse the burden on patients to seek care in a hospital or doctor’s office by bringing health care directly to them, in their home. At the same time, remote monitoring, sensors, mobile health and other technologies are helping to reduce hospital readmissions, and improving adherence to care plans and clinical outcomes, as well as patient satisfaction. Connected health tools also support preventative care efforts for chronic care patients and can empower individuals to make positive lifestyle changes to improve their overall health and wellness.
Momentum for telehealth is accelerating at an undeniable rate. As of March, twenty states and the District of Columbia have passed mandates for coverage of commercially provided telehealth services; 46 states offer some type of Medicaid reimbursement for services provided via telehealth. A study by Deloitte predicts that this year alone, there will be 100 million eVisits globally, potentially saving over $5 billion when compared to the cost of face-to-face doctor visits. This represents a growth of 400 percent in video-based virtual visits from 2012 levels, and the greatest usage is predicted to occur in North America, where there could be up to 75 million visits in 2014. This would represent 25 percent of the addressable market.
Yet, there exists an inconsistent and often archaic patchwork of state laws that have inhibited the deployment of telehealth in both the private and public sectors. As a result, both providers and patients are in a state of limbo, prompting such questions as: Can I, as a provider, deliver care while still being compliant in all 50 states? Can I, as a patient, trust the care I receive via telehealth is safe and secure? These uncertainties have created an unnecessary barrier to realizing the true promise of telehealth.
Model Policy For Telemedicine Technologies
The Federation of State Medical Boards recently took action by forming the State Medical Boards Appropriate Regulation of Telemedicine (SMART) Workgroup. Through a thoughtful nearly year-long process which included a broad cross-section of stakeholders, the SMART Workgroup developed the “Model Policy for the Appropriate Use of Telemedicine Technologies in the Practice of Medicine.” This insightful document provides states with clear definitions and principles they can look to for guidance when developing new policies that govern telehealth.
Among these principles are these key concepts:
- Evaluation and Treatment of Patient. Treatment delivered in an online setting should be held to the same standard of appropriate practice as those in traditional settings.
- Establishing a Treatment Relationship Online. A physician-patient relationship can be established using telemedicine, so long as the standard of care is met.
- Online Prescribing Safeguards. Prescribing in a telehealth encounter should be at the discretion of the physician.
- Ensuring Privacy, Security, Documentation, and Continuity. Telehealth encounters should be HIPAA compliant, include informed consent, the generation of a medical record, and support continuity of care.
I applaud the Federation, the SMART Workgroup, and their guidance. Telehealth is happening; it’s becoming an accepted practice across the United States. This Model Policy allows for regulatory certainty while encouraging future innovation by creating clear definitions and guidelines on how and when telehealth can be most effectively incorporated into quality patient care.
States must now take action and adopt these policies. Failure to do so is not just inaction, it is irresponsible. The use of connected health tools will increase, and with thoughtful, modernized policy, providers and patients can be assured that technology-enabled care will be safe, secure and uphold a standard of quality care consistent with care delivered in person. The Federation has done its work as national leaders, and now it is time for state leaders to do their part to advance health care delivery in every state throughout the country.
Since I gave a keynote at the 2013 Connected Health Symposium called “Making Health Addictive,” I’ve been posting on this topic in order to explain some of the concepts in more detail and to get your feedback. Previous posts include a framing post, and further detail on what I laid out as three strategies to achieve addiction to healthy behaviors: “Make it About Life” and “Make it Personal” and “Reinforce Social Connections.”
In early February, I wrote about tactic one, Employ Subliminal Messaging, and last month on tactic two, Use Unpredictable Rewards. This post is on the third of three tactics, Use the Sentinel Effect. I realize there is a lot of required reading if you are just checking in to this series. If you want to absorb the full concept, it is worth reading all of them. If you’d like the Reader’s Digest version, you can get away with the framing post as background.
Making health addictive is really about harnessing the power of our fascination with mobile devices, particularly smartphones. We check these devices up to 150 times per day. What if we put a personalized, relevant, motivational and unobtrusive message in front of you some of those times? Could we induce permanent behavior change? I am searching for examples of these customized mobile, personalized messages and any resulting behavior change, so if you know of any, please let me know.
The Sentinel Effect is: The tendency for human performance to improve when participants are aware that their behavior is being evaluated; in contrast to the Hawthorne effect, which refers to behavior change as a result of being observed but not evaluated. Both are useful tools in the context of connected health. I’ve emphasized the sentinel effect because in our experience, patients significantly increase their adherence to a variety of healthy behaviors when they know that their physician (or her agent) is watching.
About seven years ago, we began working on platforms to allow patients to upload biometric data. These data act as both a feedback loop for patients themselves to use as a yardstick for health improvement, and also as a mechanism for patients to give their providers access to a richer tapestry of data on which to base care decisions. This combination of feedback loop plus provider oversight worked well for us. In the case of congestive heart failure monitoring, we saw a 50% drop in readmissions; patients with hypertension achieved a significant drop in both systolic and diastolic pressure; and people with diabetes experienced a meaningful drop in HbA1c.
Because the hardest challenge for us to solve was the round-trip data connectivity — from the sensors in the home to the cloud to the EMR and patient portal — we assumed that this was the magic key to the success of these programs. However, when we asked the patients and the nurses caring for them what made these programs work, we heard a unifying theme from both parties. We learned, what mattered most, is that patients worked hard to improve their health because they knew their nurse was watching and didn’t want to disappoint her. Some patients went so far as to say they’d only participate in uploading their self-monitored data if they knew that their doctor and/or nurse were looking at it.
Here are three video clips that illustrate this:
First from one of our patients, George Ruboy, who talks about his personal experience with Connected Cardiac Care, our heart failure tele-monitoring program:
The mirrored reflections of one of our nurses, Karen Federico, are presented in this video clip:
Lastly, from a pharmacist who has used our Blood Pressure Connect program to manage many patients with hypertension:
This most basic of human psychologies is related to strategy number three, Reinforce Social Interactions. Some social interactions are motivating because we want to brag to our friends, because we want our friends to be a support group or because individuals in our social network are holding us accountable.
The Sentinel Effect is a really powerful tool, but is based on a fairly primitive psychology. Essentially, we don’t want our parents to catch us falling off the wagon.
It’s great that we learned about the synergistic relationship between objective, patient-generated data and the Sentinel Effect. If the same programs were based on self-reported data from patients keeping a written diary, for example, we’d have much weaker outcomes. Patients would report those data that make them look healthy and ignore those that do not.
Now, we’re contemplating whether we actually need a nurse or doctor as the sentinel. For these algorithmic conditions, like uncontemplated hypertension, could we employ software to do the monitoring, rather than having a doctor or nurse monitor the patient data. But would patients be as responsive and diligent managing their own health? Early results suggest this is possible. It is the next generation of connected health solutions.
Let me know your thoughts.