“I think knowing what you cannot do is more important than knowing what you can.”
“I think knowing what you cannot do is more important than knowing what you can.”
Mobile Product test
require smartphone to play
Sign-up, S M S
Any guide to design-driven development will tell you that the best way to improve your product is by watching people using it. We call these sessions usability labs or listening labs, where we serve as anthropologists looking to discover what users actually want from our products.
You may ask “Mark, how do I find my target audience?” I recommend the following approach.
1) Put an Ad on craigslist in the etc. section asking exactly for your target market. You may say that “Our target audience doesn’t use craigslist”. You’d be amazed. Most of our responses included an explanation from the respondent on how they were a part of our target demographic (In our case, Medi-cal or Healthy San Francisco members).
2) In said Craigslist ad, ask them to send a text message to a phone number to sign-up. It proves a few things.
3) The low tech iteration of this would be posting your own cell phone # in the ad. I wouldn’t recommend this since we got 20 respondents via SMS almost immediately (Some people follow craigslist ads via RSS).
This is a great time to leverage Twilio! I bought a new phone number from Twilio (a local one to San Francisco) and wrote a small MVC controller action that saved the message to the database, sent an SMS response saying that we would respond soon, then sent me an email that we got a new sign-up. You could exclude any of these, but triage.me already had this code ready to assemble. You can see the code in C# here. It took me 30 minutes to plan this, write this, and ship it.
4) At this point, it seemed like more work to respond via Twilio. I have unlimited messages with my cell phone so I punched out the rest of the sign-up questions on my cell phone. Next time, maybe I would have written more code to ask more questions like: “Could you meet on Friday between 11AM and 3PM?”. It would have saved me some time.
5) I got this week’s users signed up, but users kept signing up. I changed the message to ask the user if they would be interested in future study opportunities and if we could contact them via SMS. I wrote something to collect this input, so now the process is entirely automated. In the future, I could see us sending out some basic survey questions via SMS, offering respondents the ability to be entered in a raffle for a gift certificate or some other reward in kind.
Overall, I’d consider this to be a great success. We have 70 people willing to do surveys that we have cell phone numbers for and that number is growing by the day. SMS signups outweighed e-mail signups 3 to 1, and it cost us $2.
In the two usability labs we’ve done so far, it was great talking to people that use text messaging actively in their lives. A husband with his wife when I asked the survey question “How many text messages do you send daily” burst into laughter when she mentioned that her ballpark guess was 200. One user wanted to keep using the prototype, even if it wasn’t fully connected on the back end yet. People stumbled across a few bugs that were easy to fix once I watched them use the system. You don’t see these things when you’re in the office writing code, thinking how things will work. It’s why I love usability labs, and I hope I gave you a great way to find an audience to participate in them.
“For a successful revolution it is not enough that there is discontent. What is required is a profound and thorough conviction of the justice, necessity and importance of political and social rights.”
Over the last few weeks I’ve been searching for a concise way to convey why I’m passionate about health care.
For a market that operates inefficiently, continues to misalign economic incentives and is plagued by inequality, it’s not hard to see the negative. Conversely, it’s also not difficult to find ways to improve the system.
For someone driven to discover and build new ways to do things more efficiently, this is reason enough to contemplate a career in health care.
It goes deeper than that.
I’m financially conservative and socially liberal. In this world, it’s difficult to balance these two dispositions. Conventional wisdom is that you need to choose either the “conservative” or “liberal” position for any given issue. Rarely are we able to embrace both. From my vantage point, making a difference in our broken health system mandates a balancing of the two. Any solution must come to terms with respecting every life, all while controlling and lowering costs.
One of my favorite sayings in health care comes from those tough old nuns running major hospital systems. Shrewd businesswomen and religious, they espouse the “no money, no mission” philosophy.
Moxe Health is a for-profit company. We aspire to fix some of the largest problems in health care. We also focus on working with underserved populations. We want to help the most vulnerable members of society. It’s not only the right thing to do, but accomplishing our mission lowers the tax burden of our nation’s social health programs. It makes America more competitive globally and maximizes the economic potential of every resident. What’s not to love?
As my former employer used to say, “Do good. Have fun. Make Money”. We carry that philosophy forward here at Moxe.
I recently read “Getting Real” by the 37 signals folks. I’d recommend it for anyone who does software development. Their major mantra:
1) Only build things that need to be done, exactly when they need to be done. Forward-thinking code is code that you have to maintain that might not necessarily be used.
2) Do the bare minimum to get it ready and then ship it.
3) Iterate, iterate, iterate, iterate, iterate based upon feedback. Let users tell you what they want.
When someone sends an SMS with their problem to triage.me like “I have a problem with my pregnancy”, we ask if the user is a woman, and then if they’d like specific help re: pregnancy. Ideally when building a solution involving diagnosis, you would avoid predicting the diagnosis by parsing text. The reason for this is that it takes time to train a solution, and even then it’s prone to error. For example, here is an attempt to guess a diagnosis via word processing that mistakes Eminem lyrics for GI problems.
The ideal implementation is to drive users towards entering text that has meaning already instead of trying to derive meaning from any statement of text. Google and HealthTAP do this via auto-complete in the browser. However, when receiving queries via SMS, these options don’t exist. We have to accept any response that a user gives us and try and do something with it. As such, better understanding SMS input is important. Instead of guessing on potential use cases, I took a page from 37 signals and I decided to build a corpora of responses via survey and then test the problem/word dictionary against it.
It took 40 minutes to build out the survey’s data collection tools. We had 34 responses. I learned and responded to the following things:
1) I added a bunch of words regarding health problems for women and children (from a parent’s perspective). It’s hard to put myself in those shoes, so this data was rich.
2) It allowed me to test our word-matching for false positives against unrelated sentences. I made our word-transform matching more stringent on shorter words and weighted our rankings a bit differently. I had left these more loose originally to better accommodate for slang.
3) I got some great feedback on ways to conduct the survey. In the future, we may perform the surveys with photo descriptions or via SMS. We only have 160 characters to describe prompts to users (in multiple languages), so we can always use guidance on how we can build more clear language for users.
4) In the end, we were matching about 66% of the terms we prompted people for. We’ll see if that remains fixed as we change the format of surveys. We also reduced our rate of false positives 50%, triggering only a false positive on less than 10% of searches. Some of these are still tricky. ‘Tooth’ is a Double Metaphone of ‘today’, which triggers as a false positive every time someone used the word today in a sentence. But, we know where we need to improve and can improve this slowly.
If you want more info on this process, I’d love to tell you more. Thanks for your help! Keep helping us out as we have surveys and we’ll keep providing a better service.
4 weeks ago at 3:00 AM, I woke up from a brief sleep after preparing some changes for our new version of triage.me. I got in my car, drove to Milwaukee, flew to San Francisco, and prepared for our Rock Health meet and greet and interview the next day. When I stepped foot off the plane, it was my first time ever being in San Francisco.
Very few things in business propose a tenable upside with little to no downside. You can build a product that is never used. Most business deals share some amount of risk between both parties. Rock Health is something that we wanted to participate in because it only offers upsides. Mentorship, partnership and fellowship. The only potential con (if even considered such), was that we needed to move to San Francisco for the Wisconsin winter.
So, when we found out we were accepted into Rock Health, we knew that we needed to jump on this immediately. It’s hard to plan for the potential of moving somewhere, so in one week we packed up our earthly posessions, subletted our apartments and headed west. We spent two days in Dan’s station wagon, rolling across the plains and the continental divide. Some highlights:
Our new gorilla friends in Lincoln, NE. Also, a pretty solid gyro.
The mountains by Park City, merging into the road.
The Salt Flats, stretching forever
First day of
school, Rock Health photo. I tried to go for a grinny, wide-eyed smile, but instead just look insane. Just like any good childhood photo.
Thanks to Chris and Rebecca Burke for letting us stay with them as we were in transition!
“Alas, Poor Gmaps! I knew him, Horatio.” — Mark Olschesky, 1:00PM, applying our iOS6 Google Maps fix.
Yesterday, marked the end of an era. If you’ve been following along on facebook or twitter, you’ve probably come across someone complaining about the new maps being packaged in iOS6. While we were enjoying playing with the new 3D features on my upgraded iPhone in the office, we also greatly lamented the loss of public transit maps. Triage.me relies pretty heavily on providing public transit directions and maps to our users when possible, so needing to implement a less than optimal HTML5 Google Maps solution for iOS6 users was a step backwards for us.
Also, I personally thought that it was insult to injury that if you went to enter directions for public transit in iOS6, that Apple would suggest that you could buy other apps that offered public transit directions after hitting the search button.
But, for me the bigger impact to technology resulting from the debut of Apple Maps is that programmers can no longer rely solely on utilizing Google Maps to reach the majority of users across all devices. Before yesterday, all computers and mobile devices except for Windows Phone (which only comprises 3% of the global mobile phone market and even less of the total percentage of device users) had optimized solutions for Google Maps. You could provide a link like:
The link would take you somewhere in a cool and concise fashion on almost all devices. Services that relied on advertising revenues from page views probably spent the time to embed maps on their pages. Companies like ours cared more about making sure that public transit routes or traffic estimates were accurate for the services that we were providing to users (in the cheapest way possible). As such, punting someone to Google Maps with a link that we created with pertinent data was simple to do and perfect. Now, as developers, we either need to build more complex logic to handle routing by device, build more elegant solutions with APIs into our sites (which is time consuming) or let our users have the less than optimal HTML5 gmaps experience. If the need for a device-specific routing engine is already in place, the barrier to transitioning to an OpenStreetMaps based solution for businesses is lowered even more. Having more solutions means that the design will become more complex which means that maps functionality will inherently work less reliably. Sad but true.
Trying to look on the bright side of things yields a few positive points. Perhaps the competition will provide fuel for more innovation from maps providers. Maybe Google will go back to offering Google Maps services for free or will keep lowering their prices to handle the competition. We may even see companies like MapQuest start to take a stab back at Google Maps.
Google Maps virtually unchallenged reign on top is over in the era of constant connectivity. Long live the new democratic maps process and the pros and cons that come along with it. Here in Madison, we’ll keep improving triage.me to provide the best methods to get to clinics; even if that means that we need to build and test more solutions to do so.
I’m excited to present our newest additions to triage.me. Since our presentation at the HDI Forum, we’ve been making small changes to triage.me to improve performance and stability, as well as preparing for the time when we would inevitably be absorbing actual patient data. We’ve recently looped back to include a few incremental updates to triage.me, which include the following items.
All HRSA clinics loaded into triage.me
Yesterday morning, we loaded 9158 clinics provided to us by HRSA into Triage.me. This means that every clinic that receives federal-funding that provides care to patients at whatever cost that they can pay is now loaded into triage.me.
HRSA produces a file of clinics daily and we’ll try to upload and update a new batch of clinics from here on outwards.
This means that triage.me is a useful and cheap tool for anyone, including caregivers and volunteers, to find people care. Triage.me’s prescriptive recommendation engine provides a specific place for a person to go in seconds. The one piece of information that HRSA did not provide us with was the operating hours of clinics. We know the clinics that are always open. Unfortunately, most of them are in small towns in Alaska. As such, we still need to put some of the onus on users to determine if the clinic is open. We look to start cycling through and finding ways for clinics to update their hours soon. In the meantime, if you know that a clinic has details that need a change, we’ve added a button to let us know of the changes that you would like to see made.
We still have the existing beta care locations from Wisconsin and Washington, D.C.
Now on the triage form, we’ve included a button that allows users with supported browsers (basically all browsers and smartphone devices except for IE7 and IE8) to automatically enter their address via geolocation into triage.me
Like all geolocation services, we only gather your location upon your consent. It saves users time and improves the accuracy of our triage.me recommendations.
Triage.me was born of a hackathon. It’s continued optimization is borne from feedback from our users and our observations of the program out in the real world. One thing that I had noticed upon visiting the hospitals and offices of our users and partners was that many of our users were using older browsers on smaller monitors.
While the fundamental design has remained the same, triage.me now has a new fluid UI that better supports older browsers and monitor resolutions. It also better supports iPad usage.
Next up is building a tool that looks good and works well on mobile devices. These pages do not always look well when they are set to scale automatically, so we’re working on the formatting to display triage.me to fit these devices best next.
Other build and optimizations
The search for clinics is even faster now. Conversations via SMS occur almost in realtime. The only barrier to the speed of your conversation is the speed to which you connect to your mobile provider. We also rebuilt our SMS functionality to handle the ordering of messages better.
We’re happy to offer a new set of tools for our users, getting your feedback, and building more things that you find to be useful.
This past weekend the National Governors Association met in Williamsburg to discuss health care and health policy issues facing states. It was the longest stretch of C-SPAN I’ve ever watched.
Panelists included Dr. Jeffrey Brenner, founder and executive director of Camden Coalition of Healthcare Providers, Jennifer DeCubellis, director of Hennepin County Health, and David Vellinga, president and CEO of Mercy Medical Center. Each brought a unique and valuable point of view to the table.
One major topic of the day: lowering Medicaid costs.
Although relevant to what we’re doing with triage.me and reflective of the larger vision we’re working to realize, this post isn’t focused on Medicaid, uses of geodata or the best way to influence behavior change in high-risk patients. (Though, if you’re involved in the health care field, I do suggest watching the full 88 minute live feed of the session.)
Instead, I’d like to drill into an exchange initiated by Governor Gary Herbert from Utah challenging a statement from Dr. Brenner that “health care has not innovated”. On it’s surface, Governor Herbert espouses a point of view echoed frequently throughout the health care debate. It is latched onto by those who do not understand the underlying, systemic issues in health care, and those unfamiliar with the access to care issues suffered by our large underserved communities.
While a great topic for discussion, I’d like to push his comments aside and focus on Dr. Brenner’s response. For any technologists looking to bring “innovation” to health care, understanding Dr. Brenner’s point is critical to your success.
In summation: Problems in health care are not caused by technology (unless we’ve created them ourselves). Introducing a new IT system or network won’t fix the smallest niche problem if you don’t also reengineer the broken operational processes underpinning it all.
During my time at Epic, I became keenly aware that without first addressing the underlying operational processes, even the best designed system will fail. We’ve seen this play out as organizations and states spend hundreds of millions of dollars looking to implement massive technology projects, only to see them fall short – suffering from poor user adoption. Diving deeper into the HIT startup game, I’ve seen an increasing number of companies looking to will better workflow through the advent of a new technology or an improved user interface. So, what’s the take away?
You can’t be successful introducing innovation to health care unless you first reengineer the broken process. If you’re not a process engineer familiar with health care and the existing relationship between payors, providers, patients and government, you should add this person to your team. ASAP. They should be someone who can build a successful implementation division, as well as steer operational leaders in the complex task of reorganizing their business.
If there’s one thing you take from Dr. Brenner’s answer, it should be that technology will not fix health care. Solutions do not need to be robust and novel uses of technology, what they need to do is enable improved operational workflows. If you don’t sell, and then assist with process reengineering, you’re not going to innovate health care.
In 2003, I bought my first cell phone. It was a Nokia candybar style phone. My biggest complaint about the phone was that it didn’t even have a vibrate feature on it. The coolest feature was that it had the snake game on it. It had SMS functionality, I didn’t use it because I barely had friends enough with cell phones, let alone friends that send text messages.
In late 2007, boiling with envy at my roommate’s, I went and I bought the original iPhone. My biggest complaint about the phone was that everyone wanted to look at it (remember when iPhones were novel?) and then proceeded to immediately drop it on the ground. The coolest feature was that I was able to sync up work email without carrying around a blackberry the size of my hand. I remember balking at the idea of being required to buy 300 SMS with my unlimited data plan. When would I ever send 300 text messages? I called people. I had always had lousy phones with T9. Texting was something that my younger sister did with her high school friends. In 2010, due to some combination of texting friends around the country about the World Cup and summer planning, I managed to crush my 300 SMS limit by 113 messages, promptly causing me to cry about the money no longer in my pocket.
How did we get here? How did we go from phone calls to text messages? What changed?
SMS is the universal way for people to digest information in an easy way that is comparable to other rapid fire technologies that are now prevalent. It’s also universal. Most people own a cell phone these days; However, not everyone owns an iPhone, smart phone, whatever. It’s the type of communication that everyone can use, across all social classes and degrees of love for technology. Pew research and anecdotal research by Dr. Aziz Ansari indicates that people who text message are more likely to respond to text messages than to phone calls. As such, if your company knows that someone is likely to be a “texter”, you should probably try to communicate with them via text.
However, as a developer, I can now see why everyone isn’t writing a ton of applications to rely upon text messaging. A few of the limitations are:
Cost: Low end web hosting is pretty much free or is available for dollars a month. Even the cheapest SMS APIs charge 1 cent per text message. On its face this seems cheap, but imagine the scalability within a workflow. A basic conversation that involves 3 back and forth text messages costs 6 cents. This means that if you have 100 conversations, your cost is 6 dollars. What about 10,000? 600. 1 million? Suddenly, you’re now spending 60,000 dollars on SMS messages, which would be an insane tech budget for something so simple as 160 character message transfer.
Client-side processing is so awesome, server-side processing seems dumb by comparison: . Go to google maps and type the name of a street address near you or in a city that you’ve been to. Don’t include the city name or zip code. Did it guess the right city? Probably. Why is that? Because google is using a ton of data to guess your location. Your IP address, your searches, probably alot of your personal information. Big Brother exists, and he’s working to make sure that you find the right street named after an old colonial leader. However, with SMS-based systems, we pretty much can only rely upon the phone number to do any search on the server-side. This means that we now need to *gasp* ask someone to enter the City and potentially state that they are in to get provide an accurate set of directions. In and of itself, this isn’t terrible. But, people are so used to technology being magic, that it’s hard to set their expectations that it’s not.
It’s not as cool: There are no graphics, no design magic, no flashy tricks with SMS. Just 160 characters. Brutally practical, but not easily exciting.
We keep chugging along since we think text messaging is the most compelling way to reach people in need. But, there are challenges that are not necessarily obvious at the onset of designing a system that involves SMS. We’ll keep posting in the future about how we get around some of these hurdles as we clear them.
Now that the push to get things ready for Datapalooza is shifting into a push to capitalize on the momentum coming out of the event, I’ve had a few moments to reflect on the events that have gotten us to this point.
Four weeks ago, things looked very different. Yes, we were excited about the concept that emerged from the hack-a-thon, but we hadn’t won the event and didn’t have an invite to head to Washington. We were now a two person company with two different products in development and we needed to decide where we were going to focus our attention.
Everyone we talked to told us triage.me was worth developing further. We agreed, but it was still a difficult decision to set aside what we’d been working on for the last 5 months, especially since we still deeply believe(d) in it.
Fast forward a few days and I’m working in the Horizon coworking space, it’s getting late and we’re starting to think about a little break for “Zombies”, a.k.a. Left 4 Dead. Instead, I decided to head over with a few coworking buddies to grab a beer at the monthly “Capital Entrepreneurs” meetup. I’d never actually attended one of their events, but as luck would have it, this was the once-a-year “open” event and non-members were invited to attend.
While waiting at the bar, I ran into Alex, a member of “Team Hardware” who had taken first place at the BuildHealth event and was destined for DC. He let me know that the team wasn’t going to attend the Datapalooza and suggested we go in their place. It was an off the cuff remark and had a series of small events not taken place that day, and in the days following, there’s no way Mark and I would have ended up with a spot at Forum.
So, we had 4 weeks to design a booth and rewrite an entire product. Oh, and Mark was already committed to working 30 hours a week in Chicago. We called in whatever favors we could muster and got some wonderful help from Abby Larner reworking the front-end design. Aurora stepped in to help with the booth and drove one of their display boards across the city so we could pick it up.
Where are we now? Mark and I are fully committed to building triage.me into the platform we know it can be. We’ve received a lot of positive feedback and guidance over the last few weeks and there ia a lot of thanks due.
Stay tuned, this ride’s just getting started!