Healthy Hacking

Mid-Week Update

Wed, 25 Jan 2012 23:41:28 -0500

We have been hard at work (I know I’m repeating myself) but all of our work is starting to come together and the results are pretty cool. We have the system essentially working completely as planned. The doctors have also been working hard to get props (and a real cardiac rehab patient) lined up for our promo video. To get into a bit of tech detail, I’ll describe the application step-by-step in case you’re interested:

We start the server and all the clients connect to it. Currently, those clients are a bluetooth pulse oximeter, a system running Unity, and an interface to the IndivoX personal health record server. Each client connects over a TCP connection using a server/client framework we built on top of the .Net TcpListener and TcpClient.

Next, the server sends an initialization request to the IndivoX interface, which grabs an exercise plan and some user information (age, gender, height, weight) from the personal health record system for the current user. This is currently dummied up, as the exercise schema we generated was going to take too long to implement on the IndivoX back-end. We are using a class generated off the schema, though, so it should work essentially the same as it will once that functionality exists within IndivoX.

The server now takes some of that data to use in the calculation of target heart rate, blood oxygen level, etc. It also sends the exercise plan to the Unity system, which then maps the exercises into an instructional avatar. Once the user is ready and in front of the Kinect system, they control an avatar next to the instructor on screen. This avatar maps to their body movements, and they follow along with the instructor avatar.

While this is happening, the system is sending a stream of all the joint information (position and rotation) back to the server for algorithm processing. Based on the algorithms, the server sends data back to the Unity system which will then instruct the user to keep going, speed up, slow down, etc.

Finally, once the exercise plan has been completed, the system sends a record to the IndivoX interface so the result of the exercise can be recorded.

Tomorrow will be a day of polishing the system and making sure the algorithms are working pretty well, plus we film our promo video with their film-making team. Should be a fun last day of development to wrap up a great couple of weeks.

Monday Summary

Tue, 24 Jan 2012 14:10:36 -0500

Monday was a long day that saw a lot of things accomplished and today is shaping up the same way. We’ve gotten a lot done and are really getting into the detail work at this point.

Highlights of the current status:

Our central processing component is now accepting connections from all the different components and receiving/broadcasting streamed data to each.
We have documented our proposed schema changes to the Indivo personal health record server and created classes modeled on that schema.
We’re sending actual messages back and forth from the different clients rather than pure connectivity tests.
Our Unity code is able to take pulse rate data and provide the patient with visual and audio feedback cues (slow down, keep it up, etc)
The central processing component is able to tie in X number of algorithms to process the motion and heart rate data, and work is being done on the specific algorithms that we plan to implement as part of this project
Solid progress has been made to add a CollaboRythm (cross-platform Indivo client created by the MIT New Media Medicine group) plugin to display the data we are tracking in our system
Completed and submitted our script for the promo video

We’re still on track to complete all of our goals and achieve a little bit more than the initial plan, so the team is feeling pretty good right now.

Weekend Summary

Sun, 22 Jan 2012 11:27:47 -0500

The weekend isn’t over yet and we’re still doing some coding, but I thought I’d take a minute out to update our status. All of the major components are working with baseline functionality, with the remaining work involving the extension of some APIs to store the data we’d like to record, fine-tuning the system, mapping in the algorithms to process both the motion data from Kinect and the bluetooth pulse oximeter, and displaying the results on-screen in Unity. Our bluetooth expert has gotten far enough along that he is looking into some features that weren’t on our initial schedule. Additionally, the doctors in our group have been working on the script for our video demonstration/promotion, which we’ll start filming tomorrow.

I feel like my updates all sound kind of the same, but we’ve made a ton of progress so far so I thought it would be useful to see a list of how much has been done:

Mapped motion from Kinect to custom Unity character models
Updated character models to match the game’s target demographic
Built a TCP client-server architecture to allow the different components to “speak” to one another over a distributed, cross-platform system
Created TCP stream of Kinect data into our data processing module
Created a C# (mono) client for the IndivoX personal health record system
Created an application to read and stream pulse oximeter data over bluetooth
Acquired documentation mapping exercises to Metabolic Equivalence of Tasks (MET) scores, which are used as a gauge of cardiac health
Developed draft of the script for our demo/promo video

With each of these tasks there was a lot of discovery involved so it’s been great to work with such an efficient, helpful, and collaborative team. Plus, this has all been accomplished in only 5 days! We definitely feel like we are on track to accomplish our goals and we are hopeful that we can add a few features that we think will be useful.

A creation of the High Low Tech group, who focus on applying...

Fri, 20 Jan 2012 00:23:55 -0500

A creation of the High Low Tech group, who focus on applying technology to everyday items. Like paper. The ink acts as a conductor, forming a circuit (and making the piece of paper a very low-tech circuit board)

GlowCaps. These were created by the Tangible Media department to...

Fri, 20 Jan 2012 00:20:51 -0500

GlowCaps. These were created by the Tangible Media department to make it easier to take the right amount of the right medication at the right time. Vitality is the company that was started to sell GlowCaps, and its CEO is David Rose, who is a participant at HW2012.

Chocolate (stale). These are the result of an app called...

Fri, 20 Jan 2012 00:15:54 -0500

Chocolate (stale). These are the result of an app called “Eat Your Face”. They do a high resolution facial scan, then make a 3D print of the face using a chocolate printer.

The Media Lab machine shop. They have a bunch of cool machines...

Fri, 20 Jan 2012 00:11:49 -0500

The Media Lab machine shop. They have a bunch of cool machines in their lab. A water jet cutter, a bunch of laser cutters, 3D printers, etc. When they need something for a project and it doesn’t exist, they figure out how to make it themselves.

Day 3

Thu, 19 Jan 2012 18:41:00 -0500

There’s not too much to report from day 3. Not that we didn’t make progress, but it was more iterative over what was done yesterday.

The doctors rejoined us and tracked down a lot of good documentation for us. We’ll use that to do the medical analysis so we can provide real-time feedback to the rehab patients through the Unity interface. The dev team stayed on track, with all the pieces progressing as expected.

We also got a guided tour of the lab in the morning, so I’ll post some pictures a little bit later.

Day 2 Summary

Wed, 18 Jan 2012 20:41:13 -0500

I posted some pictures of the New Media Lab below so you can get a sense of how things are set up here. Open space, windows, and whiteboards everywhere.

As for the project today we got into the actual work. The doctors on our team were busy being doctors, so it was mainly our group of 5 devs modeling out our system, figuring out our resources, and taking on tasks. We got our custom character model in Unity mapped to input from the Kinect, so that was a great start. We also have the bluetooth pulse oximeter streaming data, and we got documentation and a walkthrough of the personal health record software we will be talking to. All in all, it was a very productive day and we got most of our goals accomplished.

We still have plenty of challenges ahead of us to get all the different components talking to one another and adding the analysis portion (probably the most technically challenging aspect), but it seems like its doable in our timeframe. I didn’t get a chance to check in on the other teams too much today, but it seems like most groups spent a lot of the day planning. I’ll try to wander around tomorrow and get a sense of where the other groups are and any challenges or cool solutions they’ve come across.

The offices of the event’s hosts, the New Media Medicine...

Wed, 18 Jan 2012 20:31:06 -0500

The offices of the event’s hosts, the New Media Medicine lab

No game room is complete without fluorescent projected fish on a...

Wed, 18 Jan 2012 20:29:45 -0500

No game room is complete without fluorescent projected fish on a ping-pong table, a 6 foot stegosaurus, and a 4x4 bank of LCD monitors that form a single screen.

One of the offices in the MIT New Media Lab. I think this one...

Wed, 18 Jan 2012 20:27:32 -0500

One of the offices in the MIT New Media Lab. I think this one focuses on digital innovations and had a lot of camera-related things at their door.

Our quickly assembled dev area

Wed, 18 Jan 2012 20:25:19 -0500

Our quickly assembled dev area

Opening Day Thoughts

Tue, 17 Jan 2012 21:13:00 -0500

So the most interesting thing to me about the first day was to see which themes emerged on the different projects. The most common conversations, by far, revolved around stickyness and usability. For the software industry, these are pretty standard concepts, but from the conversations today it seems like these are topics that have been somewhat ignored in the health industry.

None of the projects involve creating a new cure or a new diagnosis. Nearly all of them are simply focused on using new technology to make adherence an easier job. Coming up with ways to use technologies to capture people’s interest is still a difficult task, but in these cases software, hardware, and behavioral psychology all become almost equally important in the wellness process as medicine and prescribed behavior.

Tomorrow we get down to the business of actually creating something, and every project has to complete something that is demonstrated next friday, so it should be a busy and fun couple of weeks.

Oh, if you want to check things out on twitter, check out the hashtag #MLHW12.

e-Patient Dave also has set up some pages to track projects, which you can check out here. That’s it for me for tonight, but I’ll post more tomorrow. Dave also had some inspiring words about the Innovation event, so I’ll close the post with those:

What’s conspicuously different about this event compared to any other I’ve seen:

In this room, if what needs to get done isn’t getting done, it’s viewed as a system failure, not a “patient compliance” problem. These people are striving to make everything work better, in ourhands.

Opening Day

Tue, 17 Jan 2012 20:58:00 -0500

Today was a great opening day for the event. We grabbed breakfast, broke out into our teams, discussed our projects, and then presented them for the group to collaborate and brainstorm. I’ll run through each project really quickly, then add another post to talk about my overall impressions.

The first group that presented talked about a Pain Management project. Here is a description of their project goals. The big difficulty here is trying to quantify something as vague as “pain” so doctors can better prescribe treatments and measure progress. There were of interesting discussions revolving around positive feedback loops and control of pain, both of which have been demonstrated to reduce anxiety in patients and, consequently, their perceived pain level.

Post-Operative Management was the second group to present. This was a fairly expansive project with goals to do light diagnosis via a mobile device, as well as a tool for telepresence for doctors, and a kid-friendly mobile app using an avatar to provide positive reinforcement. There was some interesting discussion around FDA approval levels for systems/devices as well as some conversations about how to get proper input through an automated system to be able to provide helpful diagnostics.

The Asthma Management project aims to make a user-friendly device or game to make sure children are taking their asthma medication regularly. This was the first time (but not the last) where we stumbled into a bit of a dilemma: how do you best design a system that is engaging over a long term (in this case a lifetime), and yet does not over-encourage participation. In the case of asthma, the game/device would require the patient to blow into a device, but if they overuse the application it could trigger an asthma attack, leading to a negative correlation with the device.

The next group presented a Diabetes Management system. This primarily revolved around two devices. One is an insulin pen with a bluetooth component so it can provide data to external systems, the other is the Motorola MOTOACTV, an android-based fitness tracking device. The basic idea was to gather both minute-by-minute user fitness data along with the schedule and dosage levels from the insulin pen. Motorola is opening up their platform for this event so the data can be exported out to health tracking systems, so it should be interesting to see what applications can come out of that.

My group was next, discussing our Cardiac Rehab project. We are integrating the Microsoft Kinect (a 3D camera and skeleton tracking system) with a bluetooth pulse oximeter (measures heart rate and blood oxygen content) to provide active feedback to users in a virtual world game environment. A lot of the conversation around this was focused on how to make a rewarding game system that doesn’t encourage patients to overexert themselves in a dangerous way.

The Humanizing Health Applications was the final group. Their project is more of a system idea, where they’ll be trying to come up with a system to map health goals (physician-focused) to life goals (patient-focused). They are also focusing on social components of ongoing care to provide encouragement, emotional reassurance, and provide better tools for patients to describe how they feel.

It was really cool to see all the different groups and to get a sense of people’s passion and drive to solve some prevalent issues in the health and wellness industry. I’m excited to see what everyone can build over the next two weeks.

View of Cambridge upon getting to my hotel last night. Who knew...

Tue, 17 Jan 2012 20:16:18 -0500

View of Cambridge upon getting to my hotel last night. Who knew Boston was cold and snowy in January?

Health and Wellness Innovation 2012

Sun, 15 Jan 2012 19:36:00 -0500

Health and Wellness Innovation 2012:

Here’s the homepage of Health and Wellness Innovation 2012. Check out the projects page to see some of the things we’ll be trying to create.