Loop vs OpenAPS

Note:  This post is outdated because the features of Loop and OpenAPS have both been updated since this was originally published…The link to the updated post will be posted here shortly.

 

How do you choose the “best” DIY closed-loop system?  The answer is still a YDMV (your diabetes may vary) answer…but here’s some thoughts from a parent’s perspective.

I have a 14-year-old t1d daughter.  She is relatively independent and can count her own carbs fairly well.  After a little practice and gentle reminders, she can usually learn new things about her t1d management.  BUT, she’s not an independent adult.  She’s prone to forgetting things in the presence of her peers, getting distracted in busy environments, and not advocating for herself in the presence of authority figures.  She worries about fiddling with diabetes gear in class (“will the teacher think I’m playing on my phone if I need to text about diabetes?”).  So, as she goes out in the world with a system making automated decisions about insulin on her behalf, I have a different set of considerations about a system than an adult t1d closed-looper might.  (I’m going to try to keep the little kiddos in mind too as I walk through the differences between these two DIY closed loop systems; Loop and OpenAPS.)

In any comparison, it’s a good idea to know the person’s assumptions and priorities when they are doing the comparison.  Here’s my short list of the non-algorithm issues I considered in evaluating Loop life vs OpenAPS life.

  1. Size
  2. Cost
  3. Durability
  4. Ease of build
  5. Ease for caregivers/school nurses
  6. Troubleshooting
  7. Reliability
  8. Portability
  9. Ease of use
  10. Kid-specific features

I intentionally left BG control off the list is because either system is capable of achieving great BG results.  BUT, the ease/difficulty in the WAY you have to adjust your thinking, actions, or lifestyle to achieve the desired control was important to me.  And I’ll get to that shortly.

Size, Cost, and Durability (Advantage: Even)

These factors are virtually the same between the systems.  Both systems can be easily put in a box about the size of a tic-tac box.  The Loop system costs about $99 annually (Apple developer program), Heroku (for Nightscout) is free, and the RileyLink costs $135.  OpenAPS costs about $163 for the parts, and there are potentially some data costs depending on how you set up your system.  If you want to run multiple rigs, you may need to upgrade to the $7 per month Heroku “hobby” plan (for one rig you should be safe under the free Heroku plan still).

Both systems consist of a small circuit board and a rechargeable lipo battery…so the same care and durability should be expected between the systems.  We plug both systems in for recharge nightly and they last the whole day on the one charge.

OpenAPS Rig on the left, RileyLink on the right

IMG_5352

 

Ease of build (Advantage Loop)

I had both systems built and operational within about 2 hours of getting all the parts together.  Am I some sort of mad genius with computers?  Absolutely not.  You can do this, too.  There are guides for both systems that tell you what to do and the order to do it.  Both systems have very helpful groups on Gitter where people will help you through the setup process…Loop and OpenAPS.

That said, Loop can initially seem like a more relaxing build for the people who are uncomfortable with “code” since much of it is done through the Apple interface (think, pictures and symbols).  OpenAPS is done through linux commands and scripts.  HOWEVER, do not let this deter you.  Both systems will take you into places on your computer you have likely never been before (Loop to Xcode, OpenAPS to Terminal), and you will live to tell the story.  I am actively involved in providing some updates to the OpenAPS docs that will hopefully help new, non-code people through the setup process…just as easily as Loop can be done.  To be honest, setting up an Apple developer account was probably the most frustrating part of the Loop setup and putting two tiny screws in the explorer board was probably the most frustrating part of the OpenAPS setup.  Notice, neither complaint is actually about the software building.

I’m here to say…don’t let your intimidation stop you from trying either or both systems. (Perhaps slight advantage to Loop here…but the gap is closing.)

Ease for Caregivers/Small Kids (Advantage OpenAPS)

Here’s one that adult t1ds probably don’t have to worry about.  Can a school staff member reliably dose your kid with insulin through the closed loop system?  Here is where I was surprised to see a large difference between the systems potentially.  Anna has operated her Loop inputs fairly independently and has gotten used to some of the quirks, so in many ways she’s more like an adult user.  However, for little kids relying on school staff for meal bolusing the differences between systems may be more important.  Loop uses the app to enter meal carbs and calculate/administer boluses; whereas OpenAPS uses the pump.  OpenAPS likely has an advantage for these considerations:

  • Failed boluses  – Every once in awhile, for whatever reason, a bolus command in the Loop app will not go through.  A notification pops up on the iPhone saying “Bolus may have Failed”.  If the caregiver/kid doesn’t notice the notification, it may be some time before the missed bolus is recognized.    And unfortunately, sometimes, the bolus actually will begin shortly after the notification pops up.  The caregiver/kid will have to be trust-worthy enough to recognize the need to look for the message and if the message appears, to first check the pump to ensure the bolus is not actually already in progress before attempting to re-enter the bolus command.  Since OpenAPS is bolusing directly from the pump, failed boluses would be immediately obvious to a caregiver/kid. (Advantage OpenAPS)
  • Bolus wizard guard – The default setting for Loop is that the app will not suggest a bolus if the t1d is below, or predicted to go below, their minimum BG target.  This feature, called the bolus wizard guard, can result in the carbs for a meal being entered but no bolus being offered.  The bolus wizard guard can be removed or modified by parents, but it is definitely something to be aware of.  Many new Loop users fail to modify this feature and find no bolus being given.  Parents should have a plan in place with the caregivers in the event no bolus is offered by the app.  OpenAPS does not have an analogous bolus wizard guard.  (Advantage OpenAPS)
  • Double Carb Entry – When a carb entry is entered in the Loop app, there is a “save” button to move you onto the bolus recommendation part (just like a pump’s bolus wizard).  In both Loop and OpenAPS, an entered carb is saved by default.  Backing out of a bolus wizard does not cancel the carb entry.  In Loop, the caregiver/kid can go into the app and cancel/edit the unwanted carb entry.  However, in OpenAPS you cannot edit the entry.  So both systems require a consistent method of carb entry to prevent unintentional double entries. (Advantage Loop)
  • Yellow/Red loop – Loop will not bolus if it is having trouble communicating with the pump or receiving BG readings.  The Loop will turn yellow or red, depending on how long it has had problems with communications.  If a caregiver/kid doesn’t resolve the issue, they can still enter a bolus through the pump directly as a backup.  However, the caregiver/kid will need to separately enter the carbs into the app (Loop does not read the pump’s carb entries).  Since OpenAPS boluses are directly from the pump, meal boluses can continue as normal even in the event the looping is not functional at the time of the meal.  No special considerations are needed.  (Advantage OpenAPS)
  • Carb Absorption Time – Loop has a unique feature called Carb Absorption Time for each carb entry.  Basically, it is akin to saying how fast or slow Loop should expect the food to absorb (and thus affect BG levels).  If you have new food or classroom treats, your caregiver/kid will need to estimate the carb abs time of the food.  Loop can tolerate some inaccuracy in those estimates, but if you are off by significant amount (like 60 minutes vs 120 minutes estimate) you may end up under- or over-corrected by Loop as the meal progresses.  Also, it’s important that your caregiver/kid respect the importance of entering in the correct carb absorption time for a meal…otherwise unwanted BG impacts can occur.  In OpenAPS, the absorption of carbs is tracked dynamically by the loop.  Therefore, the caregiver/kid does not need to estimate a carb absorption time as part of their mealtime entries.  (Advantage OpenAPS)
  • Emergency tube changes – If you child needs an emergency tubing change at school, there is a special consideration for Loop users to be aware of.  Priming new tubing, without a change of reservoir, will require the caregiver/kid to delete 30 minutes of reservoir history in order to not have the primed insulin count as delivered insulin (IOB).  Parents and adult t1d users of Loop get pretty used to this action, but trying to describe this step over the phone with school staff or young kids could get pretty difficult.  OpenAPS reads the pump’s actions and will not count primed insulin as IOB.  A notation is also made on the Nightscount site automatically when a prime event occurs.  (Advantage OpenAPS)
  • Net IOB – The Loop app tracks net IOB (IOB from both boluses and basals). As such every Loop bolus and correction is automatically consider net IOB. However, Medtronic pumps do not track IOB from temp basals. In OpenAPS, since there is no app, netIOB is usually seen in Nightscout (in fact, the IOB pill will even break down the basal IOB contribution). Caregivers/kids will need to be aware that the usual “take BG before a meal and add a correction adjustment to meal bolus using pump’s bolus wizard” should not happen. Most OpenAPS users will only bolus for their meals using straight carb ratio, and not use the pump to suggest a +/- correction bolus. (Advantage Loop)
  • Corrections – BG momentum in Loop has proved to be a difficult concept for my teen to watch for in Loop.  In times of rapidly increasing, BG momentum can tend to over-predict and thus offer too strong a bolus correction.  This can also affect meal boluses in Loop.  OpenAPS boluses using straight carb ratios seem slightly safer for small kids under remote care.  Plus the bolus snooze option helps corrections take effect without having the loop go to low-temping when you might be stuck on a stubborn high for whatever reason. (Advantage OpenAPS)

Troubleshooting (Advantage even)

Loop relies on the RileyLink for communications, OpenAPS relies on the explorer board.  Both feed information into Nightscout about the status of those communications and looping.  A remote parent can receive information about potential problems that may be causing the loop to stop running.

For both systems, the solution to getting looping started again is usually pretty straight-forward.  For Loop, it’s almost always a retune.  For OpenAPS, a power button push.  However, advantage goes slightly to OpenAPS because a power button push is a little faster and doesn’t require opening an app.  And if your kid isn’t capable of retuning the RileyLink themselves, explaining how to retune a RileyLink can be a little cumbersome over text messages or phone calls with school staff.  A power button is a more straight-forward explanation.

If a fatal program error happens on either system, they can both be rebuilt fairly quickly and easily following the original steps.

Reliability (Advantage OpenAPS)

How often does each system go down?  How far can your kid get from the rig before communications/looping quit?  The answer is “it depends”.  If you live in a “noisy” radio frequency environment, you may find yourself needing to retune RileyLink fairly often.  If you live in an area with no wifi or cell service, you will likely have a poor OpenAPS experience*.

But, the advantage here goes to OpenAPS’s rig because it’s communications has a longer reach and seem more stable.  Kids don’t have to be quite as close to their rig as they do to the RileyLink.  On average, I’d estimate we retuned the RileyLink about 2-3 times per day.  On OpenAPS, we have had to power cycle the rig only once in 12 days (and I think that actually was just my impatience rather than truly a problem).

Portability (Advantage even)

The portability of Loop is pretty great.  You could take it to the middle of nowhere and still be able to run a closed-loop…no internet, wifi or cell needed.  HOWEVER, I’m a remotely-monitoring parent and I like to check Nightscout to see how things are going.  So, even Loop needs internet connection for the parent to monitor things.  (If I were reviewing portability as an adult t1d, Loop would be an easy advantage.)

The newest release of OpenAPS also has made the system truly “portable” in my eyes.  Before this update, kids might have to remember to turn on their phone hotspots when leaving a known wifi network.  That’s not practical for kids.  But, the new release is a very stable handoff between known wifi networks and iPhone hotspot.  So, as your kid leaves home to go to school, the rig will automatically connect with the kid’s iPhone via bluetooth hotspot tether.  When your kid gets to school, it can pick up on the school’s wifi automatically…thus saving you on cellular data and hassle.

*There are options for running OpenAPS without internet like Loop (aka “camping mode”).  I’m just not setup like that.  You could directly plug-in the dexcom receiver to the rig.  Or you could use X-drip to upload locally to the rig.

For summer diabetes camp in the woods…we will likely use Loop.  I think.  Maybe.

Ease of Use (Advantage OpenAPS)

Oooph.  This one is tough.  The iPhone screen is soooo sexy.  Kids LOVE the iPhone.  And they love not bolusing from the pump sometimes.  Since the iPhone is already glued to her hands, Loop gets a lot of love from my 14 year old.

But, the ease of entering a meal bolus from OpenAPS is just heaps easier than Loop.  Kids (heck, even adults) can have a hard time estimating what a proper carb absorption time for a food should be.

There’s also a really sweet feature that you can find in OpenAPS that you can’t get in Loop.  As a parent, you can remotely set a temp BG target using Nightscout’s careportal.  When my daughter goes to track practice, I can set a temp BG target ahead of time to help keep her in a slightly higher target range.  I can also easily cancel the temp target if practice gets out early or situations change.

IMG_5193

I love both systems…but I do think the OpenAPS is easier to “use” if you are a parent sending your kid to a slumber party or school dance where they will have new foods and perhaps harder bolusing than they might be used to under your care.  Also nice to have some ability to remotely help nudge loop behavior “up or down” through the use of temp BG targets.  It definitely has saved me quite a few text messages as a result.

Kid-specific features (Advantage OpenAPS)

There are some feature that OpenAPS specifically designed to adapt for kid t1ds.  We haven’t used or tested these, but they may be of interest to some of you reading here.

  • Override High Target with Low: “Defaults to false, but can be turned on if you have a situation where you want someone (a school caregiver, for example) to use the bolus wizard for meal boluses. If set to “True”, then the bolus wizard will calculate boluses with the high end of the BG target, but OpenAPS will target the low end of that range. So if you have a target range of 100-120; and set this to true; bolus wizard will adjust to 120 and the loop will target 100. If you have this on, you probably also want a wide range target, rather than a narrow (i.e. 100-100) target.”
  • Carb Ratio Adjustment: “This is another safety setting that may be useful for those with secondary caregivers who aren’t dedicated to looking up net IOB and being aware of the status of the closed loop system. The default is 1 (i.e. do not adjust the carb ratio; off). However, in the secondary caregiver situation you may want to set a higher carb ratio to reduce the size of a manual bolus given at any time. With this ratio set to 1.1, for example, the loop would multiple the carb inputs by 10%, and use that number to calculate additional insulin. This can also be used by OpenAPS users who rely on the bolus wizard to calculate their meal bolus, but who want to only bolus for a fraction of the meal, and allow advanced meal assist to high-temp for the rest.”

Conclusions (Advantage OpenAPS)

I feel like I’m cheating on my best friend Loop by saying that I’ve really enjoyed OpenAPS.  All the technical stuff above spelled out…here’s the short version of the parts I’ve noticed the most.

Fewer texts to Anna during school:  Through the use of temp BG targets, I don’t have to text her to remind her to prepare the two hours ahead for track workout.  I don’t text her anymore to ask her to retune the Loop.  I don’t get texts from her asking for carb absorption times for food from classroom parties.  I spend less time asking her if she got too far from her RileyLink.

Easier mental at meal time:  I have enjoyed not thinking about carb absorption times.  Now I’m back to my regular thoughts about whether a meal would do well to split bolus…and that thought is the same with both systems regardless.  (Anna says this part never really bothered her.)

OpenAPS pill:  I get a little more information from the OpenAPS pill.  I have enjoyed that.  I also enjoy seeing the Basal IOB using the Nightscout IOB pill.

Auto-tune:  I haven’t really touched on it here much…but there’s a newly unveiled feature called auto-tune which looks back on the previous day’s data and can adjust ISF and basals automatically where it sees the values may be slightly off.  I think that needs a special post of it’s own.

Loop enacted:  I have REALLY enjoyed never seeing that the loop is stalled out or needs retuning when I open the Nightscout app.  I have pretty much gotten used to seeing the OpenAPS rig always functioning.

Easier to switch between patterns:  Anna has definite differences between school days and non-school days.  On school days, she needs about 6 hours of a lower basal rate during the day and through track practice.  Switching between patterns on the pump has been just a little easier than switching between the two different Loop apps that I had setup.  It’s just one little less thing to do.

If I were an adult t1d, my metrics would probably weigh the various considerations quite differently.  But as a parent of a t1d, I’ve really enjoyed OpenAPS.  We have great BG control on both systems…I’m just trying to find ways that we talk less about diabetes and more about everything else.

Looping in our Life

On August 29, 2016 after my daughter’s first week in high school, she asked to change her insulin pump.  She no longer wanted to be on omnipod…she told me she wanted a tubed pump.  Her reasons were simple; she wanted to show off a pump and she wanted to just be able to directly bolus from a pump (instead of pulling out her PDM bag and unzipping).  I was thrilled because this opened the door at last to the DIY closed-looping community that I’d been reading about for awhile.

You can read about the history of the community in many places (I’ll add links to the bottom of this post later, in fact); but the short version is some really smart people got tired of data sitting unused and unaccessed.  They all developed various parts and pieces (both software and hardware) to make it so that the CGM can talk to an insulin pump and make automated decisions about basal insulin delivery to help stay in target BG range.

Blood sugars going up?  Insulin pump is told to delivery more insulin.  Blood sugars going down?  Insulin pump is told to deliver less insulin.  That’s a grossly oversimplified version of the algorithm (math and decision tree) that is the heart of a closed-loop system, but that gives the rough idea.

So why did Anna need a tubed pump to make this happen?  Well the commercial pump industry has software on each of their pumps that control how outside devices, like remote controls or remote meters, can communicate with the pump.  It turns out that certain older model medtronic pumps have a software (aka firmware) that will allow the DIY community to access the commands of the pump.  That communication portal has been shut down in newer versions and does not exist in other brands of pumps to-date.  Therefore, it can’t be omnipod, animas, t-slim, or whatever other pump brand name or model you’ll ask about.  It has to be a certain Medtronic model of pump and a certain range of firmware versions.

Now, originally I thought the only DIY closed loop system was one called OpenAPS.  And my original impressions (in August 2016) was that the rig was clunky big and the setup was only for the type of people who were comfortable with linux commands (of which I am not).  So while I was thrilled to have this option available, I didn’t think it was actually going to fly with Anna.  She wasn’t going to want to carry a ton of gear.  She wouldn’t have the ability to troubleshoot linux commands at school if the system went down.

I searched for awhile looking for options that might make the OpenAPS “rig” (gear pack) smaller.   In searching Twitter, I found this photo of a boy starting school and his iPhone screen showed something I’d never seen before in all my #openaps searching.  This kid was holding something that if I zoomed in looked like BG graphs on an iPhone.

Researching it further, I discovered the iPhone app he was using is called Loop.  Loop is a DIY closed loop system, but it is not the same as OpenAPS. This opened up worlds to me.  Loop had a small form factor (not a lot to carry) and could be on an iPhone (what kid wouldn’t want that? And it wasn’t linux, which bode well for my ability to get it built potentially).

You know those instances where the world of innovation just seem to take leaps and bounds all at the same time?  Three days after the photo above was posted, Dana of OpenAPS tweeted a photo of the new smaller OpenAPS rig.  It’s called an explorer board/edison rig.  ACK…now I had some real useable choices that could be available for my teen t1d.

But, which one to choose?  Well, the answer was simple enough.  Building the Loop system was more immediately available.  The smaller OpenAPS rig’s parts were still not in production and therefore not available.  Plus…teen and iPhone.  That’s a slam dunk.

If you know me from online at all, you know I threw myself into Loop (and finding that elusive old medtronic pump).  Our RileyLink (communications part of the Loop system) arrived on September 21st and it took me about 2 hours to “build” our system.  We closed the loop that night and never looked back.

We’ve achieved amazing results on Loop.  The sleep at night has been fantastic.  The BG control unbelievable.  The independence for the teen t1d is glorious.  I spent less time texting her at school to tell her to adjust pump settings or give corrections.  Loop was like sending along a diabetes babysitter with my kid all day.  It just simply changed EVERYTHING.

On October 29th, I went to Nightscout’s Hackathon in San Francisco.  I just wanted to see the people who created the systems (Nightscout, pebble watch faces, Loop, OpenAPS) in person.  I wanted to throw flower petals at their feet and thank them for the tireless work that they’ve done.

I did all that, but also managed to walk away motivated to pay-it-forward however I could.  Since I can’t hack, solder, code, or program…I decided that I would work to make the online documentation for Loop more robust.  I threw myself into that effort over the next month(s).  The result was a lot of wiki pages and screenshots, but I think it was well worth the time.  I learned a WHOLE lot about the Loop system as a result, which definitely helped improve my Loop success.

We would likely have never tried a different closed loop system, except for two things.

  • Loop has some little quirks that people seemed to be struggling with, in particular with kids at school.
  • I couldn’t find a good comparative review of OpenAPS vs Loop from a PARENT perspective.

Since I love trying new things, on January 18th, I ordered the necessary gear for building an OpenAPS rig and put on my big girl pants ready to slog through linux.  The gear showed up on January 21st, two days earlier than expected.  And here’s the super unexpected part….

Within a few hours of delivery, I had a fully functional OpenAPS rig going.  It was “supposed” to be a lot harder than that.  I’d heard it was really tough to build those rigs.  And I don’t know linux.  What happened?  Turns out the instructions are quite good, just intimidating on the surface.

I had to pry the Loop app out of my kid’s hands.  She didn’t go super willingly into this experiment.  She doesn’t like changes to her diabetes management system.  But, she agreed to let me try this out so that I could learn first-hand what the practical differences were between the two systems.  ESPECIALLY from the perspective of a parent and t1d kid use scenario.

So now that leaves at the current day.  We’ve been running OpenAPS rig for about 10 days and I have some good idea now of the differences between the systems.  It’s time to get that comparison going.