Note: This post is outdated because the features of Loop and OpenAPS have both been updated since this was originally published…The link to the updated post will be posted here shortly.
How do you choose the “best” DIY closed-loop system? The answer is still a YDMV (your diabetes may vary) answer…but here’s some thoughts from a parent’s perspective.
I have a 14-year-old t1d daughter. She is relatively independent and can count her own carbs fairly well. After a little practice and gentle reminders, she can usually learn new things about her t1d management. BUT, she’s not an independent adult. She’s prone to forgetting things in the presence of her peers, getting distracted in busy environments, and not advocating for herself in the presence of authority figures. She worries about fiddling with diabetes gear in class (“will the teacher think I’m playing on my phone if I need to text about diabetes?”). So, as she goes out in the world with a system making automated decisions about insulin on her behalf, I have a different set of considerations about a system than an adult t1d closed-looper might. (I’m going to try to keep the little kiddos in mind too as I walk through the differences between these two DIY closed loop systems; Loop and OpenAPS.)
In any comparison, it’s a good idea to know the person’s assumptions and priorities when they are doing the comparison. Here’s my short list of the non-algorithm issues I considered in evaluating Loop life vs OpenAPS life.
- Size
- Cost
- Durability
- Ease of build
- Ease for caregivers/school nurses
- Troubleshooting
- Reliability
- Portability
- Ease of use
- Kid-specific features
I intentionally left BG control off the list is because either system is capable of achieving great BG results. BUT, the ease/difficulty in the WAY you have to adjust your thinking, actions, or lifestyle to achieve the desired control was important to me. And I’ll get to that shortly.
Size, Cost, and Durability (Advantage: Even)
These factors are virtually the same between the systems. Both systems can be easily put in a box about the size of a tic-tac box. The Loop system costs about $99 annually (Apple developer program), Heroku (for Nightscout) is free, and the RileyLink costs $135. OpenAPS costs about $163 for the parts, and there are potentially some data costs depending on how you set up your system. If you want to run multiple rigs, you may need to upgrade to the $7 per month Heroku “hobby” plan (for one rig you should be safe under the free Heroku plan still).
Both systems consist of a small circuit board and a rechargeable lipo battery…so the same care and durability should be expected between the systems. We plug both systems in for recharge nightly and they last the whole day on the one charge.
OpenAPS Rig on the left, RileyLink on the right
Ease of build (Advantage Loop)
I had both systems built and operational within about 2 hours of getting all the parts together. Am I some sort of mad genius with computers? Absolutely not. You can do this, too. There are guides for both systems that tell you what to do and the order to do it. Both systems have very helpful groups on Gitter where people will help you through the setup process…Loop and OpenAPS.
That said, Loop can initially seem like a more relaxing build for the people who are uncomfortable with “code” since much of it is done through the Apple interface (think, pictures and symbols). OpenAPS is done through linux commands and scripts. HOWEVER, do not let this deter you. Both systems will take you into places on your computer you have likely never been before (Loop to Xcode, OpenAPS to Terminal), and you will live to tell the story. I am actively involved in providing some updates to the OpenAPS docs that will hopefully help new, non-code people through the setup process…just as easily as Loop can be done. To be honest, setting up an Apple developer account was probably the most frustrating part of the Loop setup and putting two tiny screws in the explorer board was probably the most frustrating part of the OpenAPS setup. Notice, neither complaint is actually about the software building.
I’m here to say…don’t let your intimidation stop you from trying either or both systems. (Perhaps slight advantage to Loop here…but the gap is closing.)
Ease for Caregivers/Small Kids (Advantage OpenAPS)
Here’s one that adult t1ds probably don’t have to worry about. Can a school staff member reliably dose your kid with insulin through the closed loop system? Here is where I was surprised to see a large difference between the systems potentially. Anna has operated her Loop inputs fairly independently and has gotten used to some of the quirks, so in many ways she’s more like an adult user. However, for little kids relying on school staff for meal bolusing the differences between systems may be more important. Loop uses the app to enter meal carbs and calculate/administer boluses; whereas OpenAPS uses the pump. OpenAPS likely has an advantage for these considerations:
- Failed boluses – Every once in awhile, for whatever reason, a bolus command in the Loop app will not go through. A notification pops up on the iPhone saying “Bolus may have Failed”. If the caregiver/kid doesn’t notice the notification, it may be some time before the missed bolus is recognized. And unfortunately, sometimes, the bolus actually will begin shortly after the notification pops up. The caregiver/kid will have to be trust-worthy enough to recognize the need to look for the message and if the message appears, to first check the pump to ensure the bolus is not actually already in progress before attempting to re-enter the bolus command. Since OpenAPS is bolusing directly from the pump, failed boluses would be immediately obvious to a caregiver/kid. (Advantage OpenAPS)
- Bolus wizard guard – The default setting for Loop is that the app will not suggest a bolus if the t1d is below, or predicted to go below, their minimum BG target. This feature, called the bolus wizard guard, can result in the carbs for a meal being entered but no bolus being offered. The bolus wizard guard can be removed or modified by parents, but it is definitely something to be aware of. Many new Loop users fail to modify this feature and find no bolus being given. Parents should have a plan in place with the caregivers in the event no bolus is offered by the app. OpenAPS does not have an analogous bolus wizard guard. (Advantage OpenAPS)
- Double Carb Entry – When a carb entry is entered in the Loop app, there is a “save” button to move you onto the bolus recommendation part (just like a pump’s bolus wizard). In both Loop and OpenAPS, an entered carb is saved by default. Backing out of a bolus wizard does not cancel the carb entry. In Loop, the caregiver/kid can go into the app and cancel/edit the unwanted carb entry. However, in OpenAPS you cannot edit the entry. So both systems require a consistent method of carb entry to prevent unintentional double entries. (Advantage Loop)
- Yellow/Red loop – Loop will not bolus if it is having trouble communicating with the pump or receiving BG readings. The Loop will turn yellow or red, depending on how long it has had problems with communications. If a caregiver/kid doesn’t resolve the issue, they can still enter a bolus through the pump directly as a backup. However, the caregiver/kid will need to separately enter the carbs into the app (Loop does not read the pump’s carb entries). Since OpenAPS boluses are directly from the pump, meal boluses can continue as normal even in the event the looping is not functional at the time of the meal. No special considerations are needed. (Advantage OpenAPS)
- Carb Absorption Time – Loop has a unique feature called Carb Absorption Time for each carb entry. Basically, it is akin to saying how fast or slow Loop should expect the food to absorb (and thus affect BG levels). If you have new food or classroom treats, your caregiver/kid will need to estimate the carb abs time of the food. Loop can tolerate some inaccuracy in those estimates, but if you are off by significant amount (like 60 minutes vs 120 minutes estimate) you may end up under- or over-corrected by Loop as the meal progresses. Also, it’s important that your caregiver/kid respect the importance of entering in the correct carb absorption time for a meal…otherwise unwanted BG impacts can occur. In OpenAPS, the absorption of carbs is tracked dynamically by the loop. Therefore, the caregiver/kid does not need to estimate a carb absorption time as part of their mealtime entries. (Advantage OpenAPS)
- Emergency tube changes – If you child needs an emergency tubing change at school, there is a special consideration for Loop users to be aware of. Priming new tubing, without a change of reservoir, will require the caregiver/kid to delete 30 minutes of reservoir history in order to not have the primed insulin count as delivered insulin (IOB). Parents and adult t1d users of Loop get pretty used to this action, but trying to describe this step over the phone with school staff or young kids could get pretty difficult. OpenAPS reads the pump’s actions and will not count primed insulin as IOB. A notation is also made on the Nightscount site automatically when a prime event occurs. (Advantage OpenAPS)
- Net IOB – The Loop app tracks net IOB (IOB from both boluses and basals). As such every Loop bolus and correction is automatically consider net IOB. However, Medtronic pumps do not track IOB from temp basals. In OpenAPS, since there is no app, netIOB is usually seen in Nightscout (in fact, the IOB pill will even break down the basal IOB contribution). Caregivers/kids will need to be aware that the usual “take BG before a meal and add a correction adjustment to meal bolus using pump’s bolus wizard” should not happen. Most OpenAPS users will only bolus for their meals using straight carb ratio, and not use the pump to suggest a +/- correction bolus. (Advantage Loop)
- Corrections – BG momentum in Loop has proved to be a difficult concept for my teen to watch for in Loop. In times of rapidly increasing, BG momentum can tend to over-predict and thus offer too strong a bolus correction. This can also affect meal boluses in Loop. OpenAPS boluses using straight carb ratios seem slightly safer for small kids under remote care. Plus the bolus snooze option helps corrections take effect without having the loop go to low-temping when you might be stuck on a stubborn high for whatever reason. (Advantage OpenAPS)
Troubleshooting (Advantage even)
Loop relies on the RileyLink for communications, OpenAPS relies on the explorer board. Both feed information into Nightscout about the status of those communications and looping. A remote parent can receive information about potential problems that may be causing the loop to stop running.
For both systems, the solution to getting looping started again is usually pretty straight-forward. For Loop, it’s almost always a retune. For OpenAPS, a power button push. However, advantage goes slightly to OpenAPS because a power button push is a little faster and doesn’t require opening an app. And if your kid isn’t capable of retuning the RileyLink themselves, explaining how to retune a RileyLink can be a little cumbersome over text messages or phone calls with school staff. A power button is a more straight-forward explanation.
If a fatal program error happens on either system, they can both be rebuilt fairly quickly and easily following the original steps.
Reliability (Advantage OpenAPS)
How often does each system go down? How far can your kid get from the rig before communications/looping quit? The answer is “it depends”. If you live in a “noisy” radio frequency environment, you may find yourself needing to retune RileyLink fairly often. If you live in an area with no wifi or cell service, you will likely have a poor OpenAPS experience*.
But, the advantage here goes to OpenAPS’s rig because it’s communications has a longer reach and seem more stable. Kids don’t have to be quite as close to their rig as they do to the RileyLink. On average, I’d estimate we retuned the RileyLink about 2-3 times per day. On OpenAPS, we have had to power cycle the rig only once in 12 days (and I think that actually was just my impatience rather than truly a problem).
Portability (Advantage even)
The portability of Loop is pretty great. You could take it to the middle of nowhere and still be able to run a closed-loop…no internet, wifi or cell needed. HOWEVER, I’m a remotely-monitoring parent and I like to check Nightscout to see how things are going. So, even Loop needs internet connection for the parent to monitor things. (If I were reviewing portability as an adult t1d, Loop would be an easy advantage.)
The newest release of OpenAPS also has made the system truly “portable” in my eyes. Before this update, kids might have to remember to turn on their phone hotspots when leaving a known wifi network. That’s not practical for kids. But, the new release is a very stable handoff between known wifi networks and iPhone hotspot. So, as your kid leaves home to go to school, the rig will automatically connect with the kid’s iPhone via bluetooth hotspot tether. When your kid gets to school, it can pick up on the school’s wifi automatically…thus saving you on cellular data and hassle.
*There are options for running OpenAPS without internet like Loop (aka “camping mode”). I’m just not setup like that. You could directly plug-in the dexcom receiver to the rig. Or you could use X-drip to upload locally to the rig.
For summer diabetes camp in the woods…we will likely use Loop. I think. Maybe.
Ease of Use (Advantage OpenAPS)
Oooph. This one is tough. The iPhone screen is soooo sexy. Kids LOVE the iPhone. And they love not bolusing from the pump sometimes. Since the iPhone is already glued to her hands, Loop gets a lot of love from my 14 year old.
But, the ease of entering a meal bolus from OpenAPS is just heaps easier than Loop. Kids (heck, even adults) can have a hard time estimating what a proper carb absorption time for a food should be.
There’s also a really sweet feature that you can find in OpenAPS that you can’t get in Loop. As a parent, you can remotely set a temp BG target using Nightscout’s careportal. When my daughter goes to track practice, I can set a temp BG target ahead of time to help keep her in a slightly higher target range. I can also easily cancel the temp target if practice gets out early or situations change.
I love both systems…but I do think the OpenAPS is easier to “use” if you are a parent sending your kid to a slumber party or school dance where they will have new foods and perhaps harder bolusing than they might be used to under your care. Also nice to have some ability to remotely help nudge loop behavior “up or down” through the use of temp BG targets. It definitely has saved me quite a few text messages as a result.
Kid-specific features (Advantage OpenAPS)
There are some feature that OpenAPS specifically designed to adapt for kid t1ds. We haven’t used or tested these, but they may be of interest to some of you reading here.
- Override High Target with Low: “Defaults to false, but can be turned on if you have a situation where you want someone (a school caregiver, for example) to use the bolus wizard for meal boluses. If set to “True”, then the bolus wizard will calculate boluses with the high end of the BG target, but OpenAPS will target the low end of that range. So if you have a target range of 100-120; and set this to true; bolus wizard will adjust to 120 and the loop will target 100. If you have this on, you probably also want a wide range target, rather than a narrow (i.e. 100-100) target.”
- Carb Ratio Adjustment: “This is another safety setting that may be useful for those with secondary caregivers who aren’t dedicated to looking up net IOB and being aware of the status of the closed loop system. The default is 1 (i.e. do not adjust the carb ratio; off). However, in the secondary caregiver situation you may want to set a higher carb ratio to reduce the size of a manual bolus given at any time. With this ratio set to 1.1, for example, the loop would multiple the carb inputs by 10%, and use that number to calculate additional insulin. This can also be used by OpenAPS users who rely on the bolus wizard to calculate their meal bolus, but who want to only bolus for a fraction of the meal, and allow advanced meal assist to high-temp for the rest.”
Conclusions (Advantage OpenAPS)
I feel like I’m cheating on my best friend Loop by saying that I’ve really enjoyed OpenAPS. All the technical stuff above spelled out…here’s the short version of the parts I’ve noticed the most.
Fewer texts to Anna during school: Through the use of temp BG targets, I don’t have to text her to remind her to prepare the two hours ahead for track workout. I don’t text her anymore to ask her to retune the Loop. I don’t get texts from her asking for carb absorption times for food from classroom parties. I spend less time asking her if she got too far from her RileyLink.
Easier mental at meal time: I have enjoyed not thinking about carb absorption times. Now I’m back to my regular thoughts about whether a meal would do well to split bolus…and that thought is the same with both systems regardless. (Anna says this part never really bothered her.)
OpenAPS pill: I get a little more information from the OpenAPS pill. I have enjoyed that. I also enjoy seeing the Basal IOB using the Nightscout IOB pill.
Auto-tune: I haven’t really touched on it here much…but there’s a newly unveiled feature called auto-tune which looks back on the previous day’s data and can adjust ISF and basals automatically where it sees the values may be slightly off. I think that needs a special post of it’s own.
Loop enacted: I have REALLY enjoyed never seeing that the loop is stalled out or needs retuning when I open the Nightscout app. I have pretty much gotten used to seeing the OpenAPS rig always functioning.
Easier to switch between patterns: Anna has definite differences between school days and non-school days. On school days, she needs about 6 hours of a lower basal rate during the day and through track practice. Switching between patterns on the pump has been just a little easier than switching between the two different Loop apps that I had setup. It’s just one little less thing to do.
If I were an adult t1d, my metrics would probably weigh the various considerations quite differently. But as a parent of a t1d, I’ve really enjoyed OpenAPS. We have great BG control on both systems…I’m just trying to find ways that we talk less about diabetes and more about everything else.
Setting temp BG targets to help adjust to changing situations like exercise or upcoming meal times is super easy using the care portal in Nightscout. The site also will provide a grey line showing the range and duration of the temp BG target. You can cancel the temp target and the line will end at the time the temp target was cancelled. It leaves the grey bar on the graph so that you can scroll back in time to see if your actions might need modifying later. For example, we are currently using temp targets to get our daughter to her after school track team workouts without carrying extra IOB. We set a temp target at lunch time, 3 hours ahead of the start of her workout for 120-140 mg/dl. As her lunch bolus wears off, the temp target helps keep loop from giving too much extra insulin between lunch and track workout. We keep the temp target going all the way through her 2.5 hours of track workout.
Some interesting things that you can learn about your kid’s habits and the rig’s behaviors by watching your papertrail. My kid took a shower at about 6:55am today…she leaves her pump in the bathroom, fairly removed from her rig in the bedroom. You can see the distance has caused pump comms to deteriorate to a “916, 1, -98” tune result. When she finally gets dressed and puts her stuff all back together, around 7:25am, the pump tune picks up a nice “916, 5, -79″…and that was with the rig stuff in her purse, stuffed in her backpack. Pretty good. She arrives at school and the next pump comms failure is at about 11:39am with “916, 0, -99” just as she is finishing a PE change before lunch. The loop picks back up and retunes by 11:43pm with “916, 5, -75”.
For Anna’s morning, you can’t see it, but she got in the car at 7:35am to go to school and left our home wifi. Her rig got onto her cell phone via BT connection by 7:43am. She arrived at school and the rig got on the school’s wifi network at 7:54am. How can you see all this…well the wifi networks have their name listed (like “Paso Schools”). The cell phone BT connections for iPhones have local IP addresses that always start with 172.20.10.xx. Papertrail has been so useful for figuring out if the rig is actually online while she is remotely moving around. Especially useful for little kids who might not be the most helpful troubleshooters when you are trying to decide what network the rig is on. So, if I see her loop is failing (yet has good pump communications), the next thing I do is usually to see if there might be a problem with wifi connections.