Safety and risk analysis in a closed loop algorithm

As we started to begin our order of the Tandem Control-IQ system, the sales rep was touting “safety”. I thought to myself “yeah, yeah…safety. I don’t think my definition of safety is the same definition that a commercial pump company uses.”

My primary thought was biased by the 670G system.  A system designed to keep the person “safe” would instead leave many people significantly higher than their programmed target of 120 mg/dL for long periods of time.  A system that because of “safety” would require heaps of calibrations for the sensor, and getting kicked out of automode if the sensor got nervous. And that lovely “safety” of people setting dangerously aggressive (short) insulin durations of 2 hours and super strong carb ratios in order to trick the overly “safe” algorithm that left them higher than targets so consistently. Having such artificially aggressive settings in order to overcome a bad algorithm means you’re exposed to a pretty great risk if your loop stops working. So when I heard “safe” as a selling point, I was admittedly more apt to say “ummm, that might not be the strongest word to choose as a marketing choice.”

I totally agree that closed loops are more “safe” than just winging it on normal pump therapy. And the clinical data supports that statement…more time in range and fewer incidences of hypo/hyperglycemia in the clinical studies. But, the problem I had is that we already have been in a “safe(er)” zone using Loop (and OpenAPS) for years now. We have benefitted from closed loop technology’s “safety”. Did Tandem just mean “safer” because it was raising the BG targets compared to Loop, and therefore avoiding more “dangerous” lows? (That’s what I thought they might have meant…but the answer apparently isn’t that.)

But some things happened recently that got me pushed into researching if there was something more behind the rep’s use of the word “safe”. It seemed as though she was insistent on Control-IQ having something about “safety” beyond just simply trying to sell me on closed loop in general or BG targets alone, but rather Control-IQ algorithm in particular. I wasn’t quite grasping what she was meaning, and it wasn’t something she could explain in detail (not her fault…I have a pretty demanding threshold for the details of technology. It was a hard bar to meet for an average rep, so I’d need to do my own digging). Around this same time, I attended the 2020 ATTD conference on diabetes technologies as part of JDRF’s platform to discuss the Open Protocols Initiative. While there, I had the chance to have some really interesting, in-depth discussions about algorithm developments, pitfalls, and successes with researchers and clinicians who run trials for the devices/systems.

So I rolled up my sleeves and started researching. Here’s what I learned…and it was a lot.

The HISTORY OF CONTROL-IQ

I kind of worked backwards in my Control-IQ development research because I knew current info the most, but I’ll present this discussion in chronological order for clarity of information. The University of Virginia (UVA) and JDRF play a large role in the history of Control-IQ. So, let’s start there.

In December 2005, the FDA (along with NIH and JDRF) held an open public workshop titled “Obstacles and Opportunities on the Road to an Artificial Pancreas: Closing the Loop“. About 75-80 of the industry’s leading researchers, regulators, and clinicians were in attendance to discuss the critical path items needed to move closed loop technologies forward. FDA offered to review research proposals from JDRF to assist them in progressing closed loop technologies. Shortly after in 2006, JDRF announced the Artificial Pancreas Project and funded six centers to carry out closed loop research. Among the funding recipients was a group of interdisciplinary researchers from the UVA Schools of Medicine and Engineering and the University of Padova in Italy.

Things were really getting interesting by 2008. Modeling of insulin and blood glucose regulation in the human body was an area of a lot of active research. In 2008, the UVA-PADOVA partnership produced a Type 1 Diabetes Simulator to simulate insulin-glucose changes during a meal…in other words this simulator could “pretend” to be a person with T1D (actually, about 300 virtual T1D patient profiles were available in simulator).  This is loosely referred to as the “metabolic model”. The simulator was updated in 2013 based on newer data and modeling, which improved its accuracy. How complex is the metabolic modeling?  Check out this figure (source):

Modeling allowed researchers to test control algorithms on virtual patients (in silico) before ever moving to tests on real humans (in vivo).  The FDA accepted it as a substitute for animal trials, which advanced the work of the UVA team and the entire JDRF Artificial Pancreas Project.

As the pace of research progressed, things had miniaturized and improved. Algorithms and controls were now small enough to fit on an Android phone. In 2014, UVA had an in vivo nighttime-only closed loop interventional trial using its DiAs (Diabetes Assistant) system and the Unified Safety System (USS) Virginia algorithm…basically 42 participants spent 5 nights in a hotel using a closed loop system for overnight control only. After that trial’s success, they moved on with in-home trials for a longer period of time. The UVA algorithm was moving along well.

Meanwhile during this 2014/2015 timeframe, UVA licensed their algorithm to TypeZero Technologies. TypeZero Technologies began using “inControl” for their technology’s name, replacing the DiAs name…but very similar algorithm.

The International Diabetes Closed Loop (IDCL) Trial began in 2016 and was the pathway for Control-IQ’s eventual FDA approval. As outlined in the IDCL protocols, there would be three studies leading to the pivotal trial (in support of Control-IQ’s submittal to the FDA) using Tandem’s t:slim x2 pump, Dexcom G6 sensor, and the inControl algorithm:

The initial pilot study with Dexcom G6, Tandem t:slim x2 pump, and TypeZero’s inControl was completed in December 2017. This was a supervised 36 to 48-hour pilot study in 5 subjects conducted at the University of Virginia.

The Nightlight study was completed April 2019.

The last phase, a pivotal trial, started in June 2018, completed in April 2019, and had participants use the system at home for 6 months. By now the system was termed “Control-IQ” (see paragraph below for brief explanation of the name change from inControl). The results of this trial were submitted in August 2019 to the FDA for approval of the Control-IQ algorithm.

Around the same time as the pivotal trial began, Dexcom announced it had acquired TypeZero Technologies. This gave Dexcom closer access to the closed loop algorithm, which had already been a part of over 30 successful trials with more than 450 participants at that point, to go along with its G6 iCGM system which had been FDA-approved just months before. This acquisition also explains some of the name-changing between inControl and Control-IQ as the business relationships between all the parties were changing. From the pivotal trial’s protocol “The Tandem X2 insulin pump with Control-IQ Technology is a third-generation closed-loop control (CLC) system retaining the same control algorithm that was initially tested by UVA’s DiAs system and then implemented in the inControl system.” This gives us a pretty good idea that the algorithm wasn’t changing significantly even though the naming was evolving during this time.

The FDA approved the Control-IQ algorithm in December 2019, clearing the way for Tandem to start marketing and selling the Control-IQ system in early 2020. (Note: the pivotal trial originally did not include pediatric enrollments, but that trial is finishing up now for kiddos between 6-13 years old. Initial results presented at ATTD looked in-line with the adult trial results.)

So What’s Different?

Well, all that history didn’t get me any closer to understanding what would be “safer” other than Control-IQ had undergone clinical trials. Which since we’d run our own n=1 clinical trial with my daughter for the last 4 years…all of this research hadn’t much helped me understand the differences in algorithms that might explain a “safety” difference yet. Merely doing clinical trials, while great, wasn’t exactly enough of a sway for me.

And so now I started to read all the research papers that I could find about UVA’s (and other group’s) algorithms and models. I read about the underlying metabolic model. I read about algorithm development in general, and specific. I read about control systems. It’s been non-stop.

Ultimately, there has been a difference that I learned about…and it does actually make me feel “more safe” with Control-IQ than Loop/OpenAPS. But, to explain it I need to go backwards a bit again.

Loop’s Algorithm

Loop’s algorithm has 4 contributions; insulin effects, carb effects, retrospective correction, and blood glucose momentum. Those four effects are summed together to form the predicted BG curve. Loop then automates insulin delivery adjustments based on where that predicted curve is for the next 6 hours relative to your correction range and your suspend threshold.

An important piece to understand in Loop’s algorithm is this: The algorithm does not differentiate between a unit’s potential risk at a low/high blood sugar vs normal blood sugar. In other words, so long as you are above suspend threshold, there is no “risk” assessment of Loop’s actions that would change depending on where you are in the BG range. The two “safety” factors for low BGs below correction targets are:

1. The suspend threshold being the full-brakes on insulin delivery, and
2. A rising BG (predicted to stay above suspend threshold through DIA) will get scheduled basals while below the correction range; in other words, high temp basals will not enact until the BG crosses above the bottom of your correction range.

This means at times when you are below targets, but carrying negative IOB because you have your basals scheduled too high, you’ll get a pattern of suspends alternating with scheduled basals.  If you cross the correction range’s minimum value, then you may even get some strong high temp basals to try to “replace” the negative IOB that has lead to a high predicted BG now.  That negative IOB can be a pesky term in Loop if it accumulates significantly while you are in lower BG range. That’s one reason why settings are so important in Loop. Making basals artificially high in the hopes of making Loop more aggressive will backfire with a pattern of lows and lots of negative IOB to be covered later when the low is treated.

RiSK terms in Algorithms

BIG DISCLAIMER HERE: All the commercial closed loop systems are black-box…meaning we don’t really know for sure what all their controls systems are comprised of. I am only working from the published literature and available research. So, while the papers describe the research that lead to the systems…they do not necessarily represent the final versions that end up in a commercial system. 

Much of my recent research has revealed that most algorithms, including the work by UVA’s published papers, include a component of risk-based dosing in their strategies.

What do I mean by that? Well, let’s use an example where you have the time and inclination to be a helicopter parent. If your kid’s BG was 100 mg/dL but headed down…you may choose to conservatively set a smaller temp basal of about 80% to help head off that low that appears to be coming. If your kid was 60 mg/dL, you might be more aggressive in your treatment decisions. You might set a temp basal, but also treat with some small carbs. That situational awareness is because you’re evaluating risk…and you are recognizing the inherent risk from low blood sugar events grows as the lower the blood sugar gets. Similar for increasing blood sugars getting into a high BG range. You would probably choose more aggressive treatments as you see higher blood sugars because you are perceiving a greater risk in those BG ranges. We have all rage-bolused…our instinctive desire to mitigate the high bg risk? There’s a sweet spot in your life, probably near 100 mg/dL, where you feel little risk from maintaining that BG.

This risk concept has been written in many papers about closed loop and diabetes (such as here, here, here, and here.) The concept can be visualized as shown in these graphs:

 

(Interesting side note: Notice where the low point of the risk curve is in that first graph? That’s where Control-IQ likely gets its 112.5-120 mg/dL nighttime targets…you’re in the lowest risk area during a time when the biggest variables, like exercise and food, aren’t in play. The same figure can be found in UVA paper here.)

So what does this mean in terms of making an algorithm “safer”? Well, if your algorithm incorporates some “situational awareness” as part of its dosing strategy and risk mitigation, then you will more effectively have a way of mitigating that risk. While you’re between 70-90 mg/dL, you should perhaps make more conservative decisions that reflect your BGs are in an area of greater risk. Same for if you are between 200-300 mg/dL…you should have an algorithm recognizing the risk of remaining that high is undesirable and it might be time to be more aggressive.

Control-IQ’s algorithm, while I haven’t seen the inner guts of its details for sure…I have seen the research papers on UVA’s algorithm development and watched presentations at conference discussing risk mitigations. So, I feel pretty sure that Control-IQ has risk awareness incorporated into the algorithm. (Disclaimers still apply)

This risk mitigation makes sense to me in a very gut-feeling way. Let’s look at one example where I feel like risk awareness may/could play a role…when Anna has overnight basals that are scheduled too high. What happens in Loop with that situation? Loop will suspend when the BG prediction goes below her suspend threshold (we had that set at 75 mg/dL). So, she’d get a BG dropping for awhile, hit the suspend threshold and then often times tip-toe touch her low BG alarm of 65 mg/dL.  We would treat the low conservatively…but even that action would usually just bring her (prediction curve) up enough so that scheduled basals would turn back on. She would have a lot of negative IOB built up from all the previous suspended basals…her prediction would say she would be going high, but instead the scheduled basals being on would soon send her low again. We’d be stuck in that repeat cycle of off/on scheduled basals with small treatments.

It would be nice in those situations if Loop had some situational awareness that we were in the low BG area of the risk curve…that maybe this would be a time to not just treat all my settings as the “gold standard” of math to work from. Perhaps if there was something that made the algorithm instead say “hey, I see all that negative IOB and yeah yeah…but you’re still low now and have been. Let’s ease back into this until the issue has really passed.” In other words, not jump right back to scheduled basals. And really please don’t jump to aggressive high temps if we treated a little more…enough to cross over the correction range minimum.

Visually what does this situation look like? Here’s an example shown in the orange box below. That represents about 9 hours of overnight BGs for Anna. Her scheduled basals were clearly too high (exercise before bed? Hormones changed? I don’t remember the cause that particular night, but this happened about 2-3 times a month usually). She kept getting BGs pulled back down after we treated lows with minimal carbs. The negative IOB was the source of crushing high temps if we treated enough to bring BGs above the correction range minimum (early in the orange box). And if we just let things play out, as later in that orange box, then scheduled basals would resume and stay on…eventually she still went low again because the scheduled basals were just too much.

Now, let’s compare with how Control-IQ works with its logic in a very similar situation. Here’s a graph, below, from a night recently. Anna was using a transmitter that had very low battery and we had gotten signal loss during the times in the red boxes. You can see the system, in those situations, would revert to her scheduled basal and she would have a small low afterwards. Not too surprising. BUT, look at what happens when Control-IQ turns on in the blue box. Notice how it does not go to scheduled basals despite having negative IOB accumulated? Instead, Control-IQ uses a fraction of the scheduled basal rate. Control-IQ is likely adjusting its insulin delivery not just based on a correct-to-range adjustment alone…there’s likely also a component of situational awareness about the low BG range it is in. We aren’t getting full suspend (not predicted to go below 70 mg/dL in next 30 minutes) but also are not automatically dropped off at scheduled basals like we would be in Loop. We coast along at a lower-than-scheduled basal rate.

This night above was actually the impetus for me to start researching Control-IQ’s algorithm possibilities. We had gone nearly a whole month without having one of those nights where her basals were peskily too high waking us up with nagging repeat low alarms. Usually we’d have a night or two like that during hormone shifts each month on Loop. I’d deal with it by finally recognizing the pattern was sticking around (usually after two low treatments wouldn’t fix it), and I’d edit her basal schedule lower. When this happened to us on Control-IQ that night, it kind of stood out like a sore thumb since we hadn’t had it happen in nearly a month. I downloaded the pump data to try to figure out what the difference was for that night and that led me to do the digging that I’ve tried to summarize in this post. Now, this lower-than-scheduled-basals Control-IQ behavior is not necessarily entirely the result of the risk component of an algorithm…there may be other things at play, too. But I am quite happy with the results in the end, no matter whatever combination of effects lead to it.

So…UVA’s algorithm likely has this insulin adjustment modified by a risk component. Low and high BG risks would yield appropriately modified insulin adjustments to mitigate that risk. Interesting, right? You can read this paper and the associated equations presented for this idea:

I am by no means an expert on any of this. And certainly, nobody has shared the actual equations/algorithm in Control-IQ or UVA in particular. But, I have noticed that our low and high extremes seem to be better managed (restored more quickly to targets) on Control-IQ…I’m still trying to explore why and quantify it. It’s the same kid, in the same body, with the same habits…so I feel pretty confident that our experience is at least in some part simply the difference in algorithms. The risk component that I believe is in Control-IQ in some fashion seems like a reasonable cause.

In short…I think now I can finally say that I am more appreciative of the marketing word “safe” and what it probably means for Control-IQ. I can see that whatever they are doing has resulted in some improvement in BG results for us over Loop…which had us quite happy for years before. In particular, we really notice it avoiding lows for those accidentally too-high-of-scheduled-basals times at night. We are happy to see Control-IQ choose lower basals rather than off/on scheduled basals in those situations. That is a safety measure.