**Session Date/Time:** 21 Jul 2026 12:00

[00:03:22] **Eric Kinnear**: Okay. Let's get started. Thank you, Starry. Alright. Welcome to the CCWG meeting at IETF 126. This meeting is being recorded. Make sure that you've joined the MeetEco via the QR codes that are on the microphones, by the door, linked in the ITF agenda. Make sure that you've actually joined that. That's part of how we keep track of who is here. It's also how you're going to enter the queue when you want to say something. This is the IETF note well. Please take special note to read it. Make sure that you pay extra close attention to your IPR obligations as somebody who's participating in an IETF event. This working group also operates underneath a code of conduct. So if you observe any code of conduct violations, you can reach out to the chairs, the ADs, or the ombuds team. These are some helpful links for the PDF that you can click on later. And this is our agenda for today. So we're gonna kick off with some hackathon updates, and then we'll talk about our working group documents, and then we've got a couple other proposals to talk through with some time for discussion for each one. Before we do that, would anybody like to bash the agenda? And would anybody like to volunteer to be a notetaker? It's pretty easy these days given our amount of AI generated summaries and transcripts and recordings. Yes. Thank you very much, Stuart. Alright. First up, we have Mohit. Let's talk about some hackathon updates.

[00:05:24] **Christian Huitema**: You'll have a clicker in a second.

[00:05:26] **Mohit P. Tahiliani**: Thanks, Eric. Hello, everyone. I'm Mohit. And with me, I have one of my students here in person, Abiode. We had another student join us remotely, we participated in the hackathon over this weekend. So we are here just to give a small update on what was the work that we were able to do it. So initially, we had a couple more objectives, but then the time frame of the hackathon was shortened. So we also revised our objectives. We had three objectives out of which two of them were to evaluate FQ-CoDel and FQ-PIE in different scenarios, Wi Fi scenarios, one with a mobile hotspot and one with the Wi Fi access point. And besides that, we before we came for this ITF meeting, we actually completed the integration of picoquic library with ns-3, but there wasn't sufficient time for us to do all the necessary testing. So during this weekend, we spent time on doing different tests of the integration that we have done with picoquic and ns-3. And happy to share that in the time frame that we had, we were able to achieve all these three objectives. I'll invite Abhishuddhay to give you a detailed update on each one of these object, and then I'll come back and tell also what we were able to achieve after the last ITF meeting that we had done.

[00:06:54] **Abhishuddhay**: So hi, everyone. Very good afternoon. I'm Abhishuddhay, an undergrad at NITK Surathkal, and this is my first time at IETF. So a little update on the hackathon starting with the performance evaluation of FQ-PIE and FQ-CoDel. So we did the performance evaluation on two separate devices. The first one is gonna be a mobile hotspot. So as you can see, the topology over there, we had a client, which in this case is my a laptop that you can see, the silver color one. Now that was connected to the mobile hotspot, and the mobile hotspot was in turn connected to the ITF access point, which is connected to the ITF backbone. And now that is finally gonna be connected to the Netperf EU bufferbloat server. Now what tests did we run? We ran a couple of Flent tests that included RRUL and TCP and down tests for a different number of streams. And what did we notice? So these are a couple of graphs that we were able to plot before that before that. My bad. So as you can see, the FQ-PIE and FQ-CoDel algorithms were put in the egress queues of the mobile hotspots. So the mobile has two two separate interfaces, one for the hotspot and one for connecting it to the access point. So in both of these, we tested with different algorithms. And yeah. So these were the plots some of the plots that we obtained of RRUL and the two hundred and hundred TCP download ones. Next. So this the WNDR3800 is an access point, and this runs on OpenWrt, which already has FQ-PIE support. Again, the client, which encased my laptop, was connected to this, which had FQ-PIE and FQodal running separately, and then it was connected to the PoE switch of ITF. And these were, again, a couple of graphs that we plotted. So what did we infer from these? One thing that we consistently noticed from these was that the tail latency of FQ-PIE is better than that of FQ-CoDel. Now the orange line that you can see over there is that of FQ-PIE, and the blue line is of FQ-CoDel. So if you notice the graphs, the orange line ends pretty early compared to the blue one. So that is one inference that we made. But, yeah, we have to run a couple of more iterations to conclude on that. One more thing that we observed was that, if you can see on the left bottom corner, you have the box plots. Now, as you can see, the jitter of FQ-PIE, the one on the right, is comparatively lesser than the FQ-CoDel one on the left. But still, if you look at the median one, there they perform almost very similar to one another. So you have to spend more time on that and look into how it differs. Yes. The other thing is the integration of picoquic with NS3. So we observed that NS3 doesn't have any QUIC topology simulator in it. So for that, instead of implementing QUIC from scratch or something, we took picoquic, which is a well known c implementation of QUIC by Christian Huitema, and we built wrappers on top of that so that network researchers would be able to simulate QUIC topologies using ns-3. Yep. So these were some of the things that you that we did. The QR code over there can be scanned to access the source code of it, and we'd be happy to receive feedback on it. Yeah. Thank you.

[00:10:40] **Mohit P. Tahiliani**: Thanks, Javier. So this this work of integrating picoquic and s three comes out of the fact that in the last couple of ITF meetings when we were working on the hackathon and interacting with members from CCWG, we saw there was a lot of interest in simulating QUIC on n s three, particularly for congestion control algorithm evaluation. But like said, n s three does not have a native model of QUIC. And to build something natively in n s three will take a lot of time and we already have so many nice QUIC libraries available out openly. So we picked up just to get started with picoquic and we integrated it successfully. There are a bunch of examples that we have added. One another thing is that picoquic also supports multipath quick. So we already added an example for multipath quick simulations in ns-3. And the snapshot that you see here is just to show an output of one of the examples where we have one congestion control algorithm as C4 and another as cubic. These are only some initial steps that we have provided, but the code is openly available and it can be modified for your own use cases or for the congestion control algorithms that picoquic supports. picoquic has a bunch of congestion control algorithms. Apart from the work that we did at hackathon, I'd just like to also brief about a few things that are related to CCWG. These are the things that we got it done after the ITF one twenty five. So we submitted a patch to free BSD to align a small bit of things of PRR with nine nine three seven. And I would like to thank Richard and Michael for merging that PR that just got merged on the day of the hackathon. We had submitted it couple of months back, and this was the first contribution from our team to free BSD. So it was a great experience. We also built two models. One model for Ledbat plus plus and another model for Rledbat, and the merge requests are already open on the n s three mainline repo. Ledback plus plus has already received a lot of review. It should be shown getting merged in about couple of weeks. We still need feedback on our Ledback. Would like to encourage the community to take a look at the code if you are working on those algorithms and would like to give us some feedback. We also did one another interesting thing after the last IDF. Flint is a very popular tool to do the buffer blood test. But if you would like to repeat those tests with ns-3, you do not have that supported. So we have built a native FLINT like application in ns-3, which will easily reproduce whatever you do on a real FLINT tool. You can exactly simulate that in ns-3 with the same RRUL specification and same TCP upload and download specification. Finally, we also have some new topology helpers in our nest tool that will allow you to easily spin off the topologies for congestion control evaluation. That's it from our side, and thank you for this opportunity to share the updates. Any feedback or any input or any questions?

[00:13:55] **Rafael**: Rafael, University of Aberdeen. Thank you so much for putting PICO QUIC into S3. I think it's very important achievement. So I was thinking like, okay, I'm a PICO QUIC is going to evolve. Quick is going to evolve. There is going to change. What how difficulties to maintain? How difficulties if a new version of picoquic come up to introduce them?

[00:14:18] **Mohit P. Tahiliani**: Right. So we have ensured that the picoquic Evolution and ns-3 Evolution will not interfere with each other. The wrappers are pretty independent of both the things. So unless there is a significant update in either of those tools, you don't really need to worry about its maintenance. So the wrapper is very thin wrapper that we have built. It's a very tiny little code that sits into the contrib directory of n s three. And both can evolve independently and still be able to operate with each other. As I said, unless picoquic decides to change its APIs in a significant manner, we don't need to worry about the maintenance part of it. Do you want to add something?

[00:14:55] **Abhishuddhay**: Yeah. Yeah. So only if the certain API call if the API signatures are changed, then we have to do some change in our part of the module. Else, if there is any internal change in picoquic, then it's gonna be all good.

[00:15:15] **Alessandro Ghedini**: Alessandro Ghedini, Cloudflare. Thank you for sharing this. If you could just briefly, like, what's the effort required to, say, add new implementation to, like, alternative to picoquic to to NS3? Is it just a matter of, like, implementing the same sort of modules with, like, whatever API someone wants? Or do you need, like, a a natural, like, command line sort of tool to do the

[00:15:46] **Mohit P. Tahiliani**: So this wrapper that we have built with, what we did is after we did the picoquic integration, we also were interested to do the MsQuic integration. But the way MsQuic is built is I think it uses it's it's a multi threaded library as opposed to picoquic, which is a single threaded library. And there were some challenges when we were trying to integrate it. But this effort took us quite some time. It took us about two months, but it shouldn't take a lot of effort if the structures of the quick libraries are almost similar. But it would largely depend on what's the quick library doing internally.

[00:16:20] **Alessandro Ghedini**: So so does does does it require the library to do all the IO sending packets and receiving packets, or can it just, like, pass the data?

[00:16:29] **Mohit P. Tahiliani**: Just pass the data. Okay. Yeah. Thank you. Alright. Thank you. Yeah. Thank you.

[00:16:38] **Eric Kinnear**: Thank you very much. It is delightful to see QUIC integrated in ns-3 here. And I think one of the things that we talked about a lot when we were first starting CCWG was how can we as a community help move things forward. And this is a huge part of that. Extra thank you. Next up, we have a very small number of slides from Greg to talk about another hackathon thing. And then we will hop on with the rest of our agenda.

[00:17:01] **Greg White**: Great. Yes. So I did a quick version of this presentation earlier today in TSVWG. These are the same slides. We did have another iteration of the L4S interop at the hackathon this past weekend. This has been a great opportunity for congestion control developers to test their implementations both with, in some cases in the past, real network bottlenecks as well as in more recent iterations here at ITF using a Linux box as a bottleneck. And also testing their implementations, sharing that bottleneck with other congestion control implementations. This time, again, it a it a shortened hackathon this time. The link to the summary slides is there, so feel free to take a look at some of the results we got. Highlights would be continued testing of the SCReAM integration with LibwebRTC, continued testing of the Netflix NDTC congestion controller, some testing with the Apple responsiveness tool and its L4S mode. And we had intentions to test an alternative network configuration for L4S support called SRM, which is the draft being considered in TSDWg, And also to test the latest version of iPerf2, which has a nice Android client that can be used to test different L4S functionalities. If anyone's interested in participating in a future interop, be it the next IETF in San Francisco. Also, we host a series of interop events at CableLabs with cable network equipment and OLTs, etcetera, that support L4S. So let me know if you have questions. Thanks.

[00:18:43] **Mirja Kühlewind**: Thank you very much. Alright.

[00:18:47] **Eric Kinnear**: Thank you, Greg. And next up, we have our first working group document to talk about, Ian. Let's talk about BBR.

[00:19:00] **Ian Swett**: Great. Thank you. So I will go through some of the changes that have happened since we last discussed, then we could talk about next steps afterwards. You've probably all seen these before, but, you know, there's some links to various implementations and draft on GitHub. So there have been some notable changes in the most recent draft. A lot of them are just kind of, you know, referencing things like accuracy and accuracy of time stamps as requested just to understand, like, how those are relevant to BBR potentially. There's some kind of missing variables and other kind of, like, efforts to make the pseudocode match up with the actual, like, text. In some cases, there was normative text that was missing. Not a major issue in the scheme of things. You could kinda figure out what was going on, but it was definitely less explicit than it could be. Neil did a fair amount of work trying to improve the undo definitions. So if you are particularly in TCP, some of the undo logic is fairly complicated. So if you were implementing this in TCP in particular, take a look. And I have assertions from Neil after asking him about this that indeed fixing this and making sure all the undo logic is correct does actually create real world performance improvements, at least for the workloads he's tested it. So it is not really a theoretical exercise. The extra complexity apparently does translate to real world application benefits. I, of course, asked this because it is somewhat complex. The most notable addition, and we have a few slides on it, is the precautionary bandwidth probing text. Hopefully, after I run through it in slides, if you go back and read the text, it will be nice and clear. This is one of those cases where I think a picture is very helpful. We finally led the PR that I wrote previously adding some test cases for VBR. All it does is kind of describe, I think, about 15 test cases that you might wanna write and approximately what should happen. It doesn't attempt to go into, you know, a ton of depth, but at least if you were going to write a series of of test cases, know, you it gives you kind of like a starting point of, like, here are ones we almost you almost certainly should have just a sandy check that, like, your implementation doesn't have any obvious issues. And it does have things test cases including application limited behavior, so on and so forth. Yeah. We removed BBR from a bunch of methods that had it for no obvious reason because, you know, it's the BBR draft. I think you probably know it's BBR. Just made the methods long and some other editorial improvements. And yeah. So for the most part, these changes are refinements, and the cautionary probing is probably the the only change that is more than just, like, more clearly explaining what the algorithm is. So I'll go into that. It's a little bit high. It's got I guess I should be taller. So the goal here is to reduce queue pressure and packet loss. So in particular, the observation is that after you've observed excess packet loss because you've probed up, for example, and you've set in flight long term to a value based on probing up and probing too high and getting excess excess lost, The next time you enter probe up, the suggestion is to probe very carefully and kind of, like, stick your toe in the water colloquially and then immediately pull it back. And if your toe burns off, then you probably shouldn't keep probing. And so you wait a round trip and, you know, see if your packet got lost or if there's any loss in that round trip. And then you kind of go back into the probe up phase once you're certain that that, like, you know, putting your toe in the water sort of exercise was safe. And yet this, again, came out of Neil's experience, I think, commonly in data center networks, but not exclusively, where there really was, like, a max in flight that was sustainable on the path, and going even a little bit over that, like, immediately caused loss. And so, you know, the normal probe up behavior would kind of keep pro probing up for, like, a bit and also have a sustained in flight above the in flight long term, causing more loss as well as, like, if you were competing with other flows, a fair amount more loss to other flows. So the there's a few ways of thinking about it, but, yes, it's you're kind of getting to the in flight long term. Actually, I'm gonna go to the next slide because I think it's easier to see it. Great. I love the picture. So you our

[00:24:09] **Martin Duke**: I don't have a pointer, do I?

[00:24:11] **Ian Swett**: That's okay. If you see that second loss, that's the point at which in flight long term has been set. As you see that little dotted line at the top for in flight long term, when we go back into probe up, which is the second to rightmost little light purple square that says probe up, probe BWV up. We're only going to go to in flight long term, and the moment we hit it, we're going to probe down and drain the queue and then go back into refill and probe up if by the time we exit probe down, we haven't encountered any loss. So if we basically, that gives us a confirmation that going back to in flight long term is no longer dangerous, And that's the the goal here. And remember, there are a number of ways you can exit probe up. It's not just by hitting the in flight long term and having excess loss. You can also hit it just by not seeing any extra bandwidth that causes you to exit probe up. So there might be a number of flows where this new logic kind of never comes into play, particularly if it's a more heavily buffered network.

[00:25:23] **Jay**: So

[00:25:25] **Ian Swett**: as a note, this is implemented actually next slide. Yes. Actually, it did say ah, okay. Experience. Yes. This was in both TSP Linux PBR v two and v three since 2019 in the open source code, not in the Linux kernel. And so there is actually a fair amount of experience with it. And when Neil was just kind of doing a full read of the draft, he noticed that it was not actually in the draft. It has not been implemented in either ours or the move fast quick implementations, but that should be doable in relatively short order. This has only landed, I think, four weeks ago or something. So, you know, that's just a matter of time. So certainly by next IGF and hopefully sooner, we will have that implemented. In terms of the implementations, I know Neil has been doing some work on upstreaming BBRv3 in Linux. So and, again, trying to get the the draft and the code aligned. In Google Quiche, I actually kind of did an entire rewrite of v three kind of based on the v two code, but, like, trying to simplify it, remove some oscillate experiments as well as just making the code a little bit easier to read. That is mostly done it is missing a few features from v three that's were in the draft, and it is also missing precautionary bandwidth probing. So it is not quite done done. But the idea is to have, like, more of a full rewrite rather than just keep evolving the code from the v two code. And move fast also is fairly up to date. There are two notes about, like, differences from the draft that have kind of been there for a while, and also it lacks the precautionary bandwidth probing. So this is a relatively new issue, which is that there's previously, we discussed changing the probe RTT interval to ten seconds from five seconds. I think move fast and Facebook had positive experience with this in terms of quality of experience for video playback, presumably because ten seconds is long enough that you're almost always draining the pipe at least that often, so you avoid ProbeRTT being at a bad time for video playback. However, if we also have a ten second window for minRTT, if we're not careful, we could expire our minRTT right around the time we enter ProbeRTT, and then that could inflate the target congestion window that we're trying to drain to. So, you know, imagine if your minRTT your long term minRTT is ten milliseconds, but, like, more recent RTT measurements are, like, fifty milliseconds, then suddenly, like, the amount of in flight you're trying to drain to is five x higher than it was in to be. So Neil noted this issue. He also noted that he's not sure he's actually seen this in production, but he is currently, I think, operating most production traffic with the five second threshold. Anyway, so we have two potential ways of mitigating this. We don't have a pull request for other, but if you have a strong opinion about which way to go, then then let us know. But one is basically, you know, make the filter a little bit longer to make sure it actually lasts to the end of the ProbeRTT interval, and the other one is just save the value entering ProbeRTT. So it's possible we will write up both in a PR because text is cheap these days with AI, and we'll just see what looks better. But if you have a strong preference or implementation experience, please let us know. And the next two issues are actually issues that have been open for a while, and I don't think there's anything new to discuss except to continue saying, like, other people have implementation experience on either this or the next issue, they're definitely appreciated. So yes. I don't think there's actually anything new to discuss here. So unless people do have something new to contribute on out there, these two issues, then I think that's it. Yes. So that's a summary. I have a question. Did did folks understand the cautionary bandwidth probing algorithm, why it's there, what it does, approximate in the beginning? Yes? Okay. I think Neil is on as well if people have very detailed questions for for Neil who designed it.

[00:29:59] **Eric Kinnear**: Alton, I first.

[00:30:02] **Altanai**: Oh, hi. Alton, this is Shreenidhi. I've been following this draft from, like, the day one, but I don't understand the precautionary bandwidth is like a optimization or is it or somebody doesn't implement it, then they are not compliant with the VBR version three. Is it optional or is it mandatory?

[00:30:20] **Ian Swett**: I think it's intended to be mandatory. Although, I said that with the caution that the TCP version implements it, that last TCP version implements it, and neither the two quick ones currently do implement it. So but I think the intent is that it's not optional. Does that sound correct, Neil?

[00:30:40] **Neil Cardwell**: Yeah. That's that was my intent because it is a pretty significant improvement and it in friendliness toward the, you know, the the health of the queue and the the RTT and loss rate experienced by other flows sharing the same bottleneck link. Yeah. And it's not Okay. And it's a it's a little bit extra mental complexity, but in terms of the code complexity, it's not actually too bad.

[00:31:10] **Altanai**: Got it. Thanks.

[00:31:11] **Magnus Westerlund**: Thank you.

[00:31:14] **Roland Bless**: Yeah. Roland, bless. Just a clarification question. So this is just documenting what has been implemented 2023 or so? Yeah. So this has been implemented in Linux for since I think it

[00:31:29] **Ian Swett**: says 2019. Okay. But it's not implemented in, yeah, the two quick Yeah.

[00:31:34] **Roland Bless**: Because I mean, last year, we we did some measurements, and they showed that that we we are basically create some something like a BDP of queue. And just wanna make sure that this was already then implemented. So it's not nothing new in that sense. If you

[00:31:52] **Ian Swett**: are we using the open source implementation or the upstream implementation? Okay.

[00:31:56] **Roland Bless**: The the because the upstream implementation

[00:32:00] **Ian Swett**: is one, and the open source one is two and then three is my understanding.

[00:32:07] **Roland Bless**: No. We we are using the the BBR version three Oh. Version. Oh, okay.

[00:32:13] **Jay**: Definitely. Yep. Thank you.

[00:32:22] **Eric Kinnear**: Neil, I saw you joined and then left. But if you have a thing you wanna say in the chat too, that's cool.

[00:32:28] **Ian Swett**: That's all I have unless do you have anything you wanna add, Eric? Okay. Thank you very much, everyone.

[00:32:37] **Eric Kinnear**: Thank you, Ian. Are there any folks who are currently implementing with or experimenting with BBR who have changed what they were doing since last time, would like to chat about it a little bit, had any questions that they ran into? Could we get a brief wave or thumbs up? I don't think we need, like, an official show of hands for folks who are actively involved in making some of this go. I know folks have spoken up in previous meetings, so I'm just curious. Is there any updates there on that end? Where can I nudge people to actually stand up and go to the mic? Alessandra. Thank you.

[00:33:30] **Alessandro Ghedini**: Alessandra Ghedini, Cloudflare. So we've been working on and off on, you know, BBR, some version for QUIC. It's we are I feel like we're lagging behind, like, quite a bit at this point, but we've started enabling it more for our some of our production traffic. It's still very much work in progress.

[00:33:59] **Eric Kinnear**: Thank you. Yeah. That's that's good to know. Anybody else? Cool. Alright. Feel free to chime in on the list as you run into questions or anything else too. I know there's a couple other folks who we've talked about offline or in various stages of of implementing things or playing with implementations. So if you do run into questions or wanna file GitHub issues, feel free to email the list, file an issue on GitHub, all that kind of stuff. Okay. Next up, we have SCReAMv2, Magnus.

[00:34:43] **Magnus Westerlund**: Thank you. So I'm Magnus Wettlund. I'm here representing my coauthors. Ingemar is nicely on vacation, so and he has done most of the development work. So but it's I will do my best here. And if you have you run into a question, you will have to email them in worst case, but let's see what I can. So SCReAM, real time congested control. And some of the reminders here about that we are using not the congestion window, but the reference window. And this is to enable us to set a bit more bytes in flights than what the congestion window have and dealing with some of the video codecs, for example, intra frame or other to avoid queuing up too much data in the selling side. It's changing behavior depending on how long since it's been since we've seen congestion events. If longer, we can go into multi complicated increase, not only and it reduces based on both loss, ECN L4S marks as well as estimated queue delay increases using packet pacing at 150% of the target bit rate and can increase even further when it's uncongested. It's also integrating a lot of the integration with SCReAM is towards like a media rate controller for codec, etcetera. Here is some of the challenges, but for an implementation, but it's trying to ensure that you have it, work on that side, hitting the target bit rates as quickly as possible to maintain efficient interaction with congesting controller sites. So let's see what's progress been done since 2007 and to the current working group addition. One change has been to change this estimation deviation of the Q delay. It's gone to an average between the shortest and lowest shortest and highest value and using that average as to instead of the deviation normalized deviation before. And this is trying to get a bit better robustness against clock drift and also a little bit of schedule in nature. This, in its turn, it's used to affect the scaling of the reference window delay parameters, which affects the reference window's overhead, which is how much you can overdraft the reference window and also controls the reference window growth. So it helps reducing the rate variations and queue delay. And means that we have adaptive so this adaptive reference window overhead actually gives us we will see some simulation results on the next slide. So it restricts the bytes in flight when congested. When queue delay, the deviation by average is high. We get this reduction. And it allows bytes in flight to exceed the reference window more when we are uncongested. So let's look at this. On the left side, we have if you have a static 1.0 scaling. And then on the right, we have if you have an average. As you can see here on the bit rate, it becomes more stable with this adaptive. It's and it also keeps the queuing delay more controlled and not growing SP is spiky. Yes, So and this is so at five seconds into this simulation, we drop from 10 megabit to five megabit. And then after ten seconds, it increases again. And another example here is coming up on the next slide is around the trying to improve how it handles reference window undershoot. So when the bit rate drops suddenly, the queue buildup can result in that actually, at clocking parts, etcetera, anyway and increased RTD anyway reduces the actual transmissions. So the reference window decrease doesn't need to be as hard as if that bit rate is larger than the target bit rate. So this is a bit harder to see. But if you're looking around that event at five seconds when the bit rate drops, on the right side, we don't drop as fast and you recover more quickly, which is why the Q delay after this is bouncing a little bit more up and down towards limit probing around what's available based on probing that skew delay and then falling back and going at it again. So it just becomes more responsive here in these events. One other change is trying to improve your robustness to linked layer, in other word, noncongestion related losses. So what's been implemented is that reference windows reduction upon when you have lost packets only happens if the average loss rate is above a loss rate threshold. And currently, the defaults are set

[00:40:59] **Ian Swett**: to

[00:40:59] **Magnus Westerlund**: 1% or if the queue delay is larger than queue delay targets, a quarter of the queue delay target. So if you're otherwise, there's been very little indication on congestion or we don't produce it in considered as a spurious loss. And this increases the robustness and also avoids excessive AQM related losses. And so this example, we actually have in is an example of a data limit case because we have a link capacity of 100 megabits. And the media bit rate maximum is set to 10 megabits. So on the right, you see that we're using those full 10 megabits, except a few cases where it's like drops down and there's enough packet losses to actually cause something. While on the left side, you see these spurious congestion law limits the bandwidth reduction. And this is for 05% of packet loss, random losses here applied. So we also have a comparison with and without L4S, and this is for Mobilinq, simulated Mobilinq. Christian, is it on the previous slide? Or should I just conclude on this? I will conclude on this slide, then you can this is showing that applying L4S gives much more stable low queuing delay. It's slightly lower bit rates. But so Christian, go ahead.

[00:42:56] **Christian Huitema**: Good morning. Yes. Regarding this business of data rate, I mean, much loss rate is loss versus spur you think? It's largely a question of fairness as well. I know that many implementations do that. We certainly do that for C4 and for b b r and for, even for Cubic. We could do it for Cubic, I mean. The consequence of that is that the rate determines competition, the how the competition goes between conquering flows. So if one flows react, if the loss rate is observed to be on average 1%, another use 2%, another use 20%, we will have, obviously, some unfairness issues. Yep. And I wonder whether the the working group here should take that particular point and maybe prepare a draft about it, about how do we think about it, what we should do. The other thing that is important there is that take the example of l four s. L four s has this rule for competing between non ECN and ECN flows that says that the drop rate of the non competing non ECN flows is basically the square of the marking rate of competing of ECN flows. That square equation is based on the old modeling of TCP, asymptotic with the behavior of TCP. And that equation assumes that each individual packet loss drives down the congestion window. Now if we start I mean, if if we actually do what you're presenting here and what I am doing too and what other people have been doing too, which is reasoning on average, reasoning rather than reasoning on individual losses, then all these formulas become obsolete. They are just wrong. And so I think the IETF should start a discussion about that and eventually come to recommendation. It'd be nice if everybody converged to 2.25% or whatever it is we converge on. But, I mean, the same thing. And it'd be nice also if we told that to the modeling communities and says, hey. You know? Be careful what you assume. Okay. I'll let speak.

[00:46:10] **Magnus Westerlund**: Yeah. I I not disagreeing. Think with at least it's worth thinking more about it because I think you're right that it it is a point of competition. So, Maria?

[00:46:31] **Eric Kinnear**: Try the other one. There we go.

[00:46:37] **Mirja Kühlewind**: I just want to react to the l four s part. So, like, screen has l four s enabled by default. So if it sees an l four s marking, it will react differently. This is only for non e c n or l four s enabled queues. Yeah. And but I agree on the point that BBR has the same that they also have a loss threshold. So, like, using the same loss threshold is probably not a bad idea, and thinking about this in general terms is probably also a good idea.

[00:47:10] **Martin Duke**: Martin, Google. I think Christian raises very interesting questions. I would love to see him or somebody else start to see that discussion. I I wonder if the ECN marking thing is a problem given that all these loss all this ignorance of loss is related to queuing delay increase, which is or or lack of queuing delay increase, which is where exactly where you will see, you know, ECT or rather CE marking when the queues start building. That's obviously not not perfect, but I I would hope that those particular equations would not be obsoleted by something like this. Thanks.

[00:47:55] **Christian Huitema**: Well, in in fact, they are.

[00:48:03] **Martin Duke**: That that that doesn't match my intuition, but you've almost certainly thought about this more than I have.

[00:48:08] **Magnus Westerlund**: So yeah. I I I think we have to come back a little bit more onto this later on. And I think for the SCReAM is have very much the L4S integrated, behaving very much like the in some sense, it's acts it works both on the delay and the L4S targets. And basically, what it does is it converts the delay into equivalent alpha factors for like for L4S when it's being affected by the delay. So but yes. So but so we are at the phase where we need to start actually writing down a bit about the evaluation of for SCReAM according to the RFC 9743. And one question here is should we document this in a separate document or in the appendix in the draft? Is your opinion on that? I think we also started things there. Is is does everything need to be shown by basically by simulation practical evaluation and how much can you show by theory? Where certain things based on the algorithm will tell you that, okay. This according to the algorithm, it's a fulfilled some of these criterias. So now if you have input on that, please send something. Okay. So another aspect which you may been thinking a little bit is the question of detection of rate policies of remediation. There's an example algorithm in the draft, but polices are difficult to handle. So should we ask to ignore this, should we continue working on it? So, Neil?

[00:50:20] **Neil Cardwell**: Yeah. Regarding policers Mhmm. My suggestion would be from our from our experience with BBR, be to maybe try to take an approach where the general algorithm has a response to packet loss that leads to sort of acceptable behavior in the presence of policers rather than going down the road of having an explicit detector or estimator for the presence of policers and then having a specific police response. And this is sort of based on our experience with DBR, where in version one, we we did invest a fair amount of time going down that road of sort of explicitly trying to detect the presence of a policer and then having a specific response for that. And I think, you know, we we did get to an algorithm that, we felt worked pretty well in practice, but it there was a fair amount of complexity. And at the end of the day, we decided in, after b b r b u one when we were moving on to v two that we wanted to get rid of that, detector to sort of simplify the code. So that would be kind of my advice there would be to to not go down that road.

[00:51:37] **Magnus Westerlund**: Very much appreciate that feedback. So yeah. Another aspect is which we're looking forward here is we Google, people are helping us with the WebRTC evaluation of screen L4S. It's ongoing. Hopefully, see some results later this year from this evaluation. And it will hopefully be very enlightening to see how well it's working in the larger field tests.

[00:52:13] **Gorry Fairhurst**: So, yep. Gary Fairhurst, was gonna try and speak to rfc nine seven four three. Yeah. I see Martin's at the back as well, so I don't know what Martin thinks. So my personal opinion when we wrote that was we were trying to capture stuff that had to be documented somewhere in the ITF process.

[00:52:32] **Jay**: That

[00:52:34] **Gorry Fairhurst**: might be via ICCRG technical talks or it might be via papers or it could be in a document. I don't think I ever saw each congestion control document having like some great big annex that was describing this. So probably I was thinking of a separate document. Whether that's published or not, I don't even know because it was more to make sure there's coverage and understanding. Yeah. Because there's some pretty tricky design stuff going on here, and we all know that. Somehow, we have to get good confidence. So that was my take. Yeah. I didn't see Martin run to the mic. So I guess he can comment differently if he feels like it. That help?

[00:53:18] **Magnus Westerlund**: Yeah. It sounds up.

[00:53:26] **Martin Duke**: First of all, administrative note,

[00:53:28] **Christian Huitema**: if you can if

[00:53:28] **Martin Duke**: you choose to leave, you can leave by that door that is on my right, Eric's left. That one actually closes Nice. Which is probably good or closes itself. So I agree with Gory that we did not have to have pages and pages of text in the specification discussing the 9743 evaluation, nor do we need to publish some other RFC that that has all that stuff in it. I think just having a report in the minutes of the in the materials of the working group for for internal purposes is perfectly fine. Is evaluation necessary for all cases? I do know this is targeted for experimental, which is good in terms of maybe not having to it depends what you mean by evaluation. So this is I think

[00:54:17] **Magnus Westerlund**: that was part of my question. It's like, it's one thing to write down. So, okay. I will argue in has the following properties in this condition, which this question answers. Just write down that that it should never happen to be worse or behave different than that because of these following factors. Or if you actually needed a simulation for this use case, a particular case, to show that, yes, it's actually fulfilling those properties that the RFC specifies.

[00:54:45] **Martin Duke**: So if I understand correctly, the applicability of this Mhmm. This algorithm is global deployment in the Internet for application limited flows. Yes. Okay. So to the extent you know, I think there's some language to live out that not application limited flows. I think you can soft pedal the evaluation of that because people shouldn't do that. The gold standard obviously is data from Internet wide deployments for Internet scale effects, for scenarios that are not well captured in Internet but are important, like satellite stuff and so on. You know, you probably have to do simulations. The 9743 does not preclude making kind of assertions, I guess, that based on some logical things, people can accept that and not accept that as soon as they decide. If you if you could at least do simulations of the of the cases that are applicable of the use cases that are applicable to this app to this algorithm, that would be ideal well, not ideal. Data's ideal, but Yeah. It's probably sufficient.

[00:56:03] **Magnus Westerlund**: Yep. Okay. Thank you. Stuart?

[00:56:07] **Stuart Cheshire**: I'm Stuart Cheshire from Apple. I came up to the microphone because the subject of polices has come up a couple of times, and I would agree that I don't think it's productive to try to put special code in to do something special for polices. Polices exist for one purpose, which is something in the network wants to tell the sender what speed it should be sending, and the the only tool they have to do that is packet loss.

[00:56:36] **Jay**: Mhmm.

[00:56:36] **Stuart Cheshire**: And if you don't lose packets, that means go faster. And if you lose a packet, that means go slower. It's a it's a Yeah. It's a simple binary signal. I don't like polices because they're very insidious. When when you have a queue, the queue length grows, and you can in principle, you can infer from the increasing round trip delay that you're going too fast, and you can back off before you lose packets because Yeah. Losing packets is bad because then you have to retransmit. Retransmitting adds delay. So problem with polices is you get no warning Yeah. That you're exceeding your quota until suddenly it loses a packet, and and you have no way to anticipate that and avoid it, which seems a little bit unfair. I do hold that hope that one day we'll see polices that use ECN because that's a way to give the desired feedback to the sender about its rate adjustments without forcing it to retransmit. I don't know if there's any hope of that on the horizon, But let's shoot for that as our goal. Policies that use ECN rather than trying to work around today's polices. Okay.

[00:57:45] **Magnus Westerlund**: Thank you. We also have some consideration if we should do make screen more conservative when L4S is not enabled or no L4S in congested nodes. But we'll have to see if we get some results or considerations for that. So I think that is all that's planned to present. So unless there's some other questions, I'm done.

[00:58:21] **Eric Kinnear**: Thank you, sir. Next up, we're gonna move into search. And I think, Jay, you wanted to share your own screen. Yeah. That's not helping.

[00:59:16] **Jay**: Yeah. Hi. This is Jay. I'm happy to provide the update on the search algorithm. Search stands for slow start exit at right choke point. K. So in this update, we are going to provide the update for two things. One is the slow start access signal analysis. This is respond to question, why should we use, you know, the the good bit or or, you know, basically, the good bit rather than the RTT increment. And, therefore, we did some analysis on the signal strength and stuff like that. So we're gonna give the update on that. And the next thing is we added the search draining phase to lower the the queue upon slow start exit. So we'll provide the update on that too. Okay. So as everybody know, traditional TCP uses the loss detection or the ECN from the active queue management supported routers as a signal to access slow start. And typically, especially for the losses, we have chance to off a blow. So in order to reduce that, researchers came up with, you know, mechanisms such as using a signal like, you know, RTT inflation from the base RTT or use it monitor the the delivery rate during the slow start. Actually, flatten out as a signal for congested router, the router getting congested, and we came up with those mechanisms. And the RTT inflation using that as a signal is more like HyStart and HyStart plus plus. And then delivery rate flatten out using that as a signal is either VBR or a SEARCH. K. So it's not about the the signal that we use, but it's the the quality of the measurement that actually affects the the performance of the of the slow start exit. And we don't wanna exit too early because that's based on the wrong signal because that is going to affect the the performance, you know, during the slow start. So when we actually look at the the RTT, RTT doesn't grow before it hits the the congestion. Congest and then it actually grows. K? And then if you look at the delivery rate as a signal, it exponentially grow. And then once it actually hit the congestion, then it flattens out. What about the noise in the measurement? So if you actually look at the noise in the measurement, the noise for RTT doesn't change before and after the condition point. And this graph actually shows a noise in terms of the standard deviation of the RTT measurement. And for the delivery rate, the delivery rate, what happens is based based on the window size with irrespective of the window size to measure the delivery rate, what happens is as the the pipe is actually getting filled up Closer to mic. As the the pipe is actually getting filled up, what happens is the measurement get more accurate and accurate. So so, basically, nearby the congestion point, we have a accurate measurement of the rate. So in summary, RTT as a signal, the signal measurement quality improves when you actually hit the measure the the condition point. And for the the delivery rate measurement, the quality actually increased before hitting the condition point and maintain high quality. So this graph shows the the the measurement quality in terms of SNR, signal to noise ratio. So if you actually look at the SNR of normalized diff, which is the delivery, what happens is after the SNR goes to high before the BDP, and then once you actually hit the BDP, it actually goes up. And for if you look at the SNR for RTT inflation, in this particular one, we use the RTT of the last round minimum as the base. And then if you actually do that, the SNR actually fluctuate a lot because the using the RTT of the last round as a base RTT also introduced the noise.

[01:04:42] **Eric Kinnear**: Clarifying question. Do you want

[01:04:44] **Ian Swett**: I have a question, but you you can feel free to go to more slides. I'm happy to wait till the end, actually. It's fine.

[01:04:49] **Jay**: No. I mean, this so, basically, this is the the end of the first topic, and this is basically well, you know, the one thing that I would like to actually, you know, stress on this is, you know, this is one of the reason why, you know, using the normalized difference of the the deliver rate from the sending rate is actually a good signal to use for detecting slow start exit. Yeah. Go ahead.

[01:05:16] **Ian Swett**: Correct. Ian Spread, Google. Can you go back a slide?

[01:05:19] **Jay**: Yep.

[01:05:23] **Ian Swett**: More? Awesome. So okay. So the bottom left is MinRTT and the bottom or sorry, windowed MinRTT. And the bottom right is the normalized diff. Basically, it's the deviation from the expected bandwidth. Is that correct?

[01:05:40] **Jay**: Right. So, I mean, this is actually send bytes one RTT ago, how much I sent, and then basically and now, which is one RTT later, how much was actually at?

[01:05:51] **Ian Swett**: Okay.

[01:05:51] **Greg White**: So And

[01:05:52] **Ian Swett**: after the cut after the point at which the deadline, that's the point which in theory, like, you should be exiting slow start. Right.

[01:06:01] **Jay**: Ideally. Exactly. So the red line, I forget to actually name it, is a is a BDP. Basically, when the the route the capacity is actually filled up. And

[01:06:11] **Ian Swett**: and so the fact that that solid line is drifting above the Yeah.

[01:06:16] **Jay**: That means that means the quality is actually getting better because yeah. I mean, what what this really means is this if you actually so if the send byte minus delivery byte is going to be converged to point five, and this is actually what you're seeing is going to the point five. Because in slow start, you actually send, you know, two packets for one at received. Okay.

[01:06:45] **Ian Swett**: Okay. I'm just trying to understand the data.

[01:06:47] **Magnus Westerlund**: Okay. Got

[01:06:48] **Ian Swett**: it. Think I understand. Maybe.

[01:06:49] **Jay**: Okay. Thank you.

[01:06:53] **Gorry Fairhurst**: Gori first, individual. I missed what you were actually measuring here. Are these compiled data from lots of Internet paths? Are they a particular type of equipment path? I Oh, no.

[01:07:08] **Jay**: No. No. This is what what what the sender is actually, you know, measuring as part of the ad coming back. Okay. So

[01:07:16] **Gorry Fairhurst**: What is the path?

[01:07:18] **Jay**: Oh, the path. We actually you know, this particular one is actually the emulated network. We have done the same, you know, the study for the real network, like Wi Fi. And, also, our company is doing the geo network, so we actually measure the same thing for the geo.

[01:07:37] **Gorry Fairhurst**: Do you have that data now? Can we start?

[01:07:39] **Jay**: I can I can actually provide the data if you want? I don't really have the data right now.

[01:07:44] **Gorry Fairhurst**: Okay. I'm not sure why we're seeing this in a working group if it's just an emulated path. But please continue and show us how the mechanism works because I'm really interested in the mechanism.

[01:07:59] **Jay**: Any more question on the first update?

[01:08:04] **Altanai**: Yes. I I have a question. Altanai, this is Shreenidhi. So you you just mentioned that you are implementing this at the the transport stack. Correct? How would it matter if the app is itself restricting the outgoing packet? Like, app is modifying app limited browser activity, for example. Will it throw your previous two new RTP measurements out of back?

[01:08:33] **Jay**: I didn't actually catch your question. So what was the question again?

[01:08:37] **Altanai**: Question is, if the if the browser's app is limiting its outgoing packets by itself, okay, the application is modifying the outgoing packets. Will it not throw your network stack calculations out of that?

[01:08:52] **Jay**: Yeah. So you're talking about the app limited case. Right? So, yeah, in app limited cases, which is a lot of case in the the network flow, the expectation is not exiting slow start. That's probably what's gonna happen. So, hopefully, the search will not actually, you know, kick the flow out of the slow start in that case.

[01:09:16] **Mohit P. Tahiliani**: Okay. Does

[01:09:17] **Jay**: that answer your question? Yep.

[01:09:19] **Altanai**: That does. Thanks.

[01:09:20] **Ian Swett**: Okay.

[01:09:24] **Jay**: Okay.

[01:09:25] **Christian Huitema**: Yes. Regarding your comparison between and that was the previous slide was nice. But Oh. That one the yes. These are the theoretical curves. What I'm worried about those curves is that the measurement of the data rate is depending on the measurement of the RTT.

[01:09:53] **Jay**: That is true. However, the thing is when you actually measure the the good bit, right, I mean, irrespective of the the measurement interval, you know, basically, slow start is actually pounding the network. It's adding a lot of, you know, like, packets to the network to actually congest the network in in so when it actually goes to the the capacity, if the throughput is actually go to the no. Approaching the capacity, what happen is the measurement interval, whether it's actually aligned with RTT or not aligned with RTT, it doesn't really matter. But the good put, you know, measurement gets more accurate. That's the point that I would like to make.

[01:10:36] **Christian Huitema**: Oh, yep. Well, basically, slow start let measurement boils down to looking how many packets I sent in a given time and dividing by that time. The point is that that time does include an RTT measurement. You can put it at the end, at the beginning, or whatever.

[01:10:56] **Jay**: Yes.

[01:10:57] **Christian Huitema**: And so the the RTT jitter, which you are measuring as the uncertainty on the RTT, your minimum RTT discussion also impacts the precision of the rate measurement. In fact, it impacts it a lot because it's on the divider.

[01:11:19] **Jay**: That that is true. However

[01:11:21] **Christian Huitema**: And so so I'm we have I mean, I'm not saying that we should use one or the other. I'm saying we have to be very careful that red measurement appears to be solid, but the hypothesis between that solidness the hypothesis that one, of course, the rate is constant, which it may or may not be. And the other hypothesis that the delay is constant, which is definitely not true in some Wi Fi networks.

[01:11:53] **Jay**: Yeah. So I'm not really saying we should use either RTT or the delivery rate. I'm just trying to explain the the property of those measurement. And, basically, RTT is more like a a one dimensional measurement, and the delivery rate is actually coming combining two dimensional measurement, like, you know, how much when. Right? And and basically and therefore, we can actually get more information about the network, I believe, you know, by monitoring the delivery rate. For example, I'm just saying

[01:12:30] **Christian Huitema**: I mean, we we agree we agree on that. That's not a question. Yeah.

[01:12:33] **Roland Bless**: But I mean, but but I

[01:12:35] **Christian Huitema**: But but we we have to be careful, but there's a lot of noise in the red measurement that derive some jitter in the RTT.

[01:12:46] **Jay**: Yes. I I agree with that. Yeah. But, like, you know, all I'm saying in in this graph for the normalized diff, you know, graph is that whether yeah. You're you're right. If the denominator is actually affected by the how much you have to go back, you know, is determined by the RTT measurement, yeah, that's another, you know, source of error. However, you know, when we actually do the measurement, actually, when it when it actually goes back I mean, when it approaches the the congestion point, even after a couple of rounds, I mean, it actually the the measurement error is actually reducing there. So so, basically, that's why it actually shows the, you know, smaller margin for the the noise. K?

[01:13:39] **Christian Huitema**: Okay. Thank you.

[01:13:48] **Eric Kinnear**: And I think we have Neil.

[01:13:52] **Neil Cardwell**: Yeah. Regarding the discussion of application limited flows, I I believe you said, hopefully, search does not kick the flow out of slow start. But I what is the actual mechanism that the algorithm uses to avoid premature exit of slow start for application limited flows?

[01:14:10] **Jay**: Yeah. So, I mean, that's more like a current work that we are actually doing. Probably so the current thinking is to use the the SNR. If the SNR is actually low, that means the signal the measurement doesn't really I cannot really trust my, you know, measurement too well. So in that case, we won't actually kick the kick out of the kick the flow out of the slow start. That's the mechanism that we are currently, you know, working on to use.

[01:14:43] **Neil Cardwell**: So can you give us some intuition about how that would work? So let's say, for example, that there's an application limited flow that sends a 100 packets each round trip time

[01:14:54] **Gorry Fairhurst**: Mhmm.

[01:14:55] **Neil Cardwell**: And the BDP is 200 packets, how does that keep us from exiting slow start prematurely?

[01:15:02] **Jay**: Yeah. So the thing is, you know, basically, if we actually have the sending rate and then receiving rate, most likely, if it's not if it's app limited, then basically receive rate and then sending rate at the RTT before is going to be equal. However, when we actually measure each of the bin, our current bin size is about one third of the RTT. Basically, if you actually monitor, you know, those bins, I mean, we don't really have enough bins to even, you know, make the the search exit decision. And in that case, we are not going to make any decision about the slow start exit. If we have large enough packets, you know, then basically what happened is we do have the the enough bins to make a decision. But if you actually calculate the the standard deviation of the the last two RTT, we don't really have, you know, a good confidence because the the signal to noise ratio is going to be low in that case.

[01:16:18] **Neil Cardwell**: Okay. Thanks.

[01:16:19] **Magnus Westerlund**: Okay. Thank you.

[01:16:26] **Jay**: Okay. So the next one is actually very quick one, I hope. So, basically, we added the drain base at the search exit point. That is so the earlier version let me just so the earlier version, when we search detect the exit, then we actually set the SS threshold to the the the current condition window at that time and exit it. However, that actually, you know, have the the packets in the queue build up. So we wanted to actually clean that up. So what we are doing right now is we're trying to actually drain the excessive queue in the congested router, and that's by, you know, compute the target cwnddow because now we know what is the the the the the capacity. So we have, you know, the capacity is computed based on the three bins, which is one RTT. And we know the minimum RTT and all that information, so we can actually come up with a target cwnddow. And therefore and what we're doing is, you know, we we could actually set the congestion window, you know, to that target window, but we don't do that. We slowly reduce the congestion window. Like, every three three packet worth of bytes, we reduced by one. And that was because, you know, packet loss can actually happen after after the search exit. In that case, we don't want to degrade the performance of fast retransmit because fast retransmit, you know, if you actually, you know, cut off too much, then there won't be enough enough congestion window size to do the fast retransmit. So in order to fix that, not to cause that, we were actually slowly draining the data. So, for example, this one actually shows the one example run where the search exit was detected around four v d p. And, basically, the draining is and entering to the draining phase and, you know, exit once everything is actually getting drained. And at the time, it's hand handed off to the conditional avoidance. Okay? So, basically, that's the update. And then the ongoing work right now is, since now, we have this property that the signal to noise ratio is actually not that great during during in the in the initial start. What we can do is we can actually use that to determine what rate the cwnd should grow because there is no magic about, like, base to exponential. So if we have, you know, low signal to noise ratio, that means the path is is actually not full. So, basically, we could actually even grow faster. And then, you know, when the signal to noise ratio is actually good, then, basically, we are approaching the congestion point. So we probably don't want to even do exponential base two. So we could, you know, like HyStart plus plus, we could, you know, reduce the incremental rate. So more gradually. So that is basically things that we are looking into.

[01:20:11] **Martin Duke**: Okay. Martin, do Google just a quick comment. I understand why you're picking three based on fast retransmit, and you have to start somewhere, and that's great. But don't over index on it because as you know, there's rack. There's all this other stuff. Yep. You You know, you're gonna have to as you generalize this, you're gonna need to have a more generalized approach.

[01:20:33] **Jay**: Thanks. Got it. Thank you. Any other question? Alright. Thank you.

[01:20:49] **Eric Kinnear**: Thank you. Alright. We wanted to do a quick poll. So if you're not in the Meet Echo yet or paying attention to your device, now's a great time. So the first question is simply a I have read a recent version or really any version of this document. So we just wanna get a sense of kind of who's looked at the doc, who's read it. Know some folks have given some great feedback on the list, etcetera. So thank you to all the folks who've done that. And we'll leave this poll open for a little bit. And then we will have one more coming after it, which is asking about who has implemented or is planning to implement. But for right now, this is just have you read it? Looks like that's almost settling. Oh, there we go. Couple more. I also sometimes have no opinion on whether or not I've read something. Or I forgot that I read it. That also counts. Okay. Looks good. And next, we have I have implemented this or am interested in implementing this. I know there's some folks who've given some feedback on the list saying like, hey. I'd like to see how this compares to insert x other thing here, or I'd like to see these changes before I proceed with some of it. So this is both I have implementer interested in implementing, but also if I saw these changes, I would then be willing to implement. We had asked in a previous meeting about whether or not we think CCWG should be looking at slow start in general and generally said we think we do. And we have a couple of different proposals. This is one of them. So that's what we're trying to get a sense of right now. Christian, I see you are in the queue.

[01:22:56] **Christian Huitema**: My uneasiness with taking slow start as a as a subject is that I am not sure that I mean, I think that the big problem we see address in search is a discrepancy between slow start and congestion avoidance. Is that once you exit slow start, especially on a high bandwidth high high bandwidth product network, then congestion avoidance is really unresponsive. I mean, if suppose you're on a satellite network and you you have divided your your windows because of some loss or something, it take a very, very, very large number of RTTs before you gain back. And so that puts a premium in being really, really cautious about exiting slow start. Because if you exit it too soon, then you will have consequences that go down on the connection. And I think that's a fundamentally bad design because, I mean, we have to assume that network condition are going to change all the time. And so this design in which you do all your discovery during initial phase and never change it after that or change it very, very slowly only after that is fundamentally bad. And that that that's why I'm uneasy with this.

[01:24:30] **Eric Kinnear**: I think that makes sense. If I can say that back to you to make sure I understand it. You're essentially saying if we were able to be more responsive to conditions during congestion avoidance, we wouldn't care quite so much about where we exited slow start because we'd be able to fix it rapidly anyway. Yes. Yep. And I think there's also the point that there's a substantial percentage of connections that never exit slow start before they're over. And I know in the previous IETF, we saw some data suggesting some meaningful user wins from being better about how we get to the right rate. Yep. Both of those kinda go together, I think. Alright. I think that's good for our polls. We're not asking again or anything for adoption for this particular document right here. So we will keep coordinating with the different authors of the different slow start proposals and kinda talk about that and give some updates to the group as we go. Next up, we have quick slow start evaluation at Mozilla, Oscar.

[01:25:30] **Martin Duke**: Can I make a comment about adoption? You absolutely can. I'm I'm gratified to see that people are interested in implementing this. If we're talking about adoptions, particularly as a standard track document We are are not, to be clear. Okay. That's fair. Well, I I I I think an interesting question to ask is is anyone doing Internet scale experiments with it if if we are going to if if we're considering going down that road? Cool. Thank you.

[01:26:10] **Eric Kinnear**: Convenient timing for this talk.

[01:26:11] **Christian Huitema**: Yeah. Also not an accident.

[01:26:15] **Oscar**: So yeah, if you look at those slides hi, I'm Oscar from Firefox networking team. So we are actually doing internet scale experiments on this. And yeah, let's talk about that.

[01:26:29] **Eric Kinnear**: Is this working? It should be. Thanks.

[01:26:32] **Oscar**: Ah, there it is. So, yeah, in neqo, which is Firefox's QuickStack, we have recently implemented search and HyStart++ in kind of like a runtime switchable framework than in in addition to what we already had as our default, which is just like the standard congestion event based classic slow start. And so we've done some ABC experiments on this, on desktop nightly, on desktop beta, and now on desktop release, which ran from July 2 to July 19. So this is, like, very, very recent. And, yes, now we bring some of the results, we also bring some questions as to help with interpreting what we're seeing. So let's get into it. So this is actually kind of like the dataset makeup to the point that was already brought up that like a lot of connections never exit slow start. Of course, this is the browser's perspective, so this might look slightly different for, like, CDN. But yeah. So out of all of our connections that are part of kind of this experiment, the vast majority never grew the congestion window at all, so app limited all throughout. And of the smaller part that did grow the congestion window, which is 8.61%, of those 8.61%, only 4.42% exited slow start to just give this some kind of scale. And then if we look at those four point, what was it, 4.42%, we see that it's actually a bit different for the heuristics we deployed. So search is exiting slow start slightly more often than HyStart++ and classic with about like 0.5 more. And if we then look a bit further towards how many of those slow start exits are actually exited heuristically. So out of all the slow start exits in that cohort, how many are exited through the heuristic that runs on that cohort? So for classic, this is zero as expected. And then for HyStartt plus plus, it's about 2.5%. For search, it's 12.6%. So a lot of the exits are actually based on congestion events. Of course, search here is better than HyStartt, but this like, both are fairly low percentages. And to make some sense out of that, let's, like, scale this back out of the experiment and look at kind of all of Firefox release. No heuristics running. So this is our SS thresholds at slow start exit. And this is only for connections that actually did grow the congestion window. So if we do like one GET request that has some random loss, this is not included here. And so we see p 50 is at 20 k, p 25 is at 14 k, and p five is actually at 8.6 k, which means with a w 10 that we saw a congestion at the initial window, basically. And so all of this is, yeah, too early for any of those heuristics to work. And this kind of explains why we're seeing such low percentages. Now let's actually look at kind of like how the heuristics perform, how it influences other measurements. So for packet loss, both in the ninety fifth and the ninety ninth percentile, we can see that search has lower packet loss ratio than HyStartt has lower packet loss ratio than classic. And this is consistent on a day by day basis. So across both percentiles, search every day has the lowest packet loss ratio. And then if we go to the next one which is upload throughput, we can see that this is kind of all over the place. And so there's no clear winner. So like one day search has the best upload throughput, one day it's HyStartt, one day it's classic. So for this, for our experiment, I think like upload throughput data is kind of inconclusive. Then from those high level metrics, if we look a bit lower actually at the slow start exit point. So now we're looking at the exit slow start threshold and we're looking at the percent difference to the classic cohort over all the percentiles. And if we look at the higher percentiles first, yeah, now it works. Okay. Then we can see that search is often exiting with a slow start threshold higher than classic. And for that, we got to say that we did not implement this brain phase that Jay just talked about. So this is just search detected slow start exit and we exited slow start. So yeah, search, higher number in the upper percentiles, which is probably good. So it's after congestion but before loss. And kind of inside of that, the congestion window where you detected where you would experience the loss and then that times zero dot seven with us deploying cubic. So it's kind of in that range. And then for HyStart, it's actually a

[01:32:07] **Matt Joras**: bit

[01:32:07] **Oscar**: lower. So HyStart might be exiting prematurely there. And I see Martin maybe has a question for this Yeah.

[01:32:16] **Martin Duke**: I'm I'm trying to reconcile this with the higher heuristic rate for search. So does this mean that when when HyStartt is real when HyStartts HyStartt kicks in his heuristic less often, what it does,

[01:32:29] **Jay**: it really kicks it in,

[01:32:30] **Martin Duke**: like, at a at a rate that maybe is too low?

[01:32:34] **Oscar**: So I can't like perfectly answer that. Sure. But so yeah, my take is that obviously when it kicks in that it is in those higher percentiles because that's where we see an actual difference in the SS Thresh. And it is lower than what the classic saw after reducing its congestion window by 0.7. So that's what the data says. The interpretation is kind of up for grabs.

[01:33:09] **Magnus Westerlund**: Yep.

[01:33:15] **Antonio**: Antonio Claflare. But another possible explanation might actually be that your thing would search more often. Some of those might be incorrect early exits that might be inflating your higher percentiles. So comparing against the full distribution of, like, final windows may also be interesting.

[01:33:40] **Oscar**: Sorry. Again, so what you're saying is search might have early exits, which would influence this what we're seeing you're looking at. Yeah.

[01:33:49] **Antonio**: I imagine make the the early you actually see higher exit rate

[01:33:53] **Jay**: Yeah.

[01:33:53] **Antonio**: For search. Yeah. So that could actually mean that some cases that did not exit with the classic algorithm and the other one are exiting with search and those windows are actually end up being larger for that reason.

[01:34:07] **Oscar**: Yeah. So I think that because this is by percentile. Right? So anyways, we're looking at the higher SS fresh values here. And so for those higher SS fresh values, we see that they are at the same percent or higher for search than for classic. Yeah.

[01:34:32] **Matt Joras**: Mattros Meta. Did you collect any information about, like, you know, the browser metrics besides just the connection metrics? Because it's kinda hard to reason about these in isolation. Because you had you had, like, a a very noisy graph for a p 95 of, like, upload throughput. But since I was like, does this make any difference to, like, you know, page load time, like HTTP requests, stuff like that since it's kind of hard to see that this did anything other than it being kind of what it was supposed to do on the TIN from the connection standpoint.

[01:35:08] **Oscar**: Yeah. So the problem with the browser metrics is that they are mostly about us as the browser downloading, right? In which case, that's not controlled by our congestion control of our servers.

[01:35:21] **Matt Joras**: Requests are still sent out, right?

[01:35:23] **Oscar**: Yeah, right. But so I assume for a case where really we're just sending some GET requests to then get the site, this will not show in this data anyways because we're filtering for only connections where actually were not app limited.

[01:35:37] **Matt Joras**: I wouldn't assume that. So it'd be kind of interesting to see if because like we like, so when we've changed things about slow start, it does make an impact on things like get requests and post requests and page load and stuff like that. Even if you're just from the client side. But it would also be interesting like the actual uploads themselves. So if those are large, knowing what the effect on that is would be interesting.

[01:36:04] **Oscar**: Yeah. I I don't have the data right now. We can talk afterwards and maybe we can we can look at something on the live dashboards. So yeah, then I'll pick it back up and now if we actually look at the lower percentiles, then we will see that here in the lower percentiles, so with low assessed fresh values, search is actually also seeing kind of like there's a difference to the classic cohort which means there are search exits with low SS fresh values. And so if we look into this and this has been discussed just now, so our take with that was also that we think search kind of might be struggling with app limited traffic patterns that we see in the browser. And so if we are not continuously sending, then this might break some of the assumptions that search makes and it might then mess up kind of the bytracking logic which then is causing it to maybe exit early. And so yeah, I think Jay presented some ideas on how they could mitigate this, and I would love to see that because I think for us, search was performing quite good in, like, the higher percentiles. But this, for us, would right now kind of be a blocker to just, like, deploying this as a default because in the app limited cases, we shouldn't see exits, obviously.

[01:37:34] **Martin Duke**: I'm sorry. Antonio's question made me think that maybe I'm confused. That PDF is over connections that had a heuristic exit or all exits from slow start?

[01:37:47] **Oscar**: This is over all exits from slow start.

[01:37:51] **Jay**: Okay.

[01:37:51] **Martin Duke**: So non heuristic exit should be very much like the

[01:38:04] **Oscar**: So they they should just say

[01:38:05] **Martin Duke**: classic. Right?

[01:38:07] **Oscar**: Yeah. But so so this is per experiment cohort. So we split the population in three cohorts running classic search or high search.

[01:38:16] **Magnus Westerlund**: Okay.

[01:38:17] **Oscar**: And so my assumption before looking at this data, knowing that we see so many for classic, we see so many low congestion window as a low SS fresh exits, my assumption would have been that basically for the lower percentiles, this should be one flat line because all of those are Yeah. From con congestion events.

[01:38:43] **Martin Duke**: And the percentile scale is the size of the SS Thresh?

[01:38:48] **Oscar**: Sorry. The central scale?

[01:38:50] **Martin Duke**: The the yeah. The x axis.

[01:38:52] **Oscar**: So the x axis are the percentiles, which would be the size of the SS Fresh.

[01:38:58] **Martin Duke**: Okay.

[01:38:58] **Oscar**: And this is then the percent difference to classic.

[01:39:01] **Martin Duke**: Right. Okay. Thanks.

[01:39:02] **Oscar**: You're welcome. And so, yeah, as I said, yeah, there's some problems with app limited traffic patterns. And may I also to to that?

[01:39:15] **Mirja Kühlewind**: Yeah. Hi. Thanks for doing the work. But I think I agree with Martin that at least some kind of time based metric would actually be useful. I would propose just to look at, like, how long it takes to until you exit slow start normalized by the round trip time. So how long how many round trip time it takes to exit slow start? You should see a difference there as well.

[01:39:33] **Oscar**: Yeah. Yeah. I think in retrospective, that would be a practical metric to have, unfortunately, fast. So our metrics are always aggregated so we can't sort of look at stuff in retrospective. So if we wanted to measure that, then we would kind of have to redo the whole thing. Yeah. Yeah.

[01:39:54] **Mirja Kühlewind**: And like actually, time based metric could also give you some information about the cases where you don't leave slow start, but you might still see an improvement. Mhmm.

[01:40:02] **Oscar**: Yep. That's a good suggestion. Thanks. And yes, so this is basically most of the content. Now I have some questions kind of I would love to hear from others. Does it need to be now but kind of if how your connection patterns as in percent of never growing the congestion window percent exiting slow start and stuff like that. How this is matching up with our experiences. And I would also love to hear opinions or takes on why we are seeing so many low SS fresh congestion events. It's sort of like sub 20 kilobytes. For the last one, I'm gonna make a guess myself, sort of. So I have one theory, which is that we might be seeing actively managed queues. So the theory is that our bottleneck is is the browser, is the home router. And we do actually, over all of our release population, see that, like, 10 to 20% of our congestion events are caused by EC and CE, which means there must be some actively managed queues. Otherwise, we wouldn't be seeing CE markings. And so this maybe could mean that we see congestion events earlier, both the loss and the ECN CE kind because we are operating on other buffers kind of and they are not typically large buffers with tail drop. So, yeah, that's a theory. I would love to hear your takes. And, yeah, if you have any more questions, if you have any opinions on any of this, generally, we also have some more data. Like, this is kind of the tip of the iceberg. Then, yeah, please find me. Ask them now or send me an email.

[01:41:54] **Ian Swett**: Jonathan, just

[01:41:54] **Roland Bless**: for my own understanding, this is stuff sent by Firefox. Right? So you'd expect that the vast majority of what you send is one or just a few get get requests and then sit around waiting for the response. Right? So Yes. That explains why most things never enter never, like, exit slow start?

[01:42:12] **Oscar**: Yes. So of that's very much the case. Like and that's all of those connections probably that never grew their congestion window at all. And so we filtered them out. Yeah. I would also be very interested to know what other browser vendors maybe see if they if their stuff matches up with this.

[01:42:33] **Kazuho Oku**: Yeah. Kazuhok, firstly, I I think the results that being reported here is could be kind of skewed because it only looks at the connections that experience losses, which means that for the connections that had higher BDP, they are less likely to observe losses unless you put pretty much pressure. So that means that you see more among all the connections being established by all the users, the connections that had lower BDP would appear more in these results and that will skew the results. And in case of us, we only have data from the server side sending to the client. But if we compare the results that collect all the results above 200 ks versus one minute, the SSH distribution changes by twice or something. So, numbers that you are looking could be skewed due to that. And I think Matt made a good point, arguing that probably looking at the TTFB, for example, the time until upload completes might be a good metric and I agree with it.

[01:43:50] **Oscar**: Yeah. I would also, like in retrospective, I would love to have something like Time to Last Byte. But, yeah, unfortunately, we didn't at the time think of that. Yeah.

[01:44:02] **Eric Kinnear**: Two more quick comments.

[01:44:04] **Jay**: Yeah. Just to comment on seeing a lot of low SS threshold exits. Basically, I was in in our network, I was actually fighting with that as well. And, basically, one of the reason that we actually seen, you know, the low SS threshold exits are based on, like, active queue management, actually not well configured for the the network condition. Typically, the red random, you know, OLE exit, that uses that it that is actually very much depending upon, like, configuration, and it could be actually not well configured for the situation, in which case it actually caused the burst of, you know, packet errors, packet drops. If even if, like, you know, one guy is actually just, you know, causing a problem, not only that guy, but, like, you know, it actually drops, like, all other guys, which is actually very dinky flows. And and that is one of the thing that we in our network, we try to actually, you know, fix it. So just

[01:45:12] **Martin Duke**: Alright. I'd like to also repeat what everyone is saying. We thank you for doing this work. This is this is a great start. I am I'm really wrestling with the metrics. And, like, it is low is a low slow start exit good or bad? Is is, like, a high risk rate good or bad? And it obviously depends on link conditions, which are unknowable in this kind of experiment. You know, an emulator, it's easy to figure out where the where the IDOxup point is, and you have no idea. I do. And and I and I and I keep thinking, like, well, if I just don't do if I go straight in the congestion avoidance, like, some of these metrics are very good, and that's clearly not tells me maybe the could that, you know, maybe that's not the way to go. I'm I'm wondering if looking at what happens in congestion avoidance is maybe what you want to see, assuming it's a long lived enough connection. If if, like, if congestion avoidance kinda just shoots up in a straight line after that, then you've exited too early. Whereas if you start sawtoothing around the SS thrush point, you've done a good job, like, roughly. So maybe that's how you can approach this problem.

[01:46:19] **Oscar**: So we actually do have those metrics sort of in not not over time, but sort of like looking at the final congestion window of connection maybe.

[01:46:28] **Martin Duke**: Great.

[01:46:29] **Oscar**: Unfortunately, those got heavily skewed by all of those low SS fresh losses that we saw because those also showed up there and were probably not at the right point. And so I don't really trust those for that reason.

[01:46:47] **Martin Duke**: Fair enough. I mean, maybe there's a way to filter that data. I mean, you've thought about more than I have, but thanks. Yep.

[01:46:51] **Eric Kinnear**: Thank you. Wonderful. Thank you very much. Yeah. And that could be a good conversation to continue in the hallway as well. Alright. Up next, we've got Christian. Alright. Christian, I believe you should have slides control, so you should be able to go forwards and backwards. Christian, I see advanced slides, but we can't see or hear you.

[01:47:59] **Christian Huitema**: Good morning, folks. I mean

[01:48:01] **Eric Kinnear**: Good morning.

[01:48:02] **Christian Huitema**: Guys in gold guys in the early morning are a bit dumb, and they they keep being muted when trying to speak. Sorry about that. Now, this is an update on C4. You can look at the updated draft. They are all at the version four. And you can look at the various simulation source code. They are all on the, on GitHub so that you can do that. The the goals of C4, we presented that last time, is to have a very responsive concession control algorithm that, in particular, works well for media applications. And that means media applications have two issues. I mean, if your assist bandwidth is too small, you are going to not send your video frame fast enough, and you will have a bad response time. But if that assessment becomes too large, you are going to send more frames than you should, and you are going to get priority in versions. So, basically, the idea to strike a nice balance between the two. So discussions. One of the feedback I got at the last presentation was about this test results. I mean, we are doing our test by running three dozen different scenarios and running a 100 simulation for each scenario. And so the feedback I had was twofold. It says, first, tell us how that compares to classic algorithm like C4 or b b r like b b r or cubic. And the second was and give us an idea about the distribution of all the test. So what I did there is that all those tests are very variable. They they depends on menu changes in, say, some random generation somewhere can have very large difference. So now for each test, we present both the average of the 100 runs and the worst ninetieth percentile of execution times. So that basically that means that the average tell us how good we are doing an average. The, ninetieth percentile tell us about how we are going statistically in the worst case. It's 90 and not 99% because I'm only doing a 100 simulations. In a letter phase, when we are really happy with what we have, we could do a thousand simulation and go to 99% measurements. But now expressing 99% measurement, we completely random. We present those result in the test draft as a series of tables like the one I have here that show that for most test, the time it took to execute the test, like, the time it the simulated time it took to say download 10 megabytes or something like that. All of those are measured in microseconds, and all of those are of the category lower is better. And we did the test by either to winning C4 as a tested protocol or b b r or cubic so that we can measure the times that we had. I'm happy to report that we are mostly at C4 is doing mostly better than all the competition on all the test except just a few in which case it for. But it's something like 99% better, and I'm happy about that. Now what did we change? We presented C4 at the last ATF, and we had showed that we had a small, very simple state machine. One of the change we did was to make that state machine slightly more complicated. The first change is in the the startup phases to have a separate state for careful resume. We had implemented the careful resume specification, which is basically you remember the cwnd, the congestion window, and the data rate that you had observed last time on the same connection, and you try to reuse it. That was implemented in the previous draft as some kind of a special condition for the startup. And in fact, what we observe is it's much simpler to debug it if it is a step by itself and in which we just try those previous conditions. Of course, exit if we do find any kind of congestion. But, otherwise, started for two RTTs. One RTT to observe the the the direct menu. You can only observe the data at the end of the two RTT. And we stay in that condition for two RTT because if we exit too early, it does impact performance a lot on satellite networks and similar. So it's both an improvement and a simplification. The second change is kind of related to what Ian was Ian and Neil were presenting about the cautionary prob probing in BBR. C four by design started by only doing cautionary probing. We call that pushing, but we it was let's try one RTT in which we push at a higher rate. And then after one RTT, immediately go back to an assessment phase and wait another RTT to understand whether pushing was successful or not. And the problem with that is that it's very cautious, but it's also very slow. So for example, if you got pushing phase every second RTT instead of every RTT, it basically takes you twice longer to recover from a drop in bandwidth and then going to the next bandwidth. So what we did there is the reverse of introducing the cautionary probing is that say, okay. If we think that the next probing will be successful, then we enter a pushing phase in which we do push at 25% until we don't observe any more data growth or conception control event, of course, which give us an exponential growth similar to b b r. And that make us much more responsive to say recover to the right bandwidth after an event. So that's one change that we did. The others another change was the faster response to persistent congestion. In the first season of C4, we had the this idea that, hey, all these measurements are often very noisy. So if we try to just track, for example, the measurement data rate, we let let the noise drop the data rate, and then if we drop it, then it will not raise again until another phase, etcetera. So we say, okay. We we only react to actual congestion event like ECN marks or packet losses or maybe excessive timeouts. But what we have said is that doing that, in the case where we have a link event, like a rerouting event and the bandwidth of the link drops, it give us some kind of a staircase. Like, oh, we try for one RTT. We observe a congestion. We drop by 25%. We have another RTT. We observe congestion. We drop by 25%, etcetera. It's it's staircase that takes several RTTs to adapt to the actual bandwidth that is underlying. So what we changed is that, yes, we do this staircase for the first congestion. But if we have two steps back to back, then we say, oh, something has changed. And in that case, instead of just doing this 25% drop, we just align to the latest bandwidth measurement. So that way, we drop in two activities instead of a cascade of three, four, five. And that makes the the algorithm much more responsive as well. And finally, the next change we do was for the initial startup. C four always does pacing. In the initial startup, we were pacing at twice the nominal data rate as is so far. And what we saw is that that get us in many stuck scenario in which because at the as I said, the time it was too slow, it did not increase enough. And because, I mean, it was and the packets would not stand rapidly enough. And, basically, that was bad for competition in some in a number of scenarios. So the solution was to put the floor on the pacing. It's basically, at least during slow start, don't try to pace at too low rate. And the sending will still be limited. It will limited by the value of the congestion window. But the pacing will allow, say, packet trains to go out, etcetera, and it will be much more robust. For example, if the RTT is smaller than what you have measured so far.

[01:59:07] **Eric Kinnear**: We're bumping up against time, so let's keep moving.

[01:59:10] **Christian Huitema**: And and so we try multiple values, but, basically, we pick the smaller one that we try. We pick spacing doing so side at at least one megabit per second to not get stuck. And it does remove the getting stuck conditions, and it does improve performance overall. So where are we? Well, we are at a point where the results are pretty good. I mean, we as I say, we can look at the test result. And for all the scenarios that we measure, are happy with what C4 does. We either need to add a couple more scenarios. We need to add a serial network simulation to our simulations so that, basically, we can measure what happened when your bandwidth and data rate are constantly changing. We have done an ECS simulation. We probably need to add red and red ECN simulations as well because they they are things. We have, in the simulation, a limit on the bandwidth. I mean, we the buffer the the buffer in the routers are set typically to two BDPs, sometime four. That is not quite testing buffer bloat. We need to have a test case in which the the buffer or something like 20 times BDP and see what happens. We know we have to drive down the latency in complete scenarios. Currently, in complete scenarios, we get to reasonably fair sharing between connections, but that fair sharing is driven by connections backing off in case of congestion. And that means that if we do not have congestion I mean, that means that, basically, they get to the equilibrium by pushing the network to the congested state and having long delays, at least at least one BDP of delay. We I I would like to fix that. It's it's hard to do. It's impossible to do when competing with cubic because cubic will not stop. But in case of competing with congestion control that do measure the data rate and things like that, it should be possible so that that'll be my next effort. And we are starting to deploy the old test, not just simulation, but we need that to, I mean, experiment from life networks. And that's what I have.

[02:02:03] **Eric Kinnear**: Thank you, Christian. I think we've got Neil in the queue for a very quick comment.

[02:02:09] **Neil Cardwell**: Given the time, that that's fine. I'll I'll do another chat. Thanks.

[02:02:14] **Christian Huitema**: Sounds good.

[02:02:15] **Eric Kinnear**: Alright. Thank you all very much, and we will see you on the list.