Session Date/Time: 19 Mar 2026 01:00
Marcus: Good morning to the people in the room and good anything to the people online. This is Thursday morning in Shenzhen, and this is IETF 125, and this is a joint session between BMWG and IPPM. If you were planning to attend something else, you may stay on here and learn something new, or you should find your session.
This is an IETF meeting, so you are bound by the Note Well. All of your contributions are regulated in the text that's here and there's a lot of links to very, very important process documents. If you are not familiar with them, I suggest that you familiarize yourself with them before contributing to this meeting, or any other meeting, or any other IETF contribution that you're trying to do. If you have any questions around this, you can reach out to working group chairs, area directors, or anyone else in the IETF that you might want to discuss this with.
This is a professional meeting. We expect professional conduct. Everybody should treat each other with respect. We want to have a productive environment where disagreements are handled in a civil way. If you believe that you are not treated fairly, that you experience some form of harassment or other things like that, there are ombuds teams that you can find here that you can reach out to and discuss these matters.
Some meeting tips: If you are here in person, please make sure to log into the meeting. You can use the onsite tool from your computer, you can use your phone to scan the QR code to join the onsite tool using your handheld device. For everybody who wants to make comments, please always join the queue on the MeetEcho tool. That also involves if you are—that also applies if you are onsite in the room. And if you're a remote participant, please make sure that your audio and video are off unless you are chairing or presenting during the sessions. It is strongly recommended to use a headset, that's because we sometimes get audio feedback and so on and, yeah, various audio artifacts, so just keep that in mind.
Okay, so this is a yet another joint IPPM and BMWG meeting. We have done this a couple of times now, I think this might be the third time plus an interim. And the reason for doing this is that we want to see if we can facilitate exchange and discussions between these both two working groups. We believe that there is similarity enough in the topics that it actually warrants more joint sessions. It's also so that we are not like the biggest groups in the IETF, so it also kind of is optimized to have joint sessions for a various number of reasons.
And given that this has worked out pretty well, we have started the ideas of producing some form of joint charter, but there will be a 15-minute time at the end of this session to have a discussion to get your feedback. Chen will present that. He has also collected a number of feedback from a number of participants from both working groups and basically we will be discussing various ways forward here. So, please, if you have opinions around this, stick around to the end and really join that discussion. It's very important for us to get all the views from the participants here.
BMWG charter. Do you want to say anything about it?
Giuseppe: Yeah, BMWG charter maybe, yeah, for BMWG people you are already familiar with that. We aim to produce recommendations regarding the key performance characteristics of services, devices, and systems. The set of relevant benchmarks, of course, will be developed with input from the community of users, so network operators and testing organizations, and of course affect the network test equipment and manufacturers. The scope, of course, is limited to lab environment. This is the main difference with IPPM.
Marcus: Right. And then we have an IPPM charter and, yeah, the most important thing is what we have in boldface here is that—develop and maintain standard metrics that can be applied to the quality performance and reliability of internet data delivery services and applications running over transport protocols, over IP. So the really main distinction here between IPPM and Benchmarking Working Group is IPPM is working on defining metrics, methodologies, and protocols for measuring operational networks, you can say.
We do need a note-taker. I don't know if we had one already. If anybody wants to contribute to this effort by helping out taking notes, that would be tremendously useful. The most important thing with taking notes is to capture action points. You don't need to transcribe. We have really good tools for transcribing sessions, but it's really nice to have somebody just, you know, picking up the most important points that are made and... Oh, Stuart, thank you very much. That's greatly appreciated.
Then we have an agenda, and we will be—we will be mixing up both IPPM and BMWG topics. We are right now at the welcome Note Well, chairs' slides, pretty much. And then we will have Justin talking about Integrity YANG. I think this is a very important presentation because it's a document that is already in working group last call, and we need more feedback on this one. So please pay attention. Then we have SAV benchmarking, we have this ECN class of service draft for STAMP, we have power bench benchmarking, we have STAMP extended headers, STAMP bit explicit errors, AltMark YANG, on-path telemetry YANG—a lot of working group documents as you see. We are two working groups, we work on a lot of things. So we have a lot of—and we have a suite of presentations from Giuseppe at the end as well.
Moving forward, we also have a lot of proposed work. We're happy that we are able to accommodate quite a lot of presentations on proposed works. So we will have some work on direct export, incorporating alternate marking to IOAM, we have a number of presentations on AI fabric benchmarking terminology, etc., more benchmarking, packet triggered reporting, and we have the IP measurement option by Jared. Do we have any agenda bash here?
Stuart.
Stuart Cheshire: I don't know whether you saw it, I did send some slides last night for the RPM draft. It's very brief, but I think I should update the working group.
Marcus: Yeah, sure. I think that should be fine. Let's see if we have them in the data tracker. We'll make sure to fix that. So we can put you in the end of the adopted documents then session, yeah.
Right. And finally, we have the joint session feedback as well. So at the end of this meeting we will have a roughly 15-minute slot. Chen will be presenting the work he has done on collecting feedback from working group members and discuss various ways forward in this. And then we will have a time for discussion and we will be also asking a few questions here. So at the end of this session, we would like you to consider these four questions, and we will repeat them again at the end. So basically: Was there enough time for discussions? And also for your interested topics, was this time well spent? Also, should we allocate more time to proposed work? And also, did you cross-review one or more documents between these working groups? And then we see Scott in the queue here.
Scott: Laptop issues. This is a very good thing to see. So, do you have a place on the agenda to discuss the liaison that was discussed in the email? There was some email about there was a liaison to Study Group 13 that there's been back and forth work between SRv6 and BMWG on a topic and going back and forth, so it's just—it looks like they've already addressed our comments that we've sent back, but if there's anyone that is interested in continuing to review what they've done, I just wanted to raise that as something. I can put a—once my connection comes back, I can put a link to the liaison in the chat window. I don't think we really need to reply or anything, but that's it. Thanks.
Giuseppe: Thank you, Scott. I would say for raising this point, I think, yeah, we—the response we received from them, they confirmed that actually they have taken into account the comment that we sent back to them. And unless I am mistaken, Giuseppe, we have checked, I would say, the latest version they sent. I don't think that there is something more that we can tell them. I think at this stage is just, yeah, we can just say thank you, thank you for letting us know.
Yes, but as you are mentioning, actually, the liaison statement, there is another one which is related to the work done in IPPM, by the way. And for that one, we didn't send a formal liaison statement because we don't have—because of the agenda of the meeting, so it's Rudiger actually who is the main contact point there. I don't remember, I don't have in mind, I would say, the group there. So all the discussion has been had on the mailing list. So the responsible officer of that working group in the ITU came to the IPPM mailing list, he shared an extensive review there, and the authors of the—this is actually related to the Quality of Outcome specifications. So that comments were really integrated into the specifications. He was really happy with the version. And this is the way I would really like to encourage the liaisons. It's not just, I would say, sending a dry and, I would say, text there, but just come and have the discussion and take into account.
Marcus: Yeah, I really want to second that. I think that was a textbook example of really good communication between the organizations.
Scott: And I will third that because that is what we try to encourage, is get people together on a mailing list or just communicating, and that's how you get work done. Thanks.
Marcus: Alright. And then one more thing here. So you might have seen it on the mailing list already, but we're actually running a joint adoption call for a document that is—it is an MPLS document, but we're running a joint MPLS, IPPM, BMWG adoption call because we believe that this is highly related to work that is being done in the IPPM and BMWG. So it's important to collect feedback also from the groups here. So it is this MPLS on-path telemetry network action flag for OAM. You can read the abstract here, and it's really so that it would be excellent to get feedback from both IPPM and BMWG communities because it touches on subjects that are highly related to work we're doing here on telemetry. Mohammed?
Mohammed: Exactly what Marcus said. I really, I would say, would like that, yeah, that you have some time to look into this document because actually there are parts that can be reusable and not just specific to MPLS. And we really need to separate these aspects that are common into... and independent of the solution that will be put in place, so that we can have those really, I would say, specified in a manner that other, I would say, implementation can just easily refer to that part. And this is actually one of the missions that IPPM, I would say, is more suited for, is to define these common blocks. And, yeah, please review that document and chime in and share in the mailing list.
Giuseppe: Yeah, in terms of updates for BMWG documents since last IETF: We have the MLRsearch still in the RFC Editor queue, but everything should be fine. The YANG model has passed the working group last call, but we are expecting a new revision to address the YANG doctors' review, and I just noticed this morning the new version published by Vladimir. Also the SR benchmarking methodology, the draft that is involved in the liaison mentioned before, passed the working group last call. Other two documents on SAVNET benchmarking and power benchmarking have been adopted. As other updates, we aim to resume the discussion for revisiting the BMWG RFCs, to update the old RFC 2544, and maybe we can consider some re-chartering, either for a joint future with IPPM or separately regarding the new work that is coming. So that's all.
Regarding the SR benchmarking methodology, I would like Paulo if he can provide a quick update about the working group last call, yeah.
Paulo: Sure, hello. Paulo, Huawei. Very quickly, this is the current status of the benchmarking methodology. So basically, Giuseppe pointed out we had the last call the past month. I think the result was positive, so thanks to the chairs for the support. We received some last comments. They are listed here, basically. I don't need to go through that. I would say the most important is probably the second one. We—I mean, not just the co-authors, but I would say the community probably missed one important parameter in the appendix where we define the reporting template for tests. Basically, to provide the details on the hashing mechanism used. I would say this is it. So thanks for commenting and providing feedback on the draft.
Giuseppe: Okay, I would expect a new version soon, so we can move on. Next slide.
Marcus: Alright, so a few IPPM document updates since last meeting as well. So the draft-ietf-ippm-asymmetrical-packets have received a number of IESG comments, good comments, and very swift handling of the authors, so they have been addressed in a recent version. Quality of Outcome has been receiving reviews from TSVWG, ART-SEC, GEN-ART, etc. Authors are currently working on addressing these comments. And as discussed as well, there has been some liaison interactions as well that have been very useful.
Integrity YANG is during a currently running working group last call. We need more feedback from both working groups because we really want to be able to progress this document together with the IOAM data integrity, which has been progressed from the working group already. And because they are very tightly related documents. So there will be a presentation here by Justin and, yeah, we really urge more feedback on this.
When it comes to encrypted PDMv2, authors published number 13 of this draft and intend to reach out to the previous reviewers from TSVWG and GEN-ART to clarify whether the recent document update addresses their comments and also request more feedback in general from the working groups here and also from 6MAN.
Yeah, here are the resources on... Med, please.
Mohammed: Just on the previous one. I don't know if there is any of the authors of the Two-Step specification here in the room. Just, I would like to hear some... because I am waiting for a revised version for almost six months. So just to let us know what is the plan with that specification.
Marcus: I see Greg connected. I don't know if he wants to say something.
Alright, yeah, we'll definitely make sure to follow up with the authors. Move on here. Alright, and let's go to Integrity YANG.
Justin: Morning, can you hear me?
Marcus: Yes. Do you want slides control?
Justin: Uh, yeah.
Marcus: You should have it now.
Justin: Awesome. Thank you very much. Um, so yeah, uh, good morning guys. Uh, 2:00 AM over here, so, uh, yeah, a little bit, you know, sleepy. Sorry for that. Um, so yeah, uh, thank you Marcus. You pretty much summarized the situation for this draft already. Um, so hopefully this is gonna be the last update of this one, um, which is basically a YANG model to, uh, support integrity protected options for IOAM.
Um, so just as a reminder, basically this draft, this model augments profiles that are defined in the IETF IOAM model from RFC 9617. Um, those integrity protected options are defined in a companion draft which was just progressed, you know, to the IESG and which passed TSVWG review recently. Um, so yeah, the goal obviously is to progress both at the same time. Um, this one is currently in a working group last call. Um, so yeah, we'd like to collect more feedback if possible.
Um, this draft was presented last time in IETF 123. Um, and, uh, we applied some changes since then, mainly thanks to Med regarding, you know, operational considerations. We also added an IANA maintained module to have an indirection, you know, um, to reflect the registry with the code types. Um, and there were also so many, you know, editorial changes as well. And recently this one passed an early YANG doctors review, which is a good sign of maturity too. Um, so yeah, we just want to collect more feedback and, and hopefully we could progress too. Um, so yeah. Thank you.
Marcus: Thank you. Do we have any comments right now from the floor? Or online? If not, thank you very much for this work. The working group last call is going on until tomorrow. Uh, we will of course be able to extend it if we don't get enough reviews, but we would really like to urge you to, if you haven't had a look at it yet, it's—it's in a good state, the document, as we see this past YANG doctor's review. So, so we need the feedback just so we can kind of progress these two. And this work on integrity for IOAM has been a long-standing work, there's been a lot of work going into it and we're finally kind of very close to, to putting this behind us. And yeah, I also want to thank you, Justin, for all the work you've been doing around this. It's been several years and a lot of hard work, so, yeah. Thanks for that as well.
Mohammed: Exactly. I would second what Marcus has said about the work from Justin but also about that this is just the last mile so that we push the pieces to the pipe there. Just for the information, we have the other document which is the base one which, I would say, this one is trying to complement. It is currently waiting for the IESG review and I have scheduled it by the end of April, so that we are waiting for this one so that we can have them both as a cluster to ease reviews and all of those. So, um, I would really appreciate if the people can have some more eyes on this document. It is stable as mentioned by Justin by the review, for example, of YANG doctors and so on, but having more eyes is really, is really helpful.
Justin: Yeah, thanks Med.
Marcus: All right, thank you very much Justin.
Libin Liu: Good morning everyone, I'm Libin Liu from Zhongguancun Laboratory. Today on behalf of co-authors, I will introduce the updates on the benchmarking methodology for intra-domain and inter-domain source address validation. [Presentation 3: https://datatracker.ietf.org/meeting/125/materials/slides-125-ippm-the-updates-on-the-benchmarking-methodology-for-intra-domain-and-inter-domain-source-address-validation-00]
So let's first overview the document. This document is mainly about bench—proposed the benchmarking methodology for intra-domain and inter-domain SAV, which evaluates the accuracy in terms of false positive and false negative rates, and the control plane and data plane performance of SAV, and adopts a black-box and mechanism-agnostic approach. During the IETF 124 meeting, we presented the version 08 and after that our document was adopted by the working group before this meeting. And based on the comments we received, we revised the document and uploaded the working group 01 today. I will present this version.
Before introducing the comments and updates we do, and we would like to clarify the SAV scope clarification discussed in the SAVNET workgroup. The SAVNET workgroup has the consensus that the intra-domain and inter-domain SAV should be distinguished based on whether the interface on which the SAV mechanisms operates is connected to a BGP neighbor. Intra-domain SAV operates at the non-BGP interfaces while inter-domain SAV operates at the AS-to-AS interface that carry EBGP sessions. I think this is important for understanding the benchmarking scenarios proposed in this document.
So, here is a summary of the comments. There are three types. The first one is the comment on the definitions of SAV performance indicators. Second one is the comment on the test cases of intra-domain SAV, from Xueyan. Xueyan suggested that the IPv6 should be used in the test cases. And the third type is about a suggestion on testing resource utilization. Shengnan shared that the peak CPU utilization differs over time, but long term is stable. Thank you for the sharing.
And this is a summary of the main updates. We have revised the definitions of false positive and false negative rates and we also made all the test cases for intra-domain SAV benchmarking using IPv6 and also revised the corresponding text. And we have revised the figures for describing the DSR test cases for inter-domain SAV benchmarking, and we added a recommendation for the continuous CPU and memory utilization recording.
Okay, for the definitions of FPR and FNR, we have currently FPR is refined—is revised to refer to the proportion of legitimate packets that are incorrectly classified as spoofed by the DUT to the total number of legitimate packets sent to the DUT, while the FNR refer to the proportion of spoofed packets that are incorrectly classified to legitimate by the DUT to the total number of spoofed packets sent to the DUT. Also in the document we added a statement that other computation methods may be used for the supplementary analysis but are outside the scope of this document.
Okay, in the version 01, we revised the figures for describing the DSR test cases. In the new figures, AS3 will use the prefix P3 as the unicast prefix and use prefix 7 as the anycast prefix. AS3 will advertise prefix P3 and P7 to AS2 via the DUT using BGP. Here AS1 can use the prefix P7 to send source prefix—to send traffic as the source address to send traffic, but it doesn't advertise it using BGP.
Okay, many thanks to all of you for the valuable comments and reviews on this document. So for the next steps, we will improve the clarity and completeness of the test procedures, and we will continue the discussion in BMWG and SAVNET workgroup, and we'll revise the document based on the workgroup feedback. Thank you very much. Any comments?
Giuseppe: Comments, questions? Okay, thank you.
Shaoping: Hello everyone, I'm Shaoping from ZTE. This presentation is on the update of the STAMP CoS extension for ECN. [Presentation 4: https://datatracker.ietf.org/meeting/125/materials/slides-125-ippm-update-of-the-stamp-cos-extension-for-ecn-00] The primary author and editor of this draft is Greg White. He joined this session remotely, so I present this draft on behalf of all the authors.
This draft defines the STAMP extension TLV to enable bidirectional monitoring of ECN traversal. It can be used to detect ECN field manipulation in each direction, and it can be also used to perform differential measurement of path latency based on ECN value. Existing Class-of-Service TLV enables bidirectional DSCP monitoring but only reliably enables downstream ECN monitoring. So this draft is needed.
There were two individual drafts were proposed in the IETF 122 timeframe. And with the discussion among the mailing list, the two individual drafts are merged. In the IETF 124 timeframe, the merged individual drafts was adopted by the IPPM working group.
This slide shows the previous Class-of-Service TLV defined in RFC 8972. In this TLV, there were four fields: DSCP 1, DSCP 2, ECN, and RP.
This is the format of a new Class-of-Service TLV defined in this draft. There are two newly added fields in this TLV: ECN 1 and RPE. ECN 1 is the ECN value intended by the session sender to be used as the ECN value of the reflected test packet. RPE is a 2-bit field indicating whether the session reflector use ECN 1 as the ECN value of the reflected test packet.
This is the detailed definition for RPD and RPE fields. To make the newly defined TLV backward compatible, the RPD and RPE have different syntax.
During the adoption call for this draft, there was an alternative proposal. A new TLV, CoS V2, was proposed. This slide shows the formats of the newly proposed alternative TLV. In this newly proposed TLV, there are DSCP A, B, C, D and EC A, B, C, D. And DSCP A and EC A indicate intended forward DSCP plus ECN values set by the session sender. DSCP B and EC B indicate received forward DSCP plus ECN values. DSCP C and EC C indicate intended reverse DSCP plus ECN values. And DSCP D and EC D indicate applied reverse DSCP plus ECN values. So that's the new proposal.
There are some pros and cons of the newly proposed CoS V2 TLV. Pros: First one, it's self-contained measurement record. Session sender does not need to maintain outbound information, and on-path observer can understand the exchange. Second one, when DSCP or ECN request from session sender is not followed, session reflector can indicate exactly which return value is used, as opposed to just a flag in the current draft. Third one, it's better alignment with IP header field ordering. Cons: First one, it's not backward compatible with original CoS TLV defined in RFC 8972. Session sender would either need state to know which type to use, or send both TLV and define session reflector behavior upon receiving two TLVs. Session reflector might need to support both TLVs. And second one, there's no reserved bits for future extension of this TLV. Third one, it consumes another TLV type because it's a newly defined TLV.
So next steps, working group needs to decide whether to switch to CoS V2, newly proposed TLV. And there's already an implementation of the current definition in GitHub. There may be an interoperability testing at IETF 126 Hackathon. So that's it. Thank you. Any comments?
Greg White: Hi, this is Greg White. Yeah, main author of this draft. Yeah, we really would like feedback from the working group on this. There are obviously some pros and cons that we identified in these slides, and it isn't really clear to us which direction to go. So definitely would like people to have a look at this, particularly if you already have a STAMP implementation and maybe have a STAMP Class-of-Service TLV implementation, thinking about the complexity of supporting both versus an upgrade to the existing one. Any feedback along those lines would be welcome. So yeah, really looking for the working group to help us out on making a decision here. Thanks.
Marcus: Yeah, thank you Greg. Yeah, that would be really, really useful feedback, especially if you have implementations, you have experience of this particular, the original TLV would be great, would be good to understand things like not having spare bits as extension points, if that's a problem, or if—if this would be something okay if you're concerned by backward compatibility. I would also like to say it's very great to see implementations, it's very encouraging to see that you will do some interoperability testing at the next Hackathon.
Giuseppe: Confirmed.
Marcus: Okay, okay, yeah, but that's really great to see, something we very much encourage here in the group. Thank you very much. Next up we will have power benchmarking.
Shailesh: Yes, am I audible? Just checking the audio. [Presentation 5: https://datatracker.ietf.org/meeting/125/materials/slides-125-ippm-updates-to-characterization-and-benchmarking-methodology-for-power-in-networking-devices-00] Okay, so good morning everyone. My name is Shailesh, and on behalf of the co-authors and contributors of this draft, I'd like to present an update to the draft on characterization and benchmarking methodology of power in networking devices. So this is an adopted working group draft, so it's an ongoing work. So what I'll do is I'll only focus on the recent updates that went into the 01 version of the draft rather than covering the entire document, right.
Right, so for folks who are new to this draft, I'd like to give a quick context and recap of how this draft has evolved. So the draft focuses on proposing a standardized methodology to evaluate the energy efficiency across networking devices. The whole idea is to come up with the device-level power measurement techniques under reproducible and controlled traffic conditions, right. And that enables us to compare energy efficiency across devices and also know how each system component or subsystems contribute to the overall energy efficiency, right. And yeah, as I said, this was adopted by the Benchmarking Methodology Working Group and it has been presented since, you know, IETF 119 meeting and has evolved based on the working group feedback.
So the first update focuses on the measurement stability and reporting, right. So one of the issues that was identified is that the power measurements may capture transient states, right? For example, low load transitions or system adaptation phases, right? So because of these transient states, there can be inconsistency in the results, right? And those results can be incomparable across systems, right? And that is why this draft mandates that the power measurement should also be taken after the system has reached a stable operating state, right? And as part of it, the draft also mandates that the following intervals need to be reported: The first one is the stabilization interval, which is the time taken to reach that stable operating state, and the second is the averaging interval, which is the interval during which the power measurement is actually reported, measured, right? So the whole goal is to improve reproducibility and comparability across test environments, and the measurement of these intervals will ensure that there's transparency in the way the power is measured, right?
So the second update is on the idle and idle-plus operational conditions, right? So in some systems there might be ambiguity in how the idle and idle-plus states are interpreted across implementations, right? For example, there can be the control plane activities that are running or background system processes that are running, even when there are no user traffic, right? So so that might again lead into inconsistency in the results across systems. So the draft acknowledges that now and it also mandates that the operational conditions must also be explicitly reported, right? So the intent is not to constrain the way devices operate here, but just to ensure that the conditions under which measurements are taken are clearly documented, right? So that we don't compare apples to oranges, right? And it avoids ambiguity and ensures consistent interpretation of the results.
The third update that went in is a set of updates. So firstly it adds an additional objective called the self-referential benchmarking, which is basically evaluating the system related to itself under different operational conditions, instead of comparing it to other systems, right. And it also adds a throughput clarification and introduces an overall throughput alongside the weighted throughput, which facilitates, you know, the energy efficiency ratio calculation. And finally we have also added the IMIX traffic, which is widely used in practice, right. So once again, the goal here is to better align with the realistic workloads and deployment scenarios.
So apart from the major updates, there are some minor editorial improvements that happened. Basically, you know, some of the terminology consistency across the document and BCP 14 keyword consistency across the document, right, and some other minor clarity refinements. And one more minor update is that, you know, I have joined the work starting this revision of the draft.
Okay, as the next steps, I would request the working group to review the updates that went into this version of the draft. And I also want to take this opportunity to check with the audience if we have any specific volunteers who can review the draft and provide comments on the mailing list. And I mean, depending on the level of consensus, I also wanted to check with the chairs to assess the readiness for the working group last call, right. So that's all from my side. Thank you.
Giuseppe: Yeah, thank you Shailesh. Also just one note: this document is also made in collaboration with GREEN working group, so yeah, as a chair, I would like to highlight that we need to be in sync with the metrics that they are defining there. So I don't know if there are other comments, questions.
Carsten: Carsten Rossenhoevel. Sorry, it took me a moment to add myself to the queue. Yeah, I think this is excellent work. I'm—I'm willing to review it. I'm just not sure if it's a little bit too early to ask for working group last call or if it's just the day or to get prepared even for working group last call. But I mean, nevertheless it doesn't hurt asking, right. I think what might need to be covered is a section on the test tools, because I see that a lot of people measure power in very trivial ways and that's not always appropriate, but maybe it is, so I think we should at least have a discussion. So I would welcome any suggestion from the co-authors about test tools. And then I think generally probably there's much more work behind, but of course it doesn't hurt to speed it up a little bit and not spend like years with defining this because it's a very urgent topic. So thank you very much for the hard work.
Giuseppe: Thank you, Carsten. Yeah, regarding the working group last call, of course it's maybe too early. Yeah, indeed we—that's why I mentioned to be in sync with the GREEN working group before we are going ahead. And in the end, it was just adopted, so it's too early for the last call, of course.
Shailesh: Sure.
Marcus: Okay, thank you very much. Next up is Rakesh, but I'm not sure, he had an overlap so... we could also shuffle around the agenda a little bit if—if... actually if Rakesh is not here now and he wants to join later, I would actually suggest that Stewart does his brief update on—on the responsiveness, if that's okay for you, Stewart.
Stuart Cheshire: This is really quick. I'm really here as an apology. We have not done any work on this since last time. Christoph Paasch has moved on to another job, he's not working at Apple. We wish him well with that. Randall has been busy picking up the slack with Christoph's departure. So I didn't want to dodge responsibility, I'm here to say sorry, we dropped the ball on this, we will be picking that up. Thanks for the reminder from Thomas that we need to get back to work on this. If anybody's interested in being a co-author, this opens up an opportunity for that. And as a reminder, the two issues we need to address that were discovered in testing of L4S is: in our goal to get a quick result, it's a bit too aggressive with adding new connections and the result is that even when the L4S bottleneck is signaling that congestion control should slow down, we are adding new connections faster than it is slowing down, which results in us overflowing the queue and having packet loss, which is kind of shooting ourselves in the foot. And the other result we found was that at lower data rates, the amount of on-device buffering is significant and that skews the results. So those are the things we need to work on both in code and in the document, because of course we don't want to document things we have not implemented and tested. And these issues were discovered as a result of testing the running code. So sorry I don't have more to report, but we will do better next time. Thank you.
Marcus: No problems. Actually one of these issues is—is quite interesting, like the aggressive addition of parallel connections and the resulting behavior. That is a result in itself to discover that, so I think it would be good to not just like remove or like, it would be good to have some discussion on that and discuss why you're like limiting this and because I think it is an interesting result and definitely worth documenting.
Stuart Cheshire: That's a good observation. Yes, in addition to us fixing the code, I think we should have a paragraph in the document that explains why spraying lots of parallel connections can be counterproductive.
Marcus: Thank you. And if we don't—still don't have Rakesh in the room here, we move on in the agenda. We will have Giuseppe's show with a number of documents.
Giuseppe: Yeah, good morning everyone. This is a quick update about this family of drafts about the alternate marking and on-path telemetry. [Presentation 8: https://datatracker.ietf.org/meeting/125/materials/slides-125-ippm-altmark-and-on-path-telemetry-drafts-00] Yeah, very quickly, these drafts are all stable, let's say, and we are wishing to consider the working group last call. The first draft is about the alternate marking deployment framework. So maybe you are familiar with the alternate marking methodology. All the data plane documents are in standards track and have been published since a few years. Now what we are doing is to complete the whole framework, just to close the—these alternate marking family documents. In particular, we are defining a bunch of documents for the configuration aspects: YANG model, PCEP, BGP; and for data export, in particular IPFIX YANG push, in order to provide tools for the manageability of this methodology.
So the scope of this document is to define the domain, deployment domain, basically a controlled domain for security, incompatibility reasons, define the rule of the nodes, the type of measurements that can be done, and so on. So the framework—the deployment framework draft gives—provides the overview and reference to all the relevant documents.
Another related draft is about the YANG model. This is for configuration. This draft is quite straightforward, simply define the YANG data model for the alternate marking. The tree structure and related information are quite aligned with the RFC 9617 about the IOAM YANG data model in order to, you know, keep the same structure and for easy the implementation. Okay, the YANG model is quite simple, is shown here. You can—you can see all the related information about the method type, protocol type, node action, measurement period, flow identification, measurement mode, and so on.
The third draft is about the on-path telemetry YANG data model. This draft defines the YANG model for monitoring on-path telemetry information and it includes both the alternate marking and the IOAM related information. So with this draft, we want to, let's say, fill the gap for the IOAM YANG model that is missing the telemetry part. That's why we call this on-path telemetry instead of only alternate marking YANG data model. There is the alternate marking part, the IOAM part, and we also include the on-path delay in line with the IPFIX on-path delay draft that is in last call, I think, in OPSAWG. Maybe Thomas can remind me. Okay. So we include all these aspects related to telemetry. Of course, the application scenario is the RFC 8641 about the subscription model for network telemetry.
The last but not least is related—the application of on-path telemetry for active performance measurement. This is quite related to the draft presented by Xiaomi before. It aims to generalize the scenarios where we want to apply the on-path telemetry for hop-by-hop active measurement. We know that active measurement are mainly used for end-to-end measurement, but by leveraging, you know, the use of on-path telemetry together with these active tools, we can, let's say, implement also hop-by-hop measurement beyond end-to-end measurement. So active test packets can be used in combination with hybrid methods, they can be IOAM, alternate marking, or whatever, to leverage that and to perform on-path active measurement. So you can see the case of IPv6 and MPLS, for example. So with the IPv6 header, we have the options that is together the ICMPv6 or the OWAMP, TWAMP, STAMP packet together with the hop-by-hop option that carries the on-path telemetry YANG. The same applies for the MPLS header. This draft is just informational and even if the MPLS MNA sub-stack has not yet, let's say, published as RFC, anyway this is informational, so it doesn't matter this part. For regarding the next steps, I—we believe that these drafts are ready for working group last call, in particular the deployment framework together with the YANG models complete all the alternate marking documentation. For the on-path telemetry application for active performance measurement, this is general and also is referenced by all other drafts using this mechanism, like the STAMP extension header. Comments are always welcome and, yeah, we are willing—the authors are willing to, yeah, request the working group last call for these. I don't know...
Marcus: I noted the request on the working group last call and just one brief mention to the group in the room: so there is another document on the IPFIX alternate marking currently being discussed at OPSAWG. Please also have a look at there. So before we are moving into the working group last call. Thanks. Comments? Do you want to start the call, Giuseppe?
Giuseppe: Yeah, we could run a little poll just to see how... I think we're going to start checking how many people have read any or all of these documents. So just—I'll use the show of hands tool.
Just want to add one thing regarding the IPFIX documents. Of course, it is aligned with the YANG push with the YANG push document here, so the parameters are—are the same. We are just discussing now with Benoit how to have a full alignment there.
Marcus: All right, I think it's starting to stabilize. We have six people who have read the presented documents, four people who have not. So we—we do have some potentials for input here. That's very good. So if you have read these documents and you have opinions on them, it's really useful to provide your feedback. And if you haven't read it and you're interested in it, please do so now. This is, yeah, set of documents that—that would be good to also kind of progress jointly as we move forward. And now we see that Rakesh is here, so we'll move into your documents.
Rakesh: Good morning everyone, my name is Rakesh Gandhi. Presenting the draft STAMP extension for reflecting IP headers on behalf of my co-authors. [Presentation 6: https://datatracker.ietf.org/meeting/125/materials/slides-125-ippm-stamp-extensions-for-reflecting-stamp-packet-ip-headers-00] So we look at the requirements, we did recent updates to the draft, the summary of the procedures, the implementation status, and the next steps.
So requirements is basically using STAMP for hop-by-hop and H-to-H measurements for one-way and two-way modes. The goal is to use the existing implementation of the IPv6 extension headers, for example. So because midpoint is not aware of STAMP protocol, the scope is using STAMP and STAMP TLVs for IPv6 extension headers and IPv4 and IPv6 headers.
Just a recall that we had a working group document, STAMP extension header. We split the draft into two drafts. This one is the IP data plane draft and there is another draft for MPLS data plane. We presented that draft in Montreal for MPLS data plane and we also presented that in MPLS working group this week on Monday to get the feedback from the MPLS working group there. Um, for the IP extension header draft, we receive a good comments from William regarding various clarifications as William was implementing this in open source. We also received lots of good comments from Footer about clarifications and we have updated the draft to address all of these comments. Just to summarize, a basic idea of the draft is that there are extension headers carried by STAMP, just like IOAM extension headers, for example, for let's say if you're doing a per-hop latency measurement. So on the reflector side, STAMP will copy those extension headers into STAMP TLV and send it back to the sender.
So there is an open-source implementation for this. Many thanks to William for reading the draft and implementing it in open source. So it's in pretty good shape right now. So we welcome your review comments and suggestions and we do think the draft is ready for working group last call. Many thanks.
Marcus: Yeah, thank you very much. First of all, yeah, the adoption request is noted and, yeah, we are very happy to see implementations. Just one question: Do you have interoperability experience already, or is it something you're planning?
Rakesh: We haven't done interoperability yet. So this is something because we have a open source as well as an IOS-XR implementation. Next step is to—so William has volunteered to do the testing.
Marcus: All right. Sounds like a candidate for Hackathon project, maybe, or...
Rakesh: Yeah, that's a good idea. Yeah, definitely.
Marcus: Excellent. And as always, a huge thanks to Will, who is providing so much support for STAMP development with this open source implementation. Do we have anyone commenting else from the floor here? Or from online?
Carsten: So, thanks, yeah, very interesting. Carsten Rossenhoevel. By the way, we would welcome to include this kind of test in our annual multi-vendor test event for commercial implementations primarily, but of course it's good to try it out in the Hackathon too. For what kind of bit error rates is this practical given the number of STAMP packets typically exchanged on the connection? Is it practical more for access or also for the lower bit error rates expected in backbone links?
Rakesh: Wait, I think you're talking about the next draft, right?
Carsten: Oh, okay. Let's do that next then.
Marcus: Yeah, thank you. Let's do a similar poll here to see the level of... all right, I think it's starting to stabilize. I think we see similar numbers as in the previous suite of documents. So I'd like to say the same thing here. If you've read it, please review it. If you haven't and you care about this, or you care about IPPM/BMWG documents in general, please read and review and it also helps us with scheduling last calls and so on.
Next up, Rakesh again.
Rakesh: Good morning everyone. My name is Rakesh Gandhi from Cisco Systems. I'm presenting the draft on STAMP extensions for residual BER measurement on behalf of my co-authors. [Presentation 7: https://datatracker.ietf.org/meeting/125/materials/slides-125-ippm-simple-two-way-active-measurement-protocol-stamp-extensions-for-residual-bit-error-rate-measurement-00]
So in agenda, we look at the requirements, the updates we have made since the last time it was presented, the summary of the draft, implementation status, and the next steps. So requirement is to measure the residual bit error rate. So it's the bit error that escapes CRC detection and FEC corrections and results in L3-L4 bit errors like UDP checksum failures, measure both in forward and reverse directions, and our goal is to measure very good approximation of the bit error rate using STAMP and STAMP extensions.
So we welcome Footer as a co-author. There was review done by Footer and Lee Zhang, went through the draft, provided very good comments and clarifications, so we updated the draft to address their comments. Just to give a summary of the procedure that's in the draft: idea is that use the existing the padding TLV that's defined in 8972 with a known bit pattern. And basically, reflector will check the padding with the expected bit pattern and see if there is any mismatch, count the number of bits, put it in a some TLV, and send it back to the sender. And that's how we we measure the the BER, the residual BER rate.
So there is an open-source implementation of this draft. Many thanks to William for implementing it, providing comments and cleaning up the document. There is also another implementation of this draft now in IOS-XR. We are shipping this feature and it will be deployed. Unfortunately, because this is still a working—still an individual draft, we use private code points to ship the product. But we are hoping that there is working group adoption and we get a IANA code points early allocation for it. So otherwise, it will continue to be deployed using private code points in the network. So next step, we do welcome your review comments and suggestions and requesting working group adoption since there have been several implementations of it. It's in good shape for the next steps. Thank you.
Marcus: Yeah, thank you very much. First of all, yeah, the adoption request is noted and, yeah, we are very happy to see implementations. Do you have interoperability experience already or is it something you're planning?
Rakesh: We haven't done interoperability yet. So this is something because we have a open source as well as an IOS-XR implementation. Next step is to—so William has volunteered to do the testing.
Marcus: All right. Sounds like a candidate for Hackathon project, maybe, or...
Rakesh: Yeah, that's a good idea. Yeah, definitely.
Marcus: Excellent. And as always, a huge thanks to Will, who is providing so much support for STAMP development with this open source implementation. Do we have anyone commenting else from the floor here?
Carsten: So thanks, yeah, very interesting. Carsten Rossenhoevel. By the way, we would welcome to include this kind of test in our annual multi-vendor test event for commercial implementations primarily, but of course it's good to try it out in the Hackathon too. For what kind of bit error rates is this practical given the number of STAMP packets typically exchanged on the connection? Is it practical more for access or also for the lower bit error rates expected in backbone links?
Rakesh: So this is... the approximation is good approximation is achieved by how big the packet size is in padding. We recommend for link MTU size. So you can go 9k packet for STAMP with padding. And then also it's how many packets the sender is able to send and receiver is able to process. Is it a CPU-based, or is it, you know, some other implementation that can do very fast? So idea—so it is for the... so we have customers who will be looking at... so they have a lease lines fibers and they experiencing the bit error on those lines and they will be using those for SLA purpose. So this is in the backbone of the of the network.
Carsten: Okay. Thank you. And is there any differentiation in the reporting for burst loss or packet error bursts versus like infrequent bit errors happening?
Rakesh: That's really a good thing about the way it's done is that you know exactly how many bits were in errors. And the way it's defined, those are TLVs. So if you need additional statistics to see that how many consecutive bits were in error, just the way you mentioned burst, it can be added as well. So if there is an interest, we can extend and add another sub-TLV where reflector can say the there was a burst of I don't know say 2000 bits in a row. So if you are interested in that, we can talk, connect offline and maybe do the extension if needed.
Carsten: Okay. Thank you. And last question: Isn't it kind of a conflict that you hope that the bit errors will happen in the padding, but not in the packet header, because then the whole STAMP packet would be deleted?
Rakesh: Yeah, so there is... so the pattern is carried either as part of the STAMP packet, or it can be a fixed well-known pattern, or it can be a pattern that's configured on the reflector. So make sure that pattern itself doesn't have a bit error causing the miscalculations. But if the STAMP itself, if has a errors, for example, IP header is corrected—corrupted or some other, then that's the existing STAMP behavior. So this draft doesn't really change the existing behavior of what happens. Maybe packet will be dropped or... so our focus is really to see what happens in the padding.
Carsten: Okay. Thank you.
Marcus: Thanks a lot, Rakesh. Thank you. All right, and let's go into... wait, we do the poll here, see the level of... All right, I think it's starting to stabilize. I think we see similar numbers as in the previous suite of documents. So I'd like to say the same thing here. If you've read it, please review it. If you haven't and you care about this, or you care about IPPM/BMWG documents in general, please read and review and it also helps us with scheduling last calls and so on.
Next up, Xiaomi.
Xiaomi: Hello everyone, I'm Xiaomi from China Telecom. This draft is about IOAM DEX option extensions for incorporating the alternate marking method. [Presentation 9: https://datatracker.ietf.org/meeting/125/materials/slides-125-ippm-ioam-direct-exporting-dex-option-extensions-for-incorporating-the-alternate-marking-method-00]
Motivation and objectives: Firstly, both the IOAM and alternate marking method are on-path hybrid measurement methods, necessary to unify the packet encapsulation format to reduce complexity of the forwarding chips. Second one, IOAM DEX option RFC 9326, postcard mode, have faced some problems in performance measurement, such as packet loss, including more precise overhead and bandwidth, inaccurate measurement results and inconsistent measurement methodologies. So this draft intend to augment IOAM's abilities in performance measurement by incorporating the alternate marking method.
The proposed solutions: Extend IOAM DEX option by defining the most significant two bits of the reserve field, D bit and L bit, and measurement period number field, employing 32 bits for Flow ID which may be divided into two sub-fields: Node ID, can assigned globally by the unified planning, and Flow-mon ID, can be assigned locally. So it can be deployed in distributed way in case of centralized controller unavailable. Support the three IOAM operation modes: only IOAM Trace Monitoring, only Performance Measurement, and hybrid.
Benefits: 1. Augment IOAM DEX option's capabilities in performance measurement. 2. Augment the alternate marking method in IPv6 RFC 9343 by extending 32-bit Flow ID, SN and MCN field. Only unique packet head encapsulation format is required for both IOAM Trace Monitoring and performance measurement such as packet loss, delay and jitter, thus simplify the complexity of the forwarding chips.
Flag considerations: Request a new IOAM Option-Type and use extension flag used to MNT. This table showed the collision probability of Flow ID when deploying in distributed way, using 32-bit Flow ID. When generating 10,000 common flows, CP is about zero, but using 20-bit flow monitor ID, CP will drastically rise to approximately 100 percent.
Why extend 32-bit Flow ID, SN and MCN field for the alternate marking method? Firstly, compared to 20-bit flow monitor ID, CP significantly reduced in distributed way. On the other hand, the 32-bit flow ID can be split into two sub-field, Node ID, which respective sufficient space. Secondly, by employing MCN field, it's more simple and accurate for analyzing and correct to correlate loss measurement, packet count values belong to the same measurement period, especially in the case of microsecond level measurement block for real-time network performance monitoring is required. Finally, by using SN and MCN field together, multiple duplicate packets for different load belong to the same measurement block can be identified in case of packet reordering. Furthermore, the average delay can also be obtained for every measurement period.
Next steps: Welcome any comments and suggestions. We believe the document is stable, request for working group adoption. Thank you very much.
Marcus: Thank you. And we have Greg in the queue.
Greg White: Yes, thank you. Um, thank you for the presentation and the work you put. Um, I have a question and that's something that we discussed with Giuseppe during the discussion of the alternate marking method YANG model. Have you considered that YANG model for the alternate marking can be used to define the profile of operational state information and performance metrics beyond delay and loss measurement, so that the trigger packet will act in the way similar that IOAM direct export does, so then this profile defined through the management plane rather through the packet itself? And then I believe you achieve what you're trying to achieve with your solution.
Xiaomi: Thank you for questions. Upon YANG model, there are existing related YANG model for IOAM and alternate marking, so it's similar to—similar to, yes.
Greg White: Okay, we can continue the list.
Giuseppe: Greg, if I can answer very quickly from as the author of the YANG model. Yeah, we didn't include this operational state as you suggest, but of course it's something that we can do and, you know, the YANG data model can be augmented and...
Greg White: No, right. And I think that's something that we agreed because basically, if there is interest, the model can be augmented.
Giuseppe: Yeah, sure, sure, sure.
Greg White: But if it seems like there is interest to use the benefits of alternate marking method, especially for the loss measurement, and combine with information, the richness of information that can be collected using in-situ OAM, but if it's specifically for the direct export, I think that through the management plane, the profiles can be set on all nodes and we achieve another benefit would be that information put into the packet itself, in a data packet, would be lighter and so there less impact on a data traffic. But we can discuss it on the list.
Giuseppe: Yeah, yeah, okay. Thank you.
Xiaomi: Yes, okay.
Xiaomi: Okay, the next draft is about IOAM trace option extensions for incorporating the alternate marking method. [Presentation 10: https://datatracker.ietf.org/meeting/125/materials/slides-125-ippm-ioam-trace-option-extensions-for-incorporating-the-alternate-marking-method-00]
Motivation and objectives: Similarly, the IOAM Trace options RFC 9197, passport mode, lacked the ability in performance measurement such as packet loss. Second, the IOAM and alternate marking method are both the on-path hybrid measurement, necessary to unified packet encapsulation format to reduce the complexity of forwarding chips. So this draft intend to augment IOAM's capability in performance measurement.
The proposal: Extend trace option by defining the most significant five bits of reserve field: D, L, F, S, M bit and optional Flow ID, SN and MCN field. Employ 32-bit Flow ID which can be divided into two sub-field: Node ID and Flow-mon ID, so it may be deployed in distributed way in case of central controller unavailable. Support the three IOAM optional modes: only IOAM Trace Monitoring, only Performance Measurement, and hybrid.
Benefits: 1. Augment IOAM trace options capability in performance measurement. 2. Augment the alternate marking method in IPv6 RFC 9343. 3. Only unique packet head encapsulation format is used for both IOAM trace monitoring and performance measurement. So we also request a new IOAM Option-Type.
Next steps: Welcome any comments and suggestion. We request for working group adoption. Thank you.
Marcus: Right, thank you very much. The requests are noted, as usual, the best way to kind of move towards getting things adopted is to get discussions going on the mailing list. That's the best way for us as chairs to assess the interest in the work and so on. So yeah, anyone interested in this, please have a look at these documents, discuss them, and yeah, we'll consider them in our scheduling. Thank you.
Fernando: Hey, hello there. Can you hear me? [Presentation 11: https://datatracker.ietf.org/meeting/125/materials/slides-125-ippm-benchmarking-for-ai-fabrics-terminology-training-inference-00]
Giuseppe: Hi, yeah, yeah. Just one thing, can you do in five minutes because we are running a bit late with the agenda?
Fernando: I'll do my very best. If you can share the slides for me, I'll go as fast as possible. Let me know when you're ready.
Marcus: You should have control. You should be able to advance the slides on your side.
Fernando: Oh, number two, yeah, that's simple enough. Thank you for that. So as I try to mention, the problem is pretty straightforward: we find that not even in 2544 or in 8239, all our benchmarking traffic RFCs, they were not meant to or they were not designed to benchmark AI-specific workloads, not even from the CCL point of view, the libraries when we are doing the training, or even when we are moving the KV cache from the the disaggregated transfer. So basically all of these different traffic and this pattern does not follow any existing methodology or document. And at the same time, right, the inference in the transport of this traffic, in the case of RoCEv2, we have a section as well for Ultra Ethernet transport on the Ultra Ethernet Consortium spec 1.0. All the QoS requirement, the PFC, the ECN—none of that are covered in any of the existing BMWG methodology documents.
So in a nutshell, we try to address the terminology document that it's a companion of the training one and the and the inference one. The documents are complementary, one of the other, using the same terminology and methodology, and I'll cover that a little bit later. But think about the diverse functions an AI fabric performs. So basically the training—it's been designed, the methodology's been designed even to a fabric running training for thousands of GPUs or even a fabric or a single DUT running inference per se, or a specific dedicated subnet for running MoE or mixture of experts. So the main DUT here for all these exercises is basically the network fabric. We have three topologies defined, which is simply a Clos leaf-and-spine, you're gonna find a second topology with a super spine, and then the most ultimately deployed one, which is a rail-optimized topology.
So, as I mentioned, all the documents are intertwined, they use the same methodology, and then define a set of basically KPIs that I try to briefly describe here. Six of the KPIs, two are very specific to training: the CCT or the Collective Completion Time, that will be the duration of the all-reduce or the all-gather activity or any other CCL traffic profile. And at the same time, the JCT or the job completion time, that it's specific for the training. But then there are two very specific for inference. One is the Time to First Token and what it's important I want to measure here. This does not overlap some of the work already performed by Geek, standard basically what we are measuring here is the specific network fabric contribution to the Time to First Token, not the full end-to-end application like Geek draft, and I will try to cover that on the next few slides. At the same time, on the training and on the inference document, we apply the same stimuli just to be able to exercise the PFC and ECN and measure and be able to benchmark the impact they have on the training and the inference and the different functions as well. Another important point of the KPI: the training fabric as you will find, we cover the standard RoCEv2 transport but at the same time we added a section for the Ultra Ethernet transport 1.0 spec, so you will be actually able to, if you use the methodology, compare if you can get Ultra Ethernet transport capable hardware, you'll be able to compare on the same fabric the RoCEv2 transport versus all the different transport defined by the Ultra Ethernet Consortium.
I know this is a busy slide but basically because of what I just tried to explain, the difference between the KPIs, this is why we need two separate methodology documents. They are fundamentally different network problems. On the training side you can see on the left you have the GPU, the workers, potentially you can have thousands of them, all performing different CCL activities simultaneously, the most common all-reduce one, and every training on the step. Basically we capture at the very bottom right the different KPIs: CCT, JCT, PFC and ECN, how they compare and here again we're going to be able or we should be able if we follow the methodology just to compare the RoCEv2 transport with the Ultra Ethernet 1.0. On the right as you can see, the architecture is completely different: you have a disaggregated prefill pool, the process that from the move between the KV cache through RDMA between the prefill pool and the code pool, and the traffic pattern and the requirement it's completely different, this is bursty and this is very latency sensitive, right. So at the same time we—we define on the inference basically the two most common scenarios, just the distributed PD moving with the KV cache and at the same time the MoE or the mixture of experts. So as you can see between the left and the right, different traffic shapes, different timing patterns, different requirements, different congestion behavior, so to separate the merging documents we believe was the right way to go on the right infrastructure.
Well, so what I wanted to say here is you're going to find there is the excellent work that Gake put together, all of his inference benchmarking document, and the main idea, the main difference, his's mainly focused on Layer 7 or the inference from the bottom over HTTP performance, but we have common KPIs, for example the Time to First Token we define that, so the work is complementary, they don't overlap with each other, so there is a lot of work here for us just to even be able to work together in Time to First Token also of some of those methodology when we leave our draft, which is basically Layer 2 to Layer 4 all the application layer from the compute to the gateways and everything it's been—it's been covered by him. So that's—that's the main difference, but again they're very complementary one another. Well, I do have a lot more, so the question is simple, let me go to the last slide because I talk a little bit about the different KPIs and the difference between our work and and Gake's, but basically our ask is to please if you find this work useful, take a look at the documents, provide feedback to any of us or on the list, we already received some positive feedback from some members of the list. I really want to spend or we really need to spend some time with Gake and and just to define these common KPIs and there's always room for improvement, we already have some little improvements in mind for future releases, but yet again if you believe this is important work just let's try to find a path for adoption of the documents and more than happy online offline just to address any questions you guys may have.
Marcus: Yes, thank you very much, Fernando. It's very comprehensive work and seems like you have taken already a mature test plan and contributed it to BMWG, so thank you very much. I—I don't want to criticize it, I mean it's really awesome stuff, but I feel like my gut feeling is it's kind of orthogonal to all the work that we've been doing elsewhere. It's like a document that could rather be based on other BMWG documents, but it isn't. And so for example, we do have a lot of extensions to 2544, we have the IMIX genome which is an RFC, which could be used to simply describe the packet flows to say like: we don't need to talk about AI, AI inference or whatever for a data center networking environment, this is just a flow of packets of certain sizes and certain characteristics that—that could be used, right? So that's number one. Number two, the document is very much focused on current implementations, for example, of Rocky, and RoCEv2 is established and legacy but it's soon to be superseded by Ultra Ethernet probably, who knows, but that's at least one technology we're considering to test instead, but this draft would not even apply to it anymore because it requires Rocky. It doesn't apply to UEC. So it's very, very domain specific. And third, we couldn't really test it in a—in our lab because it requires the use of many, many, many GPUs, which is very investment intensive, and if you're a test lab you don't normally have this big fields of GPUs of hundreds of GPUs or 96 at least, so it seems to invalidate the use of test tools for, from my point of view, no good reason. And because the test—if the test object is, and maybe that's my last comment, the subject of the test is not clearly defined: is it the network infrastructure in the data center or is it the server infrastructure with the GPUs? Because there are some monitoring, there are some environment requirements and there are some monitoring requirements for the GPUs and servers too.
Fernando: Yeah, so gosh, that's a long list I've been taking notes. So, first one, yes you can use some of the existing methodology or even augment what we do have, still it's gonna fall short of what are the requirements to train a model right in a fabric or just to run inference. So yes it can be done. Second is nothing that it's already there, even if you implement or you start putting amendments, will get you even close to what we describe here. The...
Marcus: Sorry just quickly to interject here. I think it's a very good discussion you're having right now, but we are extremely short on time so I would urge you to take this discussion to the list and continue. I think—I think it's very useful to have the discussion there and I'm sorry to cut you short, but we are very tight on agenda.
Fernando: It—it's good, right? and Ultra Ethernet it's been covered and it's on the document right, so if you want to summarize that on the list, yeah, you can send it to me.
Giuseppe: You can just reply, if you want, you can just reply on the list to Carsten and continue on the list.
Fernando: Sounds great. Thank you.
Marcus: All right, I'm Libin Liu from Zhongguancun Laboratory, I will introduce a new draft called Benchmarking Methodology for Route Origin Validation. [Presentation 12: https://datatracker.ietf.org/meeting/125/materials/slides-125-ippm-benchmarking-methodology-for-route-origin-validation-rov-00]
So before introducing the methodology, I will introduce the background about RPKI ROV. The holders of IP blocks register the ROA and ROV use the validated ROA information to classify the BGP BGP update. The validation results consist of three states: valid, invalid, and not found. Which will affect the propagation of BGP. So ROV is a control plane validation function and this document mainly focus on the how fast, how scalable and how stable the router implementations are when ROV is enabled. So why we propose this document? ROV deployment is increasing but there is still no standardized device-level benchmarking methodology. So this draft fills the gap.
Okay, for the goal and scope, the goal is to define the repeatable and controlled benchmarking workflow for the routers which implements the ROV and provide standard metric to allow the side-by-side comparison. So for the scope there are four aspects including the ROV processing, BGP impact, scalability, and resource utilization.
So here is a test setup. We have four components including the RTR emulator, BGP generator, DUT, and data plane tester to build the test environments.
So this is abstract about the general benchmarking methodologies which falls into five steps. So for the details you can refer to the document. So also for the benchmarking tests we have three types of—mainly three types of benchmarking test cases. The first one is latency-oriented, including the ROV update processing latency, validation latency, and BGP convergence time when ROV enabled. And the second one is about scalability, including the VRP scalability and the ability to processing different VRP churns and resource utilization.
So we also have a Hackathon project time this time and we have some candidate tools to use in this Hackathon project, and for example the BNG Blaster which contributed by Carsten. Thank you very much.
So for next step, we will seek feedback on comments, welcome the comments on the mailing list and we will continue the Hackathon next meeting. Thank you.
Giuseppe: Thank you. It's good to see an Hackathon project and this is a new draft, so please read and comment on the mailing list. Thank you. No time for questions so if you have questions, post on the mailing list. Thank you.
Marcus: Next up we have Packet Triggered IOAM Reporting, and please be very, very, very quick. We do not have much time here, so just do a brief overview of the work and skip over the most details probably. [Presentation 13: https://datatracker.ietf.org/meeting/125/materials/slides-125-ippm-draft-li-ippm-ioam-packet-triggered-reporting-00]
Zhichao Li: Okay, hello everyone, I'm Zhichao Li from China Mobile. I will present the draft Packet Triggered IOAM Reporting on behalf of my co-authors. First some background and motivation. IOAM telemetry record operational data within user packets as they traverse a network path. As alternate marking means, packet loss measurement rely on periodic color changes to delineate precise intervals. Relying on timer synchronization across decapsulating and transit nodes is highly complex and error prone. We need an efficient in-band trigger to replace heavy egress timer dependencies.
So the draft's solution is introduce a cycle identifier. The identifier use two-bit Cycle ID identifies the measurement interval, increments sequentially through values zero to three modular by four. Decapsulating node monitor Cycle ID transition in arriving packets to instantly trigger statistical reporting. So this completely eliminates the need to strict synchronized interval timers on transit and decapsulating network nodes.
This is some detail about the identifier. In Draft 0, we use 32-bit field to carry the IOAM option type header. A two-bit Cycle ID identifies the measurement interval, valid values are zero to three. One-bit K flag is the keep-alive indicator. Zero indicates a user data packet, one indicates a keep-alive packets. The other bits are reserved. And we have received comments from Justin. To comply with RFC 9197, a Namespace ID field should be included. So the format will be updated in the next version.
This is an encapsulation and decapsulation procedure. What happens if there is no user traffic during an interval? The state machine stalls and reports are delayed indefinitely. So if no data was sent during the ending interval, the encapsulating node generates a keep-alive packet used the K flag. The decapsulating node process the transition to trigger report but ignore packet for loss statistics. And this is to mitigate the jitter impact on some trigger.
And a brief summary: this draft simplifies IOAM measurement interval synchronization and the keep-alive bit solves the intermittent traffic pattern problems. And we actively welcome working group review and comments.
Marcus: Thank you very much. Please, if you're interested in this work, review it and comment on the mailing list and we'll consider them in our scheduling. Thank you so much. Sorry for rushing.
Next up is Jared. [Presentation 14: https://datatracker.ietf.org/meeting/125/materials/slides-125-ippm-ip-measurement-options-enabling-passive-network-measurements-with-minimal-additional-data-and-efforts-00]
Jared: Yeah, thank you. Um, I present a draft on an IP measurement option. Um, and we want to enable passive network measurements with minimal additional data and efforts. And yes, um, we're coming from the railway industry background where we have hard real-time protocols and safety-first systems, and um, we want to do measurements without introducing additional data in our networks. Um, but we need to measure especially the critical parameter delay and loss. And those need data from both the sending and receiving host typically to perform such measurements. Um, so we thought, yeah, um, also we want to be data link layer independent um because we have very many various data link layers in our networks, um which we cannot use to measure, and we also want to be independent of the transport layer protocols used for the metrics that we produce. So we thought we could use the IP option possibility that the IP protocols offer.
So we embedded here two things: a timestamp and a packet number sent from the sender to the receiver to do calculations on the receiver. Yes. So one important property for the network is that the host on route can be passive, so they don't need to process the IP option. And yes, we have an Linux implementation of this which we are willing to share if the company approves that we release it. And I would have to question if this work is relevant for the internet community? Are the parameters that we defined relevant? Is there anything missing? Is it sufficiently generic, etc.? So please comment on our draft.
Marcus: Yeah, thank you very much. I see we have Greg in the queue and unfortunately we're super short on time, so if you have comments on this, very much take it to the list.
Greg White: That's okay, that's fine. I just wanted to refer to RFC 7799 for terminology. So you are modifying the packet and thus it's hybrid, not passive.
Jared: Oh, okay. Thanks.
Marcus: That is a very good point. And an excellent thing to continue discussing on the mailing list. So thank you very much, Jared, for sharing this work, and if you think that this is interesting, please engage on the mailing list. And yes, now we will move on to the last phase of the agenda here.
So while Chen is presenting, I will raise a couple of questions on the hands of poll, so please look at them. Thanks. [Presentation 15: https://datatracker.ietf.org/meeting/125/materials/slides-125-ippm-feedback-on-ippmbmwg-joint-session-01]
Chen: Okay, everyone, and I will provide feedback on the IPPM/BMWG joint session. My name is Chen Wu. A little bit history recap. Actually we—you know, the goal of this IPPM/BMWG joint session is really to leverage synergy and encourage both working group to work together. This joint experimental session started from the IETF 123. Actually we run pretty much well, and so therefore continued in IETF 124 and at that time actually I has been assigned to help to coordinate for these kind of joint effort trying to do these kind of joint charter discussion. So we raised four question, for example: Is there any common ground you know can be developed by both working group? How IPPM and BMWG play the dispatcher role? How can make a good use of performance measurement directorate? Another important question is how we can coordinate with other SDO regarding the benchmarking and measurement related work. So we got a lot of positive feedback also we received some concern. So we feedback to the Ops area and talking with both working group chair and AD. We actually also, you know, to continue solicit feedback in the November interim meeting in last year and the, you know, this we can collect more feedback from the people who didn't make a voice on this kind of topic proposal.
So of this November joint meeting, we actually consult with several core working group member, nine people get consulted. I will not list all the name but I want to summarize their key points. And you know these can be break down into key concern and also benefit. I think you know key concern including for example: less focus, less time for the existing work item and new work item, and slow progress of the existing work. And some people think you know scope of both working group are separated, one focus on production network, one focus on lab environment. There's a little common you know, only except except measurement. But really this is you know the foundation you know for both working group to work on. And also some people concern early joint charter maybe cause some friction. But we also received the positive feedback, they think key benefit is in the long term we can improve work efficiency, we can you know increase new worker and also can increase the number of review and energy for the discussion.
But if we really go for this restructuring and merging, the key challenge we face in you know top one we want to highlight here, today we already have some cross-review within the IPPM and cross-review within the BMWG, so thanks both working group chair to encourage this. Do we have sufficient cross-review across BMWG and IPPM working group? Second is you know from the working group member size perspective, you know BMWG is small community compared with IPPM. They have the less experts from test community, even we have some expert from test community whether they have sufficient time to contribute. So these challenges we facing really influenced the way we make a decision to do this restructuring.
Improvement from the working group chair actually we already you know have some discussion actually chair agree actually will continue experiment on with these joint working group session, actually already encouraged cross-review by running adoption call and last call and also we plan earlier meeting actually and determine the time length whether one hour or two hour or two and a half hour and so make a good use of the session time. And also we have joint combined GitHub repo for both working groups so we can better track of the issue to prioritize what kind of work.
The possible option forward actually we have three option. Either you know first option reach other separately, second is we can consider merge target to single charter so to really you know provide focus you know going forward. So we need to reach agreement on how many interim meeting we do setup or how many you know session we should setup for this IETF meeting. Option three is you know actually it's combination between option one and two, you can see we can developer you know charter separately and then we can make a decision whether we have the joint charter. So with that, I stop here.
Marcus: Greg, please. Any comments?
Greg: So what was not clear to me, so Chen, you refer in the opening of your presentation to the synergy between two groups. I don't find that it's being firmly clearly demonstrated that such synergy exists. From what I've seen and especially in today's meeting as well is that after all this time the groups are still isolated. So they are working productively on their topics, but this expectation that there will be cross-pollination, I don't see it happening. The as any organization, this is volunteering and reviews is a community service. So people have to give their personal time to review other documents. And I really greatly appreciate every time somebody reviews my document on IPPM. I don't expect that anybody who have not been actively involved in IPPM discussions will be... I personally would not be effective in reviewing documents that I have no idea about.
Chen: Okay, Greg. Thanks for your comments. Actually we already mentioned in the concerns and the benefits. But what I want to add one more point actually, you know, currently I feel IPPM is overloaded with many topic contribution, but for BMWG we lack contribution and I think if we choose the option one actually in at some point time we will face the similar situation like this, so yeah.
Greg: But—but again, from my perspective, the fact that IPPM produces more proposals compared to other groups means that there is interest in the topics being discussed. So that really might be an indication that limiting the time for discussion during the face-to-face meeting—yes, we've using all available tools like remote meeting, interim meeting and mailing list discussions—but I concern that we are really limiting the flow of new ideas because we are so much time constrained with this experiment.
Marcus: Thanks, Greg. Maybe just to raise here on the chartering, like today at 12:45 and 13:45 in ISG room Liaoning, we start with the chartering. Everybody feel free to join us if you have interests. We definitely will reach out later on the mailing list. And just from the initial feedback we had is there is interest on proposed work, we tried to accommodate as much as possible this time, but it was a bit crowded at the end and we take that into consideration for the next time on the session scheduling. And with that, I think we can close down for today. Of course for those who are remote, we will post everything on GitHub, the different charter, separate or joint, and people can comment, contribute from the mailing list.
Dan Voyer: Do I have time to say my piece?
Giuseppe: Yeah, please. Very quick.
Dan Voyer: So I find—Dan Voyer from Cisco—it feels like the problem statement is not so much in the differences in the charters, it's more as—as people mention—the two working groups are too small and I find that is a artificial problem because what if the working group were had enough people? Would we look at option one, two, and three still? Right. So I think the synergy between—so let's assume that we're rich and we have enough in two different working groups, then what would be the... how can one take advantage of the other still? How can they work better? Like, personally I'm interested in performance measurement, not so much about the lab, but if there were some framework in the benchmarking that could made sense in the performance measurement side, I would probably reuse, not recreate everything. I think that's more how I would see it than trying to collapse the two charter and almost like huge ChatGPT clever way to come up with a outstanding charter between the two, I don't think it's the way to go. That's my piece.
Marcus: Thank you very much for sharing that and this discussion will continue, we'll work on some proposals, we'll share everything with the list. And with that, thank you so much for attending this joint session and have a great continuation of the IETF.