Session Date/Time: 23 Jul 2026 12:00
[00:00:05] Brian Pawlowski: A tiny bit and maybe related references because I'm like, yeah. I've seen that figure before.
[00:00:13] Tom Haynes: Alright. It's, 02:00.
[00:00:16] Gorry Fairhurst: K.
[00:00:19] Tom Haynes: Let's
[00:00:19] Brian Pawlowski: go. Welcome to the NFSv4 working group at the IETF 120. This session is being recorded automatically, I believe. I have a little red button in front of me. Do you wanna do the boilerplate slides, Chuck?
[00:00:44] Chuck Lever: Here's here's the note well, and, of course, you're you're all bound by code of conduct rules and everything else. There you go.
[00:00:52] Brian Pawlowski: Yeah. And you when you registered, you would have accepted that note well. So
[00:01:05] Chuck Lever: We're going to be using the queue. So please please use the queue to identify yourself when you walk up to the microphone. Speak your name clearly for the AI transcription before you proceed to talk or or ask a question. The agenda, and other information is online at these URLs. You certainly can't click on them, but this is just, I guess, for the record. Okay. Go ahead, b b. BP's gone.
[00:01:56] Brian Pawlowski: Oh, you know what? When you turn when you go mute, it removes you from the thing, so I don't there's 11 people here. Hold on.
[00:02:05] Chuck Lever: Oh, okay. And that's because your camera's turned off.
[00:02:08] Brian Pawlowski: Yeah. Yeah. No. No. Okay. Okay. Okay. Got it. Got it. Oh, this thing really makes me a little bit bananas. Okay. So, just as an aside, it's not in the agenda. But Chuck and I, over the past couple of weeks, have been kinda looking at what else is going on in the ITF. And there's, a lot of discussion about the use of AI in, within the ITF and in creating or editing specifications. I you know, we are the NFSv4 working group, and I know I focus on that. But, you might find it interesting to go poking around at, the other corners of the, IETF website and some of the traffic and messaging going on there because it will impact us. K? Did the note well, this is the agenda. Chuck and I are gonna give some guidance, for moving forward with, working group adoption of individual drafts. We're gonna read just kinda tell you what we're what's under consideration for the documents. A very important document, I sent an email out to the alias, pointing everybody at 7221 that bears reading again. It describes how the working group works, how the drafts progress, what the criteria are for adoption of work, and, kind of what we're looking for from participation. So if you haven't read 7221 recently, read it. If you never read it, read it. Okay? Go to the next slide. Alright. So starting on a high note, the internationalization draft is progressing. It's in review in the IESG. The Shepard write up is complete, and Gory has pushed that forward. Do you wanna say anything about that particular document? Sorry, Corey. Didn't mean to interrupt you.
[00:04:27] Gorry Fairhurst: Yep. Corey Fairhurst, AD. Yeah. It's come to IETf last call, so it's out of my hands for a moment to see who picks up on questions, I guess, there are a few people said they would do a look at it while it was in its final form. So this is the time for anybody else in the IETf who cares about internationalization to look at this, and then it'll to IESG two weeks after.
[00:04:53] Brian Pawlowski: Okay. Yeah. I mean, Dave Dave wrote the document, and he worked with he worked with Klensin and Alvestrand on the review of the document in outside the working group. They are the area experts for internationalization. So I he did a lot of editing to respond to their comments. I suspect we're gonna have a kind of a kind of nonevent from IESG feedback, I hope. But, but that's good. And then when Gory came on, he was looked at the document too and made some, changes before he moved it forward, made some editorial comments to Dave. So, anyway, that's that's going well. Sitting here, looking back at the slide, what we would like to do is try to rinse and repeat the internationalization document in a way that gets other documents that we're working on moving forward. These are the documents that are currently under under construction, let's call it, in various states for the working group. The big ones and Dave's gonna talk to them, like, 8881bis and the ACLs update. Dave was gonna talk about that a little bit later on. There are some other smaller documents on uncacheable files and uncacheable directories within the working group. But the bottom list is sort of large. That was one of the things that I as we were as Chuck just joined as chair. Thank you, Chuck, very much for helping out here. But we have a lot of documents floating around in the working group. And the time that we adopted them, the well, the hope was that we would be clearing documents out of the queue And as we brought new documents on, so how hard can
[00:07:03] Gorry Fairhurst: this
[00:07:03] Brian Pawlowski: be? But I'm getting concerned about the amount of documents and the amount of time people have to actually move forward the documents because that requires active participation. Alright. Go to the next slide. So if you're new to this working group, I didn't look at who all is online. The the largest document in the working group is the update to RFC 8881 to a bis document. The history here is that we were requested by the IETf to break the document to parts, and that's a good idea. The break breakdown that mostly Dave added into makes sense from an IETF process perspective, where the IETF really, really, really cares about about what we're doing on and that overlaps with areas of expertise within the IETf are internationalization and security. The core file system stuff has always been sort of a, an odd duck from the IETf perspective, though there are people that have file system expertise. But it's those other areas that are important. So that led to the that led to the disassembly of 8881bis into smaller companion documents around the major core of 8881bis, which is now the core file protocols without internationalization and without the security information in there. Just in case you're wondering what's ahead of you here. Okay. Next. So the one document that I'm struggling with personally, but I've been spent the past week or so or more.
[00:09:15] Chuck Lever: Before you before you go forward with this, Dave has reported that he is not able to get his, Meetecho software working. He's not here yet.
[00:09:27] Brian Pawlowski: And I saw him online.
[00:09:30] Chuck Lever: He is online. He's in the chat
[00:09:32] Brian Pawlowski: room. Not but he's not live live.
[00:09:37] Chuck Lever: Yeah. I'm I would like someone or maybe myself, if I knew the answer, to type in the chat room where he can go to get some help.
[00:09:50] Brian Pawlowski: Yeah. Give me a second. I just passed by it. Sorry. One second. One second. One second. Yeah. Here it is. Okay. I'm just gonna cut and paste. K. I'm gonna send it to everybody. I'm over here. Chat. Chat. Chat. Should I go I better do this in Meetecho.
[00:10:53] Tom Haynes: So Yeah. The the two bubbles at the top.
[00:10:56] Brian Pawlowski: Two bubbles at the oh, got it. Thank you. I was looking at the chat area. Okay.
[00:11:12] Chuck Lever: Excellent. Thank you.
[00:11:14] Brian Pawlowski: Yeah.
[00:11:28] Chuck Lever: David, go ahead.
[00:11:30] David Black: As I typed into the chat, I have I've had occasional problems with Meetecho not finding devices this week. Manual device selection worked around it when it when it happened. I don't know what what what Dave Noveck's fighting.
[00:11:49] Chuck Lever: Yeah. It makes sense.
[00:11:56] Brian Pawlowski: I'm looking for the
[00:12:03] Tom Haynes: I mean, something last time something. Sending the same thing that you sent the last meeting.
[00:12:07] Brian Pawlowski: Yeah. No. No. I I know. But I'm looking for when when I entered the chat sorry. When I entered the Meetecho, it it gave me a whole bunch of what do you wanna use. I'm looking for a settings button on when I'm in it. Does anybody know where that oh, there it is right there. Let's use some management.
[00:12:30] Tom Haynes: Can
[00:12:35] Brian Pawlowski: anyone tell him where the manual setting knob is?
[00:12:43] Chuck Lever: Yeah. I think I think we could just proceed with the slides now since people know his plight and can address him in the chat room.
[00:12:55] Brian Pawlowski: Okay. So on the security document, the ACL document part of security has been adopted within the working group. And I believe that there's a general feeling that we need to do abyss on the security aspects of the eight eight eight one document. The question is how much needs to be done there? Dave has been working on a security draft and after splitting ACLs and security out of a single document, and he has a current proposal for consideration by the working group. And he's put a lot of time into this. So, my my worry here is that I been looking over the mail history and the history of the doc, and there's been a few attempts to start adoption of the working group. But, I think my walk away from that is I wasn't quite sure that I was getting any feedback or getting the necessary consensus to do the adoption. So I wanna talk about that today and also provide some ground rules moving forward so how we can treat document adoption. So I wanted to just touch that just and as you're as you're commenting on the document, just for any of the other document authors, realize that this is what we wanna do for all documents moving forward so we don't get stuck in an endless lack of adoption without resolution phase. Yeah. So when you look at a document for working group adoption in general, the question is, is this work the working group should take on now? And the reason I'm being so basic here is because we now have a lot of documents in flight. And the problem with that from my perspective is that the authors are putting tremendous amount of time into the documents. And if we don't have the bandwidth within the working group to actually respond, comment, and work with the authors, then it's just unfair to everybody. So when we look at a document and document option in the working group is should we take this on now with a milestone for deliverable? And then what is the vehicle for taking it into the working group? What form should it take? That's a lot on the next slide. I'll there's actually I've been reading the, you know, 7282. Is that 7282? Reading the working group document stuff. There are different approaches you can take to the document. So, when something's proposed for taking it to the working group. But from the working group members and participants, A participant is someone who actually participates. So from the working group, taking on a working group document, there's several you know, at the most basic part is will you review the document? Will you implement the document? Because the ITF has gotten a bit stronger on looking for implementation and and to you know, tipping your hand on implementation to make sure that there's traction here. But the big thing at the end is when we do a call for a working group document, where I we've gotta get more formal and that mostly me maybe, is that silence is not consensus. If you have a view, we need to hear your reasons. If you don't post a view, moving forward, we're gonna basically take that as a no because I don't know what else to do with it because that's sort of sometimes what's been happening. Next slide. Alright, Corey. If you have any problems with what I'm saying, just let me know.
[00:17:44] Gorry Fairhurst: So Corey called my name and invoked me, but, yeah, I I get this. I mean, the security document, I think, has been up for adoption twice and didn't get much feedback then. So I I'd like to kind of get the process running a little bit more formally and move along with the approach. So yeah. Carry on talking, please.
[00:18:06] Brian Pawlowski: Yeah. So there's the chairs are neutral. I mean, we just want the working group to be successful, and manage to towards the, what do you call it, the charter and provide, stewardship for the documents that we've basically launched on the world. That involves things like the BIS documents, which are updates to the core to specifications as problems are found or things need to be extended or we add capabilities. So looking at the security document, the dash 15 document, it's actually not as simple as it as it appears. It's not a adopt or not position. Within the working group, we have a decisions that of how we take the work on. There is an adopt the security document as is, and then the working group takes on that document, has an editor owning the document, and then edits the spec into a form as edit the spec into a document for progression to, in this case, proposed standard. But that's not the only thing we can do. We could request an edit down of the document to a smaller version of what we think we want to take on in the document depending on the content of the document. Okay? So we could we could reduce it. We could split it up. But I just wanna mention also that looking at the content of the document and normative changes, we can also perhaps fold it in to the 8881bis work. I'm mentioning this because that's what other groups have sometimes done when they've adopted a document. And then the last option is basically saying not now. And that doesn't mean not ever, but it does mean that we have too much on our plate. We need to noodle on us some more. Need, to see what may and basically, review the need for it in relation to the other work we're doing. Okay? When a document document is adopted, the working group takes on change control, editor is seated by the chair. We probably need to kinda give an a a rough idea of what the scope and size of this document's gonna be. We need to understand what we're actually taking on, okay, for when we take on the document so we can tell the group what we expect from them. Is there any questions about that?
[00:21:23] Tom Haynes: No. K.
[00:21:25] Gorry Fairhurst: Don't worry. The mic again. I'm trying to work out because I've read the security document, rev -15, and it appears to look like a problem statement with lots of considered opinions about how to address these problems and some normative requirements at the end. So I when I read this, I wonder what's gonna happen when this reaches the IETf security review and when it reaches the IESG. They are gonna have a hard time reading a 150 page document. We might have a hard time or not. The problem is that they are going to read it line by line because it's a PS. Right. So we have to think carefully about what we're trying to publish as a proposed standard through this process. So there is a slight wiggle on what you presented, which I like what you presented, but the wiggle is perhaps that this document isn't or may or not or I could not. I don't know. Think about whether this document is actually destined to be a standard track document that can be conformance tested where the IESG can go through line by line and say, yes. This is implementable, and we expect implementations to have this, and this is the way people are going to do this from a discussion which is, about all the security concerns for NFS version whatever. And I think these two might be perhaps two different documents. So it's a variant of the one which is, I don't know, says update 8881 rather than update 8881. Just make a companion document, which is the standards track part of that that I would hope that people who are implementing NFS, and there are people doing security in various ways in various places, would actually be able to track through those recommendations, those requirements, and clearly sign off that that is actually what they have in the stack or what they hope to have in the stack. What whereas I think that's quite difficult with the current document. So I'm kind of wondering how that sits. So I would love other people to come to the mic and tell me I'm totally wrong or they have a different view, whatever. That's just an input. The
[00:24:00] Brian Pawlowski: chair recognizes Christoph Hellwig.
[00:24:03] Christoph Hellwig: Yeah. And, I mean, I'm in violent agreement. I voiced similar but not the same opinion many times before. I also voiced my opinion that fell on kind of deaf ears that I think this kind of split is a bad idea that splitting security outside of main document I mean, I I agree that we should try to find a way to make documents smaller, but moving normative security aspects out of the main document is a bad idea. So this kinda gets into what Corey says that I think that should or must be part of the BIS document or follow on versions. And I'm Can I try Yep?
[00:24:47] Gorry Fairhurst: In chat for a little bit just to see if that happens.
[00:24:49] Christoph Hellwig: You're so high in the pecking order.
[00:24:51] Tom Haynes: No. No. No. No.
[00:24:52] Gorry Fairhurst: No. I think I was thinking actually of a document that we published alongside 8881bis with 8881bis with whatever RFC double plus one, which was security piece, and we pushed that through together. I don't mind, but 8881bis suffers even more than security document for this secure list of security pieces.
[00:25:11] Christoph Hellwig: Hand, I mean, we're kinda doing that with XDR, which I also find find confusing that there's precedent. But yeah. I mean, if it's a very tightly coupled document, I can agree to it. It wouldn't be my preference, but I'm not upset. But the other part that you said is, I mean, the the the document has a lot of considerations, to put it nicely. And I don't think you're good anywhere near a standard track normative text. For many of them, I'm not even sure we need them at all, but let's assume we need some of them, like, a separate document that has just these considerations and non normative language would be beneficial. That helps.
[00:26:01] Brian Pawlowski: I I I'm raising my hand. Sorry. If I so I going back to Gory's statement, I am uncomfortable with the current state of the proposed draft and pushing it outside the working group if adopted for further review without a significant edit. Because the don't think the ISG needs to see how the, how the sausage is made. They just want the sausage. Right? And the normative text is the sausage, and that is surrounded by a lot of, questions for the working group, things that need to be resolved. And I would like that actually, I prefer that all to be kind of boiled down and seeing what the normative core is. And then I would be able to actually kinda talk to Christophe about, well, is this is this okay to manage as a separate document, or is this amount should this be folded back in? What are we talking about here? What's the you know, and what's the amount of work that we need to do? And that I'm not clear yet on. So I going back to the question about outside the the working group, we need to develop we need to deliver higher quality documents. There was a quite a bit of editing on the internationalization document with feedback from outside the working group that, I would like to be a little bit more streamlined moving forward. Tom.
[00:27:46] Tom Haynes: Tom Haynes, I I find the current document not reviewable, and I find the process of having interim meetings to go over it not sustainable. That's it.
[00:27:59] Brian Pawlowski: Okay.
[00:28:03] Christoph Hellwig: Yeah. So Christopher back agreement with Tom on the first part. Can't comment on the second part because I don't have time for the interim meetings anyway. But, actually, having readable diffs would allow offline review, which the current process of dumping huge changes doesn't allow. And given that Brian mentioned the internationalization draft, I actually have all the same comments for that too, and I was actually waiting on a readable diff from Dave to review it. But somehow it proceeded, which surprised me. And also the high level issues that we had for the security document applied there as well. I mean, the separate document that mixes normative language, high level consideration, applies to multiple versions is not a good document for implementers. I mean, the normative changes need to be either in a BIS document or a tiny normative document if it's important enough. And any kind of considerations preferably stay in a personal draft so that they are discoverable but not part of the standard.
[00:29:17] Brian Pawlowski: K.
[00:29:18] Gorry Fairhurst: Just following up on Christophe there. I mean, the we what we're seem to be talking about is one that is a standard strike document that we've all very carefully reviewed. And hopefully, can be doing implementation status report on as well because, I mean, when it gets to the I e s g now, they're gonna ask who is implementing this? Who who who has been the people who've checked this? And the the other document could be published still. It can be published as informational perhaps. It could really could reside as an Internet draft which is long lasting or it could be individual submission via the IIC. So there are ways to publish the background of considerations. And I don't think that is a huge issue as long as they are just background of considerations. As soon as they become the normative text, this is what the the IESG is going to check line by line through.
[00:30:15] Chuck Lever: Chuck Lever. Regarding, Gory's remark about implementation, I I took off all my hats and, looked at the document in terms of whether we might consider implementing its proposals in NFSD, the Linux NFS server. It turns out, there really weren't any that we would want to, that would require us to do any implementation or code changes because most of the stuff that was written down is already common practice. There are very few, protocol changes, but there were a couple that were, that raised some eyebrows in different ways. The what the proposal about auth sys is really an RPC wide proposal, and I don't think it belongs in NFS specific document like this one. It probably belongs in a document that a comp that is a companion to, RFC ninety two eighty nine, the RPC with TLS document. The other one is one that adds protocol, and my impression was this document being part of a bis cluster was supposed to be, documenting current implementation practice, not introducing new protocols. So I I think in that sense, there's there's more than just splitting this this particular document up into, a rationale document that might remain a personal draft and a a set of, proposals for, normative consumption. Anyway,
[00:32:05] Tom Haynes: that
[00:32:05] Chuck Lever: that's sort of what the item c on the slide means to me. I just wanted to put that in the record.
[00:32:11] Brian Pawlowski: K. Okay. Dave is unable to connect.
[00:32:30] Chuck Lever: David, lock and then I guess I should lock the queue.
[00:32:35] David Black: So we
[00:32:36] Chuck Lever: can move on.
[00:32:37] David Black: Okay. Chuck, I sort of agree and disagree with you. To the extent that AuthSys has RPC functionality implications, yeah, you gotta document it there, but you can't avoid discussing auth sys in NFSv4 Security doc. It's it's just too fundamental to, to to what is being done in some of the security properties.
[00:33:03] Chuck Lever: There yeah. We could take this offline. I'm yeah. I'm I'm happy to have that conversation. So
[00:33:12] Tom Haynes: yeah. Okay.
[00:33:15] Brian Pawlowski: Okay. To the next slide, Chuck. Just we're gonna take a we're gonna do a structured on this call. I sent an email out about how we plan to do it, and couple that to deciding whether or not we take on the work and under which of the scenarios we will take on the work, of the previous page. Again, that's I'm trying to be broader here about what an adoption call is, and that's, after talking to Gory. And, Chuck, and, you know, looking at the how the working groups work and what other working groups have done with documents. But we're gonna get a lot more formal about this moving forward, because we need to. Next. Is there another one? So I went over these these went over this before, and we we already did the discussions. But, again, the big questions are, should we the working group take for any proposed working group adoption? Should the working group take this on now specifically around security document, but it does apply to other documents, is what is the action we take? The document as is. Do we request a different version of it? Do we request a breakup of it? Which is sort of interesting because after the document was it was it right before the document got in that this what was one security document got split in it got, I guess, right before. It got split into two documents separating the ACLs out from security is my recollection. So it used to be one, I thought, but then it became two. But anyway so how do we wanna take it on? And then there is no new document or we decide that this is not the time. K? But q question three is the one that I'm gonna hammer on in a refresher on the mail. And it's not only will you review it and will you implement it. Look, it's not the author that's always the editor of the document. Okay? And this has happened in the past within this group where the editor was not necessarily the primary contributor. They were just really good at getting the document in a condition to move it through the working group into the ISG. So will you contribute? Are you going to be actually be actively adding to the document, or are you available to be an editor for a document? Again, we have a lot of things on the plate, and I'm concerned about our our carrying capacity for the number of documents. So I'm gonna send these questions out. I'm gonna to the list because in the working group, we don't make all decisions here. We have to follow-up on the list to have a wider participation, and we'll be condensing it down a little bit into the same set of questions, and focus on the security next steps on security. So that's it. Again, big one, silence is not consensus. Now we're back. I'm sorry. We ran a little bit over. Thank you for all the feedback. Tom Haines is up.
[00:36:47] Tom Haynes: Wow. The the mic fits me.
[00:36:49] Brian Pawlowski: He's literally up. Did somebody raise the mic for you?
[00:36:52] Tom Haynes: I don't know. They it's just the default setting. It's right. Okay. So I'm going to talk about uncacheable directories. I'm mainly you know, I'm wanting it to advance. Could you advance, Chuck? So the prior rounds, we we had a working group last call. We had a lot of issues raised. I thought I was attentive at addressing those issues. And I I've listed what we you know, what they were and that I I went back and reviewed them. The main thing was that I took out of it was the use case and the requirements. Can we go ahead, Chuck? And so what I what I went ahead with is saying that we're we're not saying the caching is safe. We keep the attributes for the file and directory distinct, and it does state that the directory delegations can also solve the problem. The the problem with the directory delegations is the scaling. I think if we go to the next slide, I'll talk about the scaling. No. Yeah. So the the the choices are, you know, we get rid of the reader attribute, caching across the board. We keep it as a per deployment opt out, or we require the deployments to use directory delegations. I'm voting for b because of performance. I'm going fast. One more slide, please. So we're we're asking for the work work the working group last call to be reopened and to go through the the cycle again.
[00:38:47] Chuck Lever: Okay. Chuck Lever. My intention is to, open a second working group last call, either next week or in two weeks so that we can, get this process restarted and close out any technical objections to the document and get it on, moved out of publication. David Black.
[00:39:14] Tom Haynes: Could you go back a slide, Tom? Chuck's got it, but yeah.
[00:39:18] David Black: Who's got it? Okay. We need to simplify this list. My immediate reaction is the complex in this client implementation for twenty five years rules a out entirely. Gotta gotta gotta gotta pay attention to gotta pay attention to running code that would reduce the discussion design discussion to b to, to b versus c.
[00:39:43] Tom Haynes: Okay.
[00:39:43] David Black: And if you do if you do c, need to think very carefully about back about backwards compatibility. There appear to be some serious concerns there.
[00:39:53] Tom Haynes: Yes. Agreed.
[00:40:05] Brian Pawlowski: K. It's your show, Tom, still?
[00:40:15] Tom Haynes: Christophe wants to Yeah. I mean, one thing
[00:40:18] Christoph Hellwig: is just violently agreeing with Dave Black earlier. I think two is basically the only option out of history we can do. Another thing, while quickly rereading the draft just before the meeting is one thing that is really interesting here is that traditionally in in Unix file systems, things like the file size or attributes of the inode or the file and not the reader reply. Of course, reader plus and only reader plus without reader and NFS really mixes this up. And I kinda expect there to be dragons. I'm not sure what to do about them, but we probably need at least some words missing on the language there.
[00:41:03] Tom Haynes: Alright. I can take that.
[00:41:08] Brian Pawlowski: Me. Me. Me. My hand's up. Ryan. I can't approve myself, can I? Okay. Anyway, so David Black made a comment in the chat, and I I wanna say that, wow. Kind of a big miss on my part. Never mind the carrying capacity of the working group. We have broken the ISG in the past. Literally, we became notorious. That is not an understatement when we plopped down the first gigantic spec. Right? Everybody talked about us in a hallway.
[00:41:52] David Black: There was one minor line line to that, silver line to that cloud. It made the iSCSI spec look small by comparison.
[00:41:59] Brian Pawlowski: Yeah. So yeah. So we just made everybody look better. So that's one thing. So so yeah. So spewing out a bunch of stuff that's to the ISG may just cause other problems, and I think we need to be sensitive to that very much so. And we need to be sensitive to when we give it to them. We'd better be reviewable to the best of our abilities, not not how I used to see code shipped at a prior company thrown over the wall at QA and then hope that QA could make it work. Okay? The ISG is not the working group QA department. The second thing is the law of unintended consequences. Look. Dave did a phenomenal amount of work splitting the document per the plan we worked out with how to make this better with former regimes. If you split this up, you know, if we cut the whale into bite sizes, we can eat it easier. Right? What that really screwed us over with as a working group is it basically threw a gigantic wrench into the ability to differ against the source documents. I'm gonna kinda think about how I am trying to think about how we can get to something that's diffable. But I know, David, you were talking about diffing from doc to doc. But when we hand those four split documents to the ISG for review, they're gonna go back to 8881bis, and they're gonna say the people who didn't ask us to do this in the first place, they're gonna say, what is this? So I wanna be sensitive to that. I'm I'm really thinking hard about how to minimize that shock. Okay. That's all I have to say. Thanks, Tom. Go, Tom.
[00:43:59] Tom Haynes: Oh, no problem. So speaking of large documents, let's, talk about the Flex Files v2. So I I wanted to touch on what I've really done since then. I I put up a prototype in user space allowing me to test implementations of the the protocol to see what was working and what wasn't working. I put a new trust model in to get rid of the synthetic UID and GIDs since the the v two DSs actually kind of mandate v4.2 DSs and not v3 DSs. I put a proxy server in to do repairs and also to address the issues that Christophe and David Black had raised about being able to have legacy clients be able to connect to the server. And I actually finished the drafts. That was the other part I did. Chuck, please. And the when we when we think of the erasure coding, we tend to think of the math, and I kinda think of the because we have these large deployments and these large systems of data gathering, and we need to be able to accommodate them. We're not gonna be able to do that with mirrors. We we need the you know, what I'm proposing here. Chuck, please. And so the the prior slide was the the use cases. This is the requirements. We want o one on the round trip. We want we're not guaranteeing any multi chunk atomicity much like rights right now. If you send two rights in a row, there's no guarantee they're atomic. I don't want any consensus rounds. I don't want any distributed lock management, and I wanted a bounded repair. So the trust state ID is a mechanism for, basically, when the client opens the file, the MDS sends a state ID to the DSs saying that if the client presents this state ID, you can trust it. And it allows for transporting Kerberos principles for allowing, you know, authentication. And it allows us to revoke in the state IDs immediately. So before, the revocation was doing a change mod on the file and waiting for all the clients to see that that change mod had taken place. With the revoke state ID, the the revocation is immediate, and the client will be getting errors back saying that it no longer has access. Yeah. I also did some experimentation on making sure that when the client was killed mid Stripe. So the the ride holes occurred. And with the v one server, we see that, you know, four of the six cells produced the result. With the v two, we don't see mix. And the reason we don't see mix is we go repair the file with the chunk mechanism that we've introduced. By the way, I had 30 more slides, and I cut it down thinking I didn't have time. So if you see me jump around a bit, it's because I was time conscious.
[00:47:45] Brian Pawlowski: We appreciate that, Tom.
[00:47:47] Tom Haynes: Yes. I know Brian's the only one who saw all the slides. So
[00:47:52] Christoph Hellwig: And they weren't any good?
[00:47:54] Tom Haynes: They were just as good as these ones. Next slide, please. So the the other thing I did was I went through and I did the Reed Solomon implementation, the Magent implementation. And, basically, below a meg, we see that the the encoding, the setup cost is what dominates the traffic. And once we bypass that meg, we are seeing gains. The other thing to consider here is at the four plus two, we're sending one and a half times the payload. At the eight plus two, we're sending one and a quarter times the payload. Can we proceed, please? The one of the points that Brian brought up in a review was we need a mandatory to implement encoding. We we can't have every different vendor supplying their own implementation and not having a common encoding type to to talk to. And so the proposal is the RS-Vandermonde encoding that I present be the MTI. And the reason for it is we were able to provide a non patent variant so that we don't have to worry about lawyers.
[00:49:29] Brian Pawlowski: Wait. Oh. Wait. I need to talk. Okay. Talk. Okay. I'll remove myself from the queue. Anyway, so wait. I don't recall saying we need a mandatory to implement thing, by the way, but, but, I do think we need to show that it is general enough to handle other encoding schemes, which may only be provable by having another coding scheme. Okay? I wasn't I didn't use the word mandatory, but it it may end up at the same place. Yeah. Okay? So that that doesn't matter. There was one other thing here. Oh, patent. Sorry. As we were reviewing documents and I was reading over things in preparation for this meeting over the past two to three weeks, I actually started looking at what was being submitted for the Mojet encoding. There may or may not be patents on the Mojet encoding that Hammerspace acquired with the acquisition that Hammerspace became owner of with the acquisition of the Roseau fs company. So I just wanted to make sure this is a disclosure. I will provide more information, in the future. I don't think, with my chair hat off, my Hammerspace hat on, I'm gonna say that I don't think the patent situation with Mojet as it stands will be any concern to the IETf, regarding use. We when I found out about this, some people were surprised that we had them. So don't don't worry about it.
[00:51:37] Tom Haynes: I was surprised.
[00:51:38] Brian Pawlowski: Well, I mean, worry about it, but this is the disclosure. We will get more information, Hammerspace, on that patent situation.
[00:51:50] Christoph Hellwig: So, Christoph Helvick again. So violent agreement on needing to agree on maybe one mass supporting coding or at least a tiny number that everyone is comfortable with. I know Hammerspace has some whatever commercial interest in the Mudget one form. Like, every other criteria, it does not look like a very compelling option talking that with, like, a well, Linux kernel in general head on, not NFS maintainer head. That would be someone else. And in implementations of block or file system based rate, people have standardized on a relatively small family of algorithms in various open source projects. And I think sticking to something there would be beneficial Okay. Prescribing a
[00:52:51] Tom Haynes: specific If you if you give me the list, I'm more than willing to go. I mean, we've already talked about SnapRaid, and I'm implementing it.
[00:52:58] Christoph Hellwig: Yeah. And for example, the standard SnapRate one that's out there right now and not the fancy one we were talking about is an extension of the Linux rate six code. So for single and double parity, it's fully comparable to Linux rate six. Okay. For example, ZFS and FreeBSD and Solaris is fully compatible to Linux rate six for a single double parity. Once you go beyond triple double parity, implementations are diverging, but usually just by the choice of the polynomial and which which should allow people to Yeah. Come to common ground some way.
[00:53:43] David Black: So I put a question into chat. This this may require a longer discussion of interoperability because even if you've got a common a an intersection of the encodings that the client and server support, that doesn't help you if the data on the server is in a is in an encoding that the client doesn't understand.
[00:54:08] Tom Haynes: David, you're actually this is the the point you raised at the prior meeting. Mhmm. And my next slide talks about that.
[00:54:16] David Black: Fine. I'll hold it.
[00:54:20] Tom Haynes: Oh, I'm sorry. I didn't I interrupted more than I wanted to. So I provided a proxy server, David, and a proxy server is it can register these are the encodings I talk about and allows a client to connect, say, via v three, NFSv4.2, or whatever, and say, please translate this encoding that you understand to something I understand. And I I can see the the raised hand. Christophe, are you still in the
[00:55:03] Christoph Hellwig: Yeah. I'm I I I think the one before was just left over.
[00:55:06] Tom Haynes: Okay. So oh, alright. And so I provided a measured cost. I went ahead and did I didn't do a a a client implementation. I did do a simple user land reader and writer, and I compared it versus the PS. We can see the times two to three on top of the native. Just pointing out that we can supply a proxy server, and we can then also allow the clients to con direct connection. I mean, on the
[00:55:47] Christoph Hellwig: one hand, it's good to have such a proxy. On the other hand, it still kinda defeats the purpose if that isn't the last resort to get your data out of a degraded cluster. And, I mean, that important thing to remember is if you have something that's more like a disk based rate, I mean, like a erasure coding and I'm really bad at the math erasure coding terms. I just know the rate terms, but something that is primarily the systematic data units, not like one of the mojett Yeah. Modes where it only has nonsystematic data units. Then if you have a non degraded file or, I don't know, whatever you call your shard that the degradings happen on. I mean, you're reading just as a stripe unless you have to rebuild. So actually getting the data out of a non degraded cluster with a non stupid encoding is very easy.
[00:56:42] Tom Haynes: Okay.
[00:56:43] Christoph Hellwig: You still want to get your data when it's degraded, of course. Hopefully, it doesn't happen all the time. And this is where your proxy server comes in. Regularly writing through proxy server probably means we have a failure in the amount of algorithms we offer that people don't actually end up implementing. And, I mean, the other other kinda interesting thing is the the MDS itself shouldn't even know about the encoding because, basically, it would just be the clients including the proxies because the MDS shouldn't ever have to touch the actual data that
[00:57:22] Tom Haynes: Correct.
[00:57:24] Christoph Hellwig: It still doesn't really reduce the problem. It just frames it slightly differently. Yeah.
[00:57:34] Tom Haynes: And so the the implementation is something I call RefFS, and it's user land. It implements a proxy server, data servers, MDSs. David?
[00:57:50] David Black: Mark me a skeptic on the performance hit on this. I see the point for, help. I need to I need need to extract my data from the black hole it's about to fall into. I'm skeptical about, that kind of performance hit for production use.
[00:58:05] Tom Haynes: Okay. So the only thing I did not do was the client because clients are more complex in the having to be on the edge of the POSIX semantics. Go forward, please. I did two different back ends just trying to show that the protocol did not dictate the on disk layout. I'm not claiming these are production back ends either. I'm claiming they're prototype back ends. Proceed, please. And this is the talk we've been having all day. So the first draft is at 216 pages, and the PS draft is already at 62 pages. So my proposal, if we go forward one more slide, please. Is to split it into eight different documents where we for example, the encoding one six and seven could be handed to people who are who do encoding and not people who don't do it. So we could split up the the review cycles. I still get down to a 123 pages on the chunk operations because there are 11 new operations, and they take two or three pages each to describe. I'm I'm proposing that we we do split the document. Is this not reviewable? Any no. And the the first one is the only one that's not a PS. The others are proposed standards.
[01:00:03] Gorry Fairhurst: So, Gary here. Yeah. You still end up with the same page current going to the IESG.
[01:00:10] Tom Haynes: Yes. But it's
[01:00:11] Gorry Fairhurst: But the if the page current goes and you say the working group has firm consensus on this and this has been implemented or whatever. These these words are always good. These words have a lot of weight. So if you can split it up, that's good. Maybe into eight documents, which is rather a lot of deliverables. Does it is it?
[01:00:31] Tom Haynes: So the the did we go back? The the the only two that I really wanna split up are the encoding ones because I wanna split up the impact of potential patents on a document. The others, I can I can fold them back in and look at other ways to do that?
[01:00:53] Brian Pawlowski: So sorry. Michael, you guys Yeah. Raise my hand. So, Tom, just my comment is, two. Again, I don't I don't I'm worried about the patent stuff. I know Gory should say, no. Don't say that. You should be worried about everything like that, but I'm not worried about it. Could I talk to Tron, and we have no intention of trying anything. We want this to be open. But the second thing is, if if we came out with a document that only had an embedded coding in a single document, that troubles me because we haven't proved that someone could do a squint your eyes, another layout for erasure coding as a separate standalone document. And and, having the split from the start might help. And, David, I was really trying to remember what was done with p n f s early on. Was there an embedded transport for PNFS with the document when it came out and then other things were added?
[01:02:11] Christoph Hellwig: There still is, and it's a problem.
[01:02:15] David Black: P p pnfs was was sorta sorta half and half. I believe the old flex files was put straight into, the original the original doc, but the other layouts, were were separate docs. Also, comment in chat that there is more than ample press in the security area for splitting all of the encoding algorithms out into into in individual documents. And if you expect more encoding algorithms inbound, that's a good thing to do from the outset.
[01:02:49] Christoph Hellwig: Yeah. So NFSv4.1 was actually really weird, and it's still biting us. The original files without flex is in the document, and that's actually pretty bad precedence.
[01:03:02] David Black: Yeah. In twenty twenty hindsight, we prob that that we probably should not have done that, Christophe, but twenty twenty hindsight is wonderful.
[01:03:09] Christoph Hellwig: I mean, the one interesting thing here is that we actually also have the zero encoding, which is just striping because striping essentially is is a parity or systematic erasure coding without any extra parity or read redundancy. Actually, keeping that in the main document probably makes sense just because it's so trivial and you have something that works. But anything where we actually do need to specify an actual encoding probably should be out. Yeah.
[01:03:50] Tom Haynes: Alright. I know it's a lot of pages. Section twelve, eight eleven, 24, 25 are good to read. You know, the questions that Brian was asking earlier, who will implement well, I will. And they're already there. Who will review deeply? We've had some reviews, and we've addressed those along the way. If we go one more slide, I think we're about done. And I'm basically a call of a call for adoption, but we're gonna do that on the list as we talked about earlier. So these two slides, we don't really need to talk about. David?
[01:04:42] David Black: So, Tom, maybe I wasn't paying attention. Where did you leave us on the interop requirements?
[01:04:52] Tom Haynes: Could you state that differently? I'm
[01:04:54] David Black: not Okay. Comment comment comment in chat. Is MTI enough? I would think client needs to understand all encodings that servers are using. I think your answer was the proxy server, which I'm not sure I'm not sure about. Is that correct?
[01:05:09] Tom Haynes: That is my answer. Yes. And then the other the other answer is, as Christophe pointed out, maybe we provide a common set of encodings that everyone should be supporting.
[01:05:23] David Black: Or very or or the very least that every client that that that every client has has to support. I I think we're far enough along, that you're gonna have to write up, something about, what the interop what the what the interop requirements and and results are. It sounds like a single MTI is probably not going to do what's really wanted here.
[01:05:47] Tom Haynes: Okay. Fair enough. I'll take that in for the next revision.
[01:05:51] Christoph Hellwig: Yeah. So so violent agreement on the letter. And I think the other interesting story about different encodings is I mean, you definitely in the long run, hopefully not initially, you want an NPS to support multiple encodings if you so that you can life upgrade from previous one to a future better one. This is one thing.
[01:06:16] Tom Haynes: Yes.
[01:06:17] Christoph Hellwig: The other funky thing, which I don't know if we want to support it, but I know it's possible in similar things, you can't actually support multiple encodings for the same file data. Think you're having a new encoding that you think is better in every way. So what you can start doing is encode your parity for the new encoding without deleting the old one, and only once all clients know about the new one, release the blocks that store the old.
[01:06:53] Tom Haynes: And that's doable with the PS. Yes. So no. I mean, it's in the the current draft, that that capability.
[01:06:59] Christoph Hellwig: Yeah. So that's another thing. So there's no, like, one thing the clients need to agree on. Also, writing old data with multiple encodings is, of course, not really something you wanna do in the long run. Yes. But to loop back to what Dave said, it's kinda putting requirements on what clients have to support is kinda hard. Right? I mean, who's gonna enforce that? It's much easier to enforce the compatibility on the servers. Okay. But I still don't really have a good answer. It's kinda trying to enforce multiple encodings on different entities is always gonna be hard.
[01:07:41] Tom Haynes: So I I actually like the statement you made earlier about the the empty encoding. It's like, in my mind, that kind of translates to you have to support v4.2. You don't have to support a layout type. Yeah. Right? And So, I mean,
[01:07:55] Christoph Hellwig: the empty, the pure striping is a no brainer. One other thing we could consider and I'm actually really curious about your use cases. I know some some really big sites like National Labs actually do double level parity or Azure coding where they do one in the network layer and one at the local files in there, which means in some cases, single parity of the network might actually be useful. And if you only do a single one, the export based classic rate five style is another Okay. Complete non brainer that's trivial to support. And the other interesting thing is that basically every Reed Solomon encoding ever used for disk rate always use that for the first parity. So you'd actually branch out
[01:08:39] Tom Haynes: to different
[01:08:40] Christoph Hellwig: ones after that. The yeah.
[01:08:46] Tom Haynes: So, Brian, have
[01:08:48] Brian Pawlowski: Two super quick comments. Sorry. You've been presenting this work in progress and how to implement with the, bakeathons. Have you? Okay.
[01:09:04] Tom Haynes: Yes. We so we did the last bakeathon.
[01:09:07] Brian Pawlowski: Right. I just wanted to get that on record. And I remember that, Gory, I forgot to well, we have enough to do for this meeting. I was I promised a write up on the bakeathon. We'll take that to the list and give a status for the last bakeathon on the testing. Okay?
[01:09:23] Tom Haynes: Okay. David?
[01:09:28] David Black: Yeah. I also wanna call attention to Christophe's, comment that there might be a set of algorithms that that at least the open source community has agreed on that could be that that could be made made man mandatory across the board. That was that was one. Number two was I would strongly suggest sending migration between encodings off to a separate draft. Don't try to do it in round one.
[01:09:55] Tom Haynes: Oh, it is in a separate draft already. But you're meaning not not in all set? Okay.
[01:10:08] Chuck Lever: I wanted to this is Chuck Lever. I wanted to circle back to something that we talked about on the list, and that was the requirements around interoperation with file FlexFiles v one. I didn't see mention of that in these slides. Possibly, you redacted those due to time and slide deck capacity. But what are your what are your intentions there in terms of adding them to the to these drafts?
[01:10:39] Tom Haynes: So my intention there, Chuck, is that it is an ingestion method to allow fast writes and then a translation a migration to a a quick a more viable storage solution.
[01:10:55] Chuck Lever: So you intend to lump that into the migration document? This is an organizational question.
[01:11:03] Tom Haynes: I I'm trying I yes. As far as I understand the question, I think the answer is yes. Okay. I'm not trying to sidestep the question. I have addressed it on the list before.
[01:11:16] Chuck Lever: No. That that's fine. I mean, it it means you're I what I hear is you're thinking about it, and that's all I wanted to know. Yes. It wasn't. It was left out of the slides, but but you are thinking about it. So that's good.
[01:11:32] Tom Haynes: Believe we're done.
[01:11:42] Chuck Lever: So at this point on the agenda, Dave Noveck was supposed to present on the 8881bis respecification effort, but he's not able to join us. How do you want
[01:11:57] Brian Pawlowski: to did Dave said he's gonna take it to the list and would like to queue up an interim meeting to have the discussion. And in preparation for that, I would recommend that everybody actually read the slides that have been posted that he did. But we'll take that offline and follow-up after this meeting.
[01:12:24] Chuck Lever: Okay. I'm not I'm not terribly pleased with the idea of an interim. It's a question mark about who can attend. Usually, they're sparsely attended. Yeah. And Dave's had some technological problems that doesn't bode well for his ability to attend those, and I'd really like to prioritize moving this onto the mailing list instead.
[01:12:45] Brian Pawlowski: That's fine. That's fine. Yeah. Okay. I should have said either or something, and then maybe. The last item on the agenda was if there's any other business that came up in your while we were having the presentations. Any words from the AD? Well, our previous ADs always had closing comments. No. They didn't. Gory, they didn't. Sorry. We were lucky if they were in the room. No. I didn't say that.
[01:13:20] Tom Haynes: No offense. No offense.
[01:13:22] Gorry Fairhurst: Sorry, Fairs. Yeah. Divide and conquer. Get some documents reviewed. And thank you for everybody who attended and those people who didn't make it well. Sometimes technology can be hard. But
[01:13:36] Brian Pawlowski: Oh, we do have we do have something somewhere between thirteen and fifteen people on the, dialed in. By the way, don't know if you were looking at that. So that's
[01:13:48] Gorry Fairhurst: That is it. That's good. I mean, if anybody wants to add any other business or any comments, a lot of things were said. And if you think that they didn't take the right direction, then just pipe something up and join the queue. I would encourage people to use this meeting to try and shape the direction as best they can.
[01:14:09] Brian Pawlowski: K. K. I just wanna say, hey, David Slick. I see you on dialed in. It's been a while. Hope everything's well. Other than that, I think we'll close the meeting, and, we'll be getting the minutes up.
[01:14:31] Tom Haynes: Bye bye.
[01:14:33] Brian Pawlowski: Thank you for participating. Even you, Soren. Okay?
[01:14:37] Tom Haynes: Thank you very much. Okay. Will review. By the way, I will review. So
[01:14:43] Brian Pawlowski: Okay. Thank you. Thank you.
[01:15:18] Gorry Fairhurst: Well, that might be okay.
[01:15:27] Christoph Hellwig: I mean, currently, we're basically having both drafts and dates. Yeah. It's I mean, I think Chuck is one of few somewhere
[01:15:35] Tom Haynes: up there. So I thought I thought it might have been more productive what happened.
[01:15:41] Gorry Fairhurst: Yeah. I mean, you thought
[01:15:43] Tom Haynes: No. I I thought it was I thought it was more productive with the way things happened.
[01:15:47] Gorry Fairhurst: I think yeah. And we're probably still being caught.
[01:15:51] Tom Haynes: Yes.
[01:15:51] Christoph Hellwig: So
[01:15:53] Tom Haynes: alright. Aside.