Session Date/Time: 22 Jul 2026 12:00
[00:00:17] Andy Newton: Alright. It is 02:00. We're gonna go ahead and get started. Welcome to the technical deep dive. Today, we're doing the topic is CBOR and CDDL. These are the classic agenda slides. This is the note well. I'm hoping by now you've seen a note well. If you have not yes. If you have not, please come see me. You should have seen it by now. I'm gonna give you about the they said give you thirty seconds, but I'm gonna give you fifteen seconds to look at it. Alright. Again, if you have any questions about the know well, you can come talk to a chair or an AD about that. Alright. Meeting tips. This is standard stuff again. If you are remote and you wanna ask questions, and we do have remote capability for this meeting, please use a headset, or at least be in an area where you're you will not pick up background noise. And also state your name when you're at the mic even if you are in the room. Alright. Is this can we see this? Yes. So I'm Andy Newton. I'm one of the responsible ADs for this working group or for this team. Med is down here. He's the other one. The this has been run by as chairs by Warner and I for a while. I think the last two sessions. I have gone ahead and fired myself, because I wasn't doing a very good job. We actually, we created what we what you can consider a program committee. Basically, we we've increased the chair team significantly. We sent out a a you know, we asked for some volunteers and a bunch of people replied, and then we got a set of names and we picked some volunteers. So some of them are up here on the front row. If you could put your hands up. These are our new chairs, some of our new chairs. And there's Warren too. He's over there. He didn't put his hand up. But, these are our new chairs. If you have a topic you wanna see for the technical deep dive, or you just wanna share, suggestions about how this should function in the future, please go ahead and and talk to the chairs, and, they'll see if they can accommodate you. Any questions about that? Alright. So today's topic is about CBOR, which is RFC eighty nine forty nine, and then CDDL, which is the data definition language for CBOR, which is RFC eighty six ten. I am not going to actually describe it here. You can read the text, but, we have Carsten who is going to give the presentation. So let me share those slides.
[00:03:33] Warren Kumari: There you go.
[00:03:39] Carsten Bormann: It sounds like it wants to collapse in a minute. Yes, good afternoon. Thank you for your interest in the subject today. So I will try to spend about half the time talking about CBOR. That's difficult because Cboe is way too simple to spend half the time on it. And then the second half is going to be about CDDL, which is a data definition language, also known as schema language by some, which can be used for representation formats like CBOR, JSON, or other formats that try to emulate typical programming languages. Okay, let me put that back. Is it not working? Did you give the slide checker control? And you need NAS control. You have to give the slide checker control. Share screen. So click the participant's record of the slide clicker.
[00:04:54] Andy Newton: Yeah. See. See, Jim, you're worried about this. There Yes.
[00:05:08] Carsten Bormann: Here we go. Thank you. Okay. Before we go into details, I think it's a good idea to stay at the surface before going into the deep dive and look at the global map, the roadmap that we are in here. So we are talking about data representation formats. So very generally about how do we actually get data interchange between different components of a system or peers in the network. And this has a long, long, long history, this subject. And the ITF has had one standard in this space from the 1980s. XDR, you might have heard about it, basis of NFS. So a lot of knowledge about this has accreted both in the general community and in the IETF about that. And one result that is pretty much unanimously understood at this point is that we seem to have something like a three layer model. These are not strictly layers, but they are different perspectives that can be relatively clearly separated. The top layer is the information model, which is essentially something that's going on in our head. How do we want the data to provide meaning? It's also called semantic interoperability. That's another word for that. Taxonomy, ontology, meaning, semantics. So this is what I'm not going to talk about today. But of course, it's the part that is really interesting for us. So it's an interesting gap. The next lower layer is the data model layer, where we try to map the concepts that we have at the information model level into actual data that can be interchanged. So that's where things like JSON or XML or Cboe come in. They provide you with an abstract syntax, sorry for the word, for the data that is going to be the subject of the interchange. And then we have layer below that that is called serialization, encoding, message transfer format, marshaling scheme, or concrete syntax, which is the actual byte sequences being transferred. We usually communicate in byte sequences, And we somehow need to get the data model mapped into that serialization. So we will talk a very small amount about the serialization. But that's for most people who work with this technical detail they don't really care about. And the next higher one is the data model level where we think in terms of structured data, usually tree structure, but there are also other structures, graph structures, directed acyclic graphs, all that that we can use to describe our data. So that is one view, and that's essentially a view I've been working on for, like, forty four years in various forms. And, yeah, it's nice to see that we have been making tremendous progress in this forty four years. But it's also maybe not too surprising. Slightly closer perspective, a perspective followed by a fewer people. Those of you who have worked in the XML domain know that this is a text based representation format. And there always was a need for being able to interchange data in a more concise binary format. And at some point, the XML people have created something called XE. Who of you has heard about XE? Yeah. Thank you.
[00:09:35] Andy Newton: Carson, can you get a little closer to the mic?
[00:09:37] Carsten Bormann: I tried to. So is there any What? Watch? Yeah. Is there
[00:09:42] Paul Bastian: a watch? Yeah.
[00:09:44] Carsten Bormann: You're right. Start a time. Very good idea. Thank you. The other observation was that XML really came out of a document format, SGML, and is still optimized for many things that happen in documents, but not necessarily in other forms of data. And one of the formats that we are working on today a lot is JSON, which is more data oriented and is text based. And the gap that CBAW was designed to fill was the lower right, getting something that is data oriented and not document oriented, maybe two orders of magnitude simpler than XE, and binary. So that's where we are. And if you go out now, then you have 80% of the information of this talk. So what is a data representation format? It's something we use to serialize data into a file, into a packet, into a TCP stream, whatever, communication channel. And for most programming languages, it's it's an exercise of an afternoon to do another data representation format. And that's why there are so many of these. It's really easy to come up with a data representation format. But it's a little bit harder to come up with one that makes sense. So we have platform specific formats. Python people have heard about pickle. Perl people have heard about data dumper and storable. Erlang people probably have heard about ERTS and BERT, and so on. So these are all representation formats that are really optimised to work with a specific platform. I'm to use the word platform to mean the implementation on which we build our systems. And that includes both the software, like Perl, but also in part the hardware. Because in the end, of course, we want to do things with our data that is kind of supported by our CPUs, so we don't have to run 100 instructions to get at an integer. Are platform specific. There are also formats for specific areas of application. So most of you have had a CSV file in their hands at least once. And CSV, by the way, is a great data representation format for its niche. And people have been trying to extend it to to cover larger areas, they they usually have failed. There are surprising interchange formats, like the SQLite3 database format. It's used as an interchange format in a large part, in a large community. It's maybe something that we don't see. That's maybe also take home message. This technology is used in a very wide area of applications, and we don't know all of them. HDF5 is a very popular format that's used for very, very complicated array oriented physics data. So I'm not going to talk about those, even though they would make an interesting dinner conversation. But I'm trying to look at platform independent and widely applicable formats, or at least formats that have grown to fill this mode. And yeah, this is the other take home slide. I'm going to limit myself to talking about four data representation frameworks. One that is well known under the term ASN1, but that's actually not the data representation framework. That's actually the data definition language. Everybody in this room probably has heard about XML at least once. Also, JSON. And finally, why have I chosen these four? Well, are widely used. Also, it's interesting. They have come onto the scene at a distance of approximately ten years each. That's a really surprising thing, and maybe not quite a coincidence. And all four formats have heavily borrowed, or stolen is maybe the more appropriate word, their concepts from something that already worked at the time they were written down. So starting with ASN one, that came out of the x 400 project of CCITT at the time. And I saw IAC. They had a language to see from called Courier. If you look at Courier, which was a Xerox project at the time, you probably won't notice that ASN one is is its brainchild because they just redid all the surface feature. But the basic idea is the same. We have a language for describing the structures that we want to be able to interchange. And then we add something called b r or d r as an afterthought. Oh, actually, we need to encode it as well. And here here are a few ideas how to do that. So this is ASN one. Very, very powerful, has grown over the forty years that it has been out there, and it is not going away. The second one, XML, you may be more familiar with. That came from SGML, which had been a very, very successful format for doing data interchange for logistics applications. We always think about books, documentation, manual pages, and so on. But the place where it really earned its place was logistics. And logistics in the sense of, I have this fighter jet here, and that has 40 binders of documentation in it. And I need to find out which replacement part goes right there. That's what SGML was doing in the eighties. And XML was a simplified version of that, developed by a few people who organized in the W3C, the World Wide Web Consortium. And this is interesting because the direction was changed. They didn't start with a data definition language and then did an encoding as an afterthought. But they really looked at how can we make XML files valid HTML files in certain situations with certain additions, and still leave a lot of the complexity that made HTML hard to use away. So that's XML. XML dominated the the late nineties and and the early two thousands. But since it still was SGML at at its its heart, it was about documents. And not all data that we interchange really fits the document idea. So JSON was the next one. I would say JSON wasn't invented. It was discovered. So Douglas Crockford just discovered it's very easy to come up with a representation format. But how do you get people to actually agree on one? Well, you just take something that's already there. And in this case, it was the JavaScript object notation that was used for that. And he rigorously informed executed a policy of minimizing the complexity of the result. And yeah. So the result is JSON with bugs and features, which I'm not going to talk about today. JSON, like XML, had a progression of various data definition languages. But none of them reached prominence before finally JSON Schema did this in 2000 around 2020. And yeah. But there are other ways of describing the JSON data. So both XML and JSON actually support the idea that you can have one representation format with different data definition languages, which I think is a very powerful thing that any system should have. Yeah, let me finally talk about CBOR. And around 2013, we were trying to get internet support for Internet of Things devices, for constraint devices. And we needed something that is easier to run on a very limited platform. And so we looked at Jason said, the data model is almost right, but the serialization is too complicated. So we invented a simpler serialization, which of course was not invented by us, but somebody else, the message pack people. And we just restructured that a little bit. So it's bit more logical in the structure, added a few data types, and that was that. We also did a data definition language for that, mostly because JSON didn't have one at the time. And even if it had one, we would have had to change it to cope with binary data. So CTDL was created. I will talk about that in the second half of this. So these are the the four data representation formats want to talk about. And I don't think any of these will go away at any time. It's going to be our portfolio that we will be working with. And of course, ten more years have elapsed. So maybe there is a bright young grad student somewhere who thinks there will be a successor to SIBO. I I explained why I don't think that's true, but let's go through this in a second. So when you design a data representation format, there are two big questions you have to answer. And the first question is schema dependence. So the schema or the data definition language, do you really need that to work with the data? And one example for data representation format that requires this is XDR, The format that is working behind NFS is essentially based on the idea that the decoder knows what the data should look like and partially what they mean. And therefore, there is very little additional information in the data. It's like, imagine using JSON, but leaving out all braces and brackets. Because you know from context whether this should be an array or object. And this is how XDR works. This is also how SGML partially worked before the XML people threw away that that dependence. So that's convenient because you don't have to type braces or brackets, which is hard on a German keyboard. So I have some sympathy here. But it also means that you have to have the schema. And probably because schemas evolve the current version of of the schema to be able to even understand the structure of the data, much less the the semantics. So that's one way. And the other way is to say, we should be able to actually do an intake of data without necessarily knowing the schema, which means there will be more information in the instance. There will be some information that might be considered redundant because the recipient does know generally how how this looks like. And when we look at things like ASN one, they actually had to to add a little bit of that of schema independence when they started to think about extensibility. And found out that it's hard to know in an implementation what will be extended next. And therefore, you have to have some markers in the data definition language where the extensibility points are. So BR is not concerned with that, but other serializations of phase one like PR need to know where the extension points. Anyway, the three more recent data representation formats that I had in the table, all are schema independent. And that has a second good effect. It means you can wholesale swap out your schema language and still have nicely interworking systems. You can even translate between different schema formats. So one the recently completed RFCs I've been able to work on actually contains two appendixes with schemas, one in CGDL and one in a version of JSON schema. So it's useful both for looking at the schema and for using tools to look at them. Okay. So that's one decision that was taken for Cboe. We are not going to be schema dependent. We'll have a little bit of redundant additional information. But we believe that this can be limited. That doesn't mean that everything has to be schema independent. For instance, the TLS presentation language has or is a format for describing the packets that you find in a TLS interchange. And that is nicely schema dependent because it's very clear from the context where you are in a TLS conversation. They could save a few bits by doing that. Here's an example of what happens when you are schema independent. So this is a data model of JSON. This is everything you need to know about JSON on one slide. Because it just tells you a JSON value is one of those seven cases. And six five of them are already pre predefined in CDL. And the six and the seven can be written as shown there. So that's all the possible JSON documents you can have in your life in one slide. The other big question is, when you do the serialization, do you try to be text based, or do you use the fact that our communication channels are byte transparent, data transparent? Text based serialization, of course, has the advantage that it's text. So at least theoretically, you can look at that. Not sure that's really true in a time where we encrypt everything. But fundamentally, that's a nice thing. That certainly was useful in the first fifty years of the internet, that people could just look at a line monitor and find out what is being interchanged. I don't want to say drawback, but the characteristic you get from this is that you usually have something like delimiters, like the quotes in JSON to delimit strings. You have escaping. And you just generally have something that looks a lot like a parser in the end for ingesting the text based serialization. And the other result of using text based serialization is that to get binary data transparency, you need to do something. And that's something usually is Base64 or base 64 URL in the IETF. So that's not readable at all. It's text based, but none of the benefits of being text based. Anyway, the alternative is to do a binary serialization. We all have seen those early RFCs with the boxes in there, plus minus, plus minus, plus minus, and so on. These usually have been supplanted by some type length value coding, or have been complemented by type length value coding. Because as soon as those data structures get more complicated, cannot just create boxes for all the data that goes in there. As I said, the early internet protocols, the internet application protocols often were text based. GCP and IP are not text based, but what we put on top of that was usually text based. And most of the ITF has been moving to binary encoded protocols, at least in places where where the performance matters. So we now have something like HTTP two and HTTP three because the benefit of text based encoding is no longer there when you encrypt everything anyway. Okay. So now you have two coordinates that you can use to to put various data representation formats into a draw. Yeah, this is mainly a reference slide that you can use. Maybe if you weren't there at the late '90s, early 2000s, developers were pretty miffed with the complexity of XML, that it's really a document model and not a data model that is supported by that. So JSON was kind of a breath of relief. A very simple data model. So the data model fits into one row. It has a different color, if you can see that. And the serialization was essentially taken from JavaScript. So it's delimiter based. What was not so easy to do, apparently, was actually nailing down the data model. So it's not entirely clear what exactly should be allowed in a JSON instance. And we even have an RFC, RFC 7,493 called Internet JSON. Who of you has heard about Internet JSON? Okay. So you can leave now and you still have learned something. This tries to answer the question, the JSON format, the grammar of which is very well defined. What does it mean in terms of the data? So for instance, the text about numbers in the JSON spec says, numbers look like in your usual programming language. Which is not true. The only programming language they look like is JavaScript. Because our other languages have a distinction between integers and floating point weights. That is not inherited by JSON. So people had to do a lot of interpretation, and in the end came up with 74, 93 as a list of things that must be considered to make JSON really interoperable. With simple JSON files, you don't notice the things. It's always when you get into the borders and to the dark corners where you have to know what to do. JSON has also given rise to a common understanding of a JSON style of doing data. You essentially put JSON objects, which are mathematically maps everywhere, and have a number of string based keys to describe the various properties of this object. And those string based keys are also an extension point. So you can put things in there with additional information. And there is a principle that could be called ignore what you don't understand. So that's usually the way JSON is used. But of course, your protocol definition can do something different. And I talked about data definition languages. I mentioned JSON schema. CDDL is a data definition language that applies to JSON. But YANG, for instance, has a way to produce JSON based serialization. So YANG is a data definition language for JSON. And that's why Ying also is a data definition language for Cboe. But I'm not going to talk about that. But keep that in mind. Maybe there are data definition languages coming in from various places. Okay. So CBOR, again, is built on the JSON data model. So it can represent every JSON document in CBOM. And that was inspired by message pack, which was essentially an attempt by people who were trying to drink from fire hoses, logging data, and things like that. They just needed a faster JSON. So they came up with concisepec that's the pec in the name representation format for JSON. And we just rearranged that and simplified it in some places, but also added extensibility on the other hand. JSON is famously not extensible. You cannot add anything to JSON. JSON is fixed. While CBOR has points where you actually can extend the mechanism. They are called tags. I'm not going to go into a lot of details about them, but they are a well defined place where you can find out. Oh, there's information in there in a format that is tag 24. Either you know what tag 24 is, then you can work with that. Or you don't know what tag 24 is, but then you at least know that you cannot work with that. So that's an important aspect of extensibility. Yeah, the other additions were byte string data, because it's trivial to do byte string data in a binary format. And in particular, that often means that when you get some data in and want to process it, you can just slice the data and give the data to the actual component that needs a particular byte string. So there's no escaping, no copying that you need to do. You can just point the application to a specific substring on the input. Or maybe you don't want to do that because you don't want the application to borrow space from the incoming package. But it's at least something that has been very useful in memory constrained environments. Yeah. We embrace the struct idea of how a map, JSON object can be used, but we also embrace the other ways like records. Records are just arrays where the position in the array have a specific meaning. The whole thing is TLV based. But we are only using length information near the leaves. So we never get into the situation that to update something, you have to go up a cascading letter of variable length encoded length information. So that's the nightmare of every BR implement. So we designed this for IoT so we could run it on very simple devices. My initial implementation of CBO had seven sixty eight bytes of code, not kilobytes or megabytes, but bytes. And you could do useful things with that. Now it's used in other places as well, particularly in the security space where you can be quite happy if what you are doing is relatively simple. So preparing for this, I looked at the numbers again. And the fun situation is that right now there are more IETF documents referencing CBOE than there are referencing JSON. That's certainly not a useful indicator, but it's funny. Anyway, so this is a table out of the RFC that just shows different formats. And you see why we call this concise. You can always design a format that is more packed than CBAW. Always. Already because of the schema independence, but also for other reasons. This is byte oriented. They're not trying to pick beyond byte boundaries and so on. But for a low resource, low overhead mechanism, it's reasonably short compared to other forms. By the way, the most interesting form that we looked at was RFC seven thirteen from 1977 where people tried to come up with a data representation format. They didn't have Unicode, obviously, and and they did have a few other things that take for we take for granted. But it really looks amazingly similar to to C1. I wanted to have one slide about security considerations because I think every talks should have one slide about security considerations. There are many security considerations for something that is as focally placed as your data representation format. All data you you send out and all data you get in are going through this place. So you really have to to know what you are doing there. And the interchange format itself can make a contribution to to security problems you might be having. There there have been lots of bugs in decoders in various representation formats. I don't think there is any representation format that has not had those bugs. One is the the problem that in many TLV formats, the length information has to agree with different levels of hierarchy. And an attacker can use this need for agreement as a wedge to put in their handcrafted exploits. So that's why in CBOW lengths are only used at leaves of the tree. The other observation is that decimal number encoding, something you learned in the first three weeks of your computer science program, are way more complicated than you think. And in 2010, we have had two different CVEs that were devastating, both for Java and for PHP, where they just had bugs in the decimal number decoding. So decimal numbers are something you do not want to use in a security critical environment. Okay, those of you who like to look up things, look for the LangSeg movement. These are people who really have been thinking about this problem. And in particular, they have said you really shouldn't use a format to communicate with a potential adversary that is not supported by some form of recognizer, validator, however you want to call them. That's maybe why the validation languages are so much more important now. Talking about security, so one of the very early things we did after we were done with C1 itself was to look, could we steal something from the JSON community to support security? And Jose was just about done at the time. And we decided there should be a COSE, CBOR object signing and encryption format. And that's exactly what we did. And this is now another internet standard, RFC 9052, that defines how to do certain kinds of object security. Of course, it doesn't cover all kinds of object security, but the most basic ones that you want to use are in there. We did this originally to support Internet of Things, Web of Things kinds of applications. But, yeah, it immediately got people interested. And, yeah, it COSE is used in 100 plus iATF specs. Sorry. I was counting non working group drafts there as well, so it's not quite as many. But it seems to be attractive. And it has the three typical data structures. MAC checked item, assigned item, and an encrypted item. And this exists in two variants, one for just one recipient and one for multiple recipients. You probably know all that from CMS or from wholesale. So again, no big surprises, but it's a usable platform. I should say one thing at the end. Many security formats require deterministic encoding, which just means if you take the same data and encode them on platform A and on platform B and platform C, you get the exact byte wise same encoding. That is deterministic encoding. And if you actually have to manipulate data that you have transferred through a representation format before you can do hashes or other cryptographic operations on that, you need deterministic encoding. So that's, for instance, why x five zero nine uses ASN1's DER, distinguished encoding rules, which provide deterministic encoding. The hosted people were smart enough to avoid that, and COSE has has copied that approach. So we don't need deterministic encoding for that. But we do need deterministic encoding for C509, which are CBOE encoded x five zero nine certificates. So deterministic encoding is a nice thing to have available. And one of the nice things about CBOE is that there are no hard things. It's really easy to switch on deterministic encoding. I have an implementation that's not much more than 10 lines of code. So it's really not that complicated. Last thing I want to say about CBOR. Of course, we need various tools. I will talk about tools in connection with CDDL in a minute. But this is a tool you can use to play around with CBAR itself. So here we have some CBOR, which of course is shown in hex because you couldn't see anything. Otherwise, so you can see this is a map with three value pairs. This is the unsafe numbers, 19, and so And on the other side, you see some things that we call CBOR diagnostic notation. Because we found that one of the reasons why binary formats are sometimes hard to handle is that there is no standard way of representing them in text. So CBOE came with a diagnostic notation from the outset. We have improved this once. And we are right now working in the CBOE working group to do the next revision of that. And look at your whole, this almost looks like JSON. So of course, we make sure that every JSON document also is an instance of CBOE diagnostic notation. And you get some CBOE that implements that JSON. Okay. Any clarification questions? I have some time at the end for discussion.
[00:45:03] Andy Newton: So just so you have a timing, so forty three minutes is left in the session Yes, itself,
[00:45:09] Carsten Bormann: right. Thank you. Okay, so let me talk about CDDL quickly. This is something that came after CBOW because we noticed we need some form of data definition language. We cannot write reasonable documents if everything is in English. So we came up with the idea to do a concise data definition language again. So we are now here. We have been down there. We are now at the data model level. So we don't care about serialization for the rest of this talk. We just want to define the data models. So if you say something like, we are using JSON as our interchange format, this is approximately as meaningful as saying we are using ASCII as our programming language. I think most of you program in ASCII. But that's meaningless. You want to know whether you write C or Rust or whatever. So the structure that is behind the data actually is important. And we need to have these data definition languages to define these structures. I showed you a generic data model some 20 slides ago, which showed all of JSON in one slide. So that is something we call a generic data model, everything that the representation format can represent. And we have specific data models which are defined by an application by saying, well, you need to have an object with at least a map key application colon. And then I know what to do with it. So this data definition can be done in English or in a machine readable way. And we wanted to do this in a way that this can be checked automatically. So machine readable was the only way to go. This is another reference slide, so you can read up here why this is actually not much different between JSON and CBOR. The things that come in with CBOR are byte strings on one side and the extension mechanism called tags. So we had to put in a little bit of support for that. But otherwise, you probably can use your JSON knowledge. So how do you go about and do another data definition language? First of all, you look at all the existing ones, decide those are not the ones that you want to copy. And then we notice that we actually have a pretty good role model, which is ABNF. Several 100 I have stopped counting. IGF protocol definitions use ABNF. So many people in this room are familiar with ABNF. Who's familiar with ABNF? Oh, that's half of the people here. And the idea was to just use the general style of ABNF to describe structured data as well. So ABNF has a few idiosyncrasies. So if you studied computer science and had something like BNF or EBNF, the occurrence indicators are on the other end. And a few more details, but you feel at home immediately. So we decided we are going to do an ABNF like grammar for structured data. And we also had a little bit of help from the XML world because they have something called ReLXNG, which is very, very close in concepts, not in syntax, but in concepts, to what we came up with. Yeah. This was published in 2019. And there is work on extensions going on in the CBO working group. There are several RFCs that have been published since. I didn't put them on the slide. So this is ongoing work both in the specification and in the tools. Now let me show you an example. RFC seventy seventy one, which defines something called a reputation object, is defined in this English language. And that only happened after Barry Lieber sat down and essentially invented a dialect of English that had only certain sentences. And then he could translate the data model he had in mind into that English. And this is the same thing in CDDL. These are already comments. Comments are made with a semicolon like in ABNF. And this essentially says we have a reputation object that has two map keys, application and reputons. And reputons is an array of zero or more reputons. And a single reputon is another map or object that has four mandatory fields, three optional fields, and then a general catch all for extensions. So this is again the ignore unknown extensions rule coded in CTL. So this is a relatively not so noisy way to describe the structure. I know a lot of people who tell me they suddenly understood what seventeen seventy one was doing after looking at this slide. So what is the difference between ABNF and CDDL? Well, main difference is that we are talking about data types and not just text strings. And actually, we have two different concepts here, data types and groups. Groups are just sequences of potential arrangements of sequences that can go into something like array or a map to build a data type. So apart from that, it's like ABNF. So specification is a set of productions. Each production assigns a name to a sub primer, like the reputation object and the reputron. And the difference is ABNF. In ABNF, these are strings. In CGL, are subtrees, essentially, or groups of subtrees if you have a third group. So examples of types. There are of course a few predefined types like unsigned integer or text. Since we have both text and byte strings, we always have to distinguish whether we are talking about a text string or a byte string. So text and bytes are our two string types. Time would be a more complicated type. And these types are not necessarily directly mapping to what you find in your platform. So we might have a type in the CDL that says a foo is either the number one, or the Boolean true or the bar, the text string bar, you have to do a little bit work to map that to your platform types. But it's very hard not to have to do that if you are trying to be platform independent. So most people think in terms of 16 bit integers and 32 bit integers and 64 bit integers. And when I tell them there is a 54 bit integer in JSON, they usually look a bit surprised. But there is. We have literal notations, of course. So this is a little bit more than we have support nonfinites. And apart from that, it pretty much looks like JSON. And we have the type choices that are already used up here. So in COSI, for instance, a label that you put on an item in a map is either an unsigned integer or text, sometimes an integer or a text. So that's an important part. So that's one concept. The other There's more stuff there which you can look up. There are various ways to write down byte strings. None of them is beautiful, but they are serviceable. There are ranges. There are control operators where you can attach additional properties to a type. So an AI is not just a text, but it's a text that follows the regular expression to the right. And ten years later, we finally standardized how these regular expressions should be looking like in adjacent path document. Groups are there to create sequences. So for instance, if you have an RGB triple, you could define it like this. And r, g, and b are types for a byte. And a byte is an unsigned integer of size one. So this is almost immediately readable. And we have the occurrence indicators. So this has zero or more labels or exactly three bytes. Again, very much like a, and f. And these, of course, can be combined. So we have the sequence of a message type and zero or more parameters. So you have seen your first CDL today. Groups also can be given names, not just types. This is sometimes a bit confusing, but it's necessary for larger specifications. So here we have a group called opt palms that is used in two different kinds of messages. So the round parentheses mean we are not opening a new array that is a sub array of the thing that's contained. And we just go on with the structure of the original array. So this is up palms here just means we have zero parameters right there in this array. That is the group concept. Groups can also be used in maps. So the combination of member name and member value can also be part in. And finally, you already saw the little noise reduction feature. So something that is to the left of a colon always need the codes. So you can have less code noise on your page, which is useful if you're trying to find your way. Yeah. I looked this up on the slide. Our current indicators work like you would expect them to work. We have plus and question mark in there. You can have group choices. So this is an example of a data structure that has either a street address with a text string for the street, an optional house number and a city, or it has a PO box, or it has an indicator that we're using a package pickup. And the city actually is a group which has a name and a ZIP code. So that's how you can build things together. Yeah. And yet you can take these CDL snippets CDL snippets and put them into a generator. This just generates an example for using this CDL. And that's extremely useful because most surprises, you catch right there. You never get to the next stage where you would have to catch a surprise. Write your CTDL, do a generate, and see, oh, I didn't mean this to be there. I meant it to be there and fix this. So groups here to create sequence. Sequence is kind of implicit in ABNF, but CDDL is about trees, so you have to be more explicit about sequence. And you use the map and array operators to actually create maps and arrays from the groups. So we have one concept supporting both maps, JSON objects, and arrays. We have various features that I'm not going to go into a lot of detail that are really meant for larger specs. So you probably know the equal slash feature from ABNF. We have that as well, but we have that both for types and for groups. We have special names that do not have to be initialized before you can slash equals them. We have extremely simple generics mechanism, which can save you a lot of typing. We have cuts, which is maybe another talk. We have unwrapping, which means we can take out the group out of an array or a map and reuse this for a different map array. And we have an extension point, the control operators like the dot size that you already saw, where we can put in things that applications need, like basic default classic sloppy. That's one of my preferred control operators because that exists so much in real world specifications that we gave it its own name. You work with CDL by using CDL tools. I'm lazy, so I only put my two tools on there, but there are other tools out there as well, including one that takes CDL and generates a complete CBOR parser out of that CDDL. So the typical things you do is you generate instances of a specification to do a quick smoke test. Does this thing work at all? You can validate something against a specification, which of course you do in your CI scripts when writing specifications. And the other tool is cdlc, which allows you to make a little bit more complicated operations on cdlc specifications that draw input from different sources. So the last line on this slide really says, I need a definition of installation media from RFC 9,393. So please pull me in all the CTDL I need to know what installation media is. So it it's trivial. I mean, it's not complicated to do that because the the underlying structure of CDL is that of ABNF, and that has been designed in 1977 when there were no computers with a megabyte of memory. So, yeah, relatively easy to build tools like this. And that is all I wanted to say, and I still have twenty six minutes of time for discussion.
[01:01:47] Andy Newton: Yeah. So if anyone has any questions, you can jump in the queue. I'm gonna prime the the questions. If you said that CDDL is used for defining for the for defining JSON itself. Is that did I hear that correctly? Like, the schema? Okay. So what happens if I have a CDL schema that has CBOR tags in it? How does that work against JSON? Is that just something you're just not supposed to do or or what?
[01:02:22] Carsten Bormann: Well, you have
[01:02:22] Paul Bastian: to You might wanna talk with the microphone, by the way.
[01:02:28] Carsten Bormann: Monitor doesn't work. Sorry. So I'm sorry if I don't understand you. You have to be careful, actually, because CDL has the whole data model, generic data model of SIBO at its disposal. And of course, you can write some CDL there that cannot represent in JSON. And one nice idea for a tool would be to find out whether you have been doing that. Nobody has written the tool yet, so the pain cannot be has been too large. But yes, you have to be careful.
[01:03:01] Andy Newton: Alright. Joseph?
[01:03:06] Joseph Heenan: Hi. Yep. Joseph Heenan. So thank you very much for that. That was very useful. So I'm actually the chairs of one of the working groups that's actually trying to adopt CBOR and CTDL called the DCP working group at the OpenID Foundation. We're getting a lot of pushback from people that the developer tooling just isn't there for CBOR, for CDDL, for CoSE. Is there any response to that? I mean, I've seen the list of CBOR libraries on cboard.io, but is there an equivalent list for COSE and CDDL?
[01:03:42] Carsten Bormann: Well, it's easier to make a cboard.io page because there's so much out there. And I get a new one about every week. Now I'm I'm behind with validating the the input. It's really hard to write a COSE library because COSI is essentially a kit. Have various things that go in there, and you have to put them together the right way. And there are COSE libraries, but they are very different to use from from maybe a a Cboe library. On the CDDL side, yeah, there there is one pretty dominant Rust implementation for a CTDL tool, so I would point to that first. But right now, there hasn't been a lot of work on getting this going for other platforms. So, yeah, go ahead. Please do it.
[01:04:53] Joseph Heenan: Okay. Interesting. Thanks. I mean, do you think this is something that will improve? Are people in the working group planning to produce more libraries and a list of implementations? Or
[01:05:04] Carsten Bormann: The the reality is that if you are using CDDA for spec development, you are done when it works in your CI environment. And since the tools that are there work in the CI environment, the pain required to to get new work done is not large. But as I said, there is this one tool that takes the CTDLs back and knows that that will be encoded in Cboe and generates a Cboe decoder that has the CTDL knowledge baked in. This was very innovative work by Nordic Semiconductor, and I hope other people will pick up the challenge and build some things like that. But they they have to have a pain point to do that.
[01:05:52] Joseph Heenan: Okay. Great. Thank you. Brian?
[01:05:57] Brian Cepus: Yes. Brian Cepus. I've been using c d d l and CBOR for a number of years. And, like you said, using it in CI really keeps you honest about are is what you're writing valid? Is there any plan to have a kind of a CDDL doctor sort of service that will help spec writers out that are not so fastidious?
[01:06:25] Carsten Bormann: We could do that. So when we started SIBO, we we said, that that was actually one of our our design goals. We don't need SIBO doctors. Now maybe with CTDL, this is no longer quite true. And maybe we can copy what's going on in the Yang world to do that. Certainly, are looking at that in the late stages of various RFC developments. But this is not an organized activity as it is for Yang, where it's impossible to submit something to the IATF without immediately getting back a report card, how well that that worked, how the yang in there worked.
[01:07:06] Brian Cepus: And it really is a subjective thing that I've seen, the difference between a a clean schema and a kind of a smelly schema.
[01:07:15] Carsten Bormann: Yes.
[01:07:18] Andy Newton: Carl?
[01:07:26] Carl: Hi. Thanks for the talk. So I've had a look at the COSE RFC, and the pattern I've recognized is that the stuff that is that the signatures and the MACs are being generated over are written in Cboe, and then, like, the encoded Cboe as a byte string is then actually included in like the data structure. And I was wondering this like wrapping of CBOR and another byte string, like where the motivation for that is coming from? Because like CBOR being like a binary format, you technically don't need the byte string to like verify it? Or like was deterministic CBOR not a thing back then? Yeah. Just wondering what
[01:08:09] Carsten Bormann: Yeah. The thing We called it embedded CBOR. So it means we have a byte string somewhere that has data in it that the initial decoder is not supposed to take apart. And then you can save that, for instance, and use it as signing input, use it to actually get at the data that are encoded there. So this was a kind of thinking that was pretty much rooted in the DOM style decoders that we had at the time. So you just ingest the whole stuff, which is for a security option, no problem at all, a kilobyte or so. And then you work with this and at some point you unpack the part that you need. Now you could and we have had a discussion about this on the zero mailing list just the last two weeks you could make it a standard feature or an option for a CBOD decoder to annotate each decoded object with a slice of the input that this is the decoded data item for. So in my original implementation for constrained devices, this actually was the way I did this. But in the meantime, people usually have worked on DOM style environments where keeping the input data available is not necessarily such a natural thing to do. But I think people are finding out that this is a nice thing that an implementation should offer as an option. And we may see that in CBA coders in the future.
[01:09:59] Carl: Okay, thank you.
[01:10:09] Andy Newton: Yeah, go ahead.
[01:10:10] Speaker 7: I'm curious to hear about what people think of something I've been doing lately, which is apart from using CBOR in protocols, you can use CDDL to denote test vectors in specs, which I found is a very useful use of CDDL because you get all these concise and unambiguous differences between text strings and byte strings and hex byte strings. And you can have hex integers distinct from byte strings and all of those things. So you can have as much white space as you like, which is nice for formatting and such. So I'm curious to hear what other people think of this use case because I haven't heard any other opinions about it.
[01:10:53] Carsten Bormann: We do that all the time. And the the most recent two ROCs that I have been involved to and or have not been involved with have been ROC '92 9942 and 9943. I was involved with 9942, and that works just fine. And 9943, we just stopped preparing the errata report. I hope that answers your question. The actual CI scripts that we use for that are like seven lines of code of bash. So it's not hard to do that. And everybody should be trained.
[01:11:34] Speaker 7: It's very easy to produce these kinds of test vectors as well, which is also a nice thing about it. So see that as a tip to other spec editors, I guess. Thank you.
[01:11:43] Carsten Bormann: And maybe offline afterwards, we should talk about what tool change is used for making that happen.
[01:11:54] Jonathan Hoyland: Hi, Jonathan Hoyland. Two questions, if I may. You said that if you don't know a tag sorry, you could have an extension tag and you might not know what it means, the decoder. But if lengths are always pushed down to the like leaves, is there a way of skipping an unknown tag or do you just have to abort?
[01:12:20] Carsten Bormann: You definitely don't have to abort, because the fact that the tag is unknown is just semantics. So all the structural work that is going on in the data model level works. You just, at the end, know that you have this thing there that is in tag 24. And if you hand this to somebody else who understands tag 24, They will be able to do what the semantics of that was. By the way, that was wrapped in support.
[01:12:51] Speaker 9: But but if I so, like, let's say I get let's say
[01:12:53] Jonathan Hoyland: my extension has two fields inside. When I see tag 24, I don't have a length there because it's not a leaf. So how do I skip it?
[01:13:05] Carsten Bormann: The tag always has an item we call the tag content. Like an array, an array has zero or more items, which are the elements in the array. Yeah. The tag always has exactly one item. And you skip this by
[01:13:22] Speaker 9: By skip okay.
[01:13:23] Jonathan Hoyland: But it so you can't have two items inside a tag?
[01:13:27] Carsten Bormann: Yeah. I mean, maybe it's difficult to say how obvious this is. So the syntax, the serialization, is schema independent. So you can decode the thing without understanding the tag.
[01:13:48] Jonathan Hoyland: But you don't know what it means. Okay. And then my my other question was, does CTDL let you define things like bounds? Like, this value must be between six and forty three.
[01:14:01] Carsten Bormann: Yes. And I have a slide for that, I think. Yeah. So the third bullet item, that
[01:14:15] Jonathan Hoyland: Ah, okay.
[01:14:16] Carsten Bormann: Ranges. But there's also something called dot NE, or dot GE, or .lt. If you know your forte and you know what these are, it just fit the syntax without anything. You can say this has to be less than or this has to be more than or it cannot be that particular.
[01:14:38] Joseph Heenan: Very cool. Thank you.
[01:14:46] Paul Bastian: Hey. Paul Bastian. Do you have an opinion on CBOR web tokens?
[01:14:53] Carsten Bormann: Yes.
[01:14:57] Paul Bastian: Because I'm also in in the group that Joseph, the first speaker was, where we're trying to rebuild presentation protocol for verifying the credentials, and it seems we're going to send over the wire. And work from previous groups has built on, like, Cozy only, and there was a discussion whether it makes sense to pursue this approach or whether there is enough benefit of following Cboe web tokens that kind of builds a framework on top of Cozy for these kind of applications. Do you have an opinion on this?
[01:15:34] Carsten Bormann: Well, see see where big programs are? RFC eighty one thirty two?
[01:15:39] Paul Bastian: That's the equivalent of JWTs, like JSON Web Token, for for for CBWAR. I don't know the RFC number from heart.
[01:15:47] Carsten Bormann: Can anyone So so it is well defined what that is. It is certainly an area where we don't have a complete set of examples how people might be using that. That's because Siebelware, Cryptocos like JWTs are very, very, very flexible. So you can do a lot of things with them. And they become useful by combining them with COSI. The SIBO Web Token is essentially just a map with a registry that says what the labels in this map actually mean. And you can do things like expires at and not valid before and issued by. These are all predefined. And then there are things like proof of possession keys that you can put there that are defined. And if it's not defined, you define it and register So it's infinitely flexible. Of course, you don't want to do something stupid like doing another expiry date that has a different number. But I think there are people in this room who have played with Siva Backprogress. Christian, do you want to quickly intervene?
[01:17:10] Paul Bastian: Okay, thanks.
[01:17:13] Carsten Bormann: So showing your hand to me doesn't work. You have to show your hand to MeetEcho session.
[01:17:20] Christian Amsüss: Yeah. Christian Wohmann. So first of all, I think Cboe support has increased quite a bit over the past few years. The first time I worked with Cboe was like five, six years ago, it was quite a pain, I would say, in most languages. It's gotten a lot better, which is nice to see. But the statement before, like COSE support, I think, is still quite lacking in a lot of languages. It's quite hard to work properly with it. Compared to, for example, the wholesale libraries we have for everything, Jose is quite a bit behind. If you really want more of Jose being used, I think that is one thing as a community that needs to be solved. And I understand a lot of the design decisions, and they make sense to me. But there are some things that in the real world have problems. Like for example, for CBOR tags. I understand why they are there. But for example, for CWT, it defines a tag that might be there or might not be there. In reality, it broke a lot of implementations because some implementations expected it to be there, others expected it not to be there. Then for example, in most programming languages, you create something maybe tagged to deal with it. It's causing some weird friction in actual implementations, I would say.
[01:18:36] Carsten Bormann: Yeah. Of course. And so we don't do that then. But, yeah, maybe a document that lists the pitfalls would be something good. So the the Jose people, actually came up with a BCP about five years after. They were focusing on security issues that Jot had, which maybe isn't that important at this point. Avoiding pitfalls would be a nice kind of title for a document. We can always start with a Wiki page today. So we can wait until we have the critical and can say, hello, working group. Let's actually publish this so we know it's a consensus thing and not just a Wiki.
[01:19:27] Christian Amsüss: I think it might be really worth it. Yeah. I I think it might be really worth it to to start collecting the common pitfalls we have.
[01:19:37] Carsten Bormann: You.
[01:19:40] Speaker 9: Hi. So speaking of pitfalls, my question was mostly about is is half question, half a remark about JSON compatibility. Because as you said, like, any JSON is also gonna be a valid CBOR. And I get the the intent because at the beginning, maybe you want to drive adoption of CBOR and so on. But, like, I kinda see the pattern, like, of more often than not. Like, in some protocol that are going to actually map byte for byte what you have in JSON, for example, I don't know, like, you have a map key that is going to be, like, 12 characters in JSON is gonna actually be, a texturing also in Cboe, whereas it could have been just like a simple integrator. So, like, it's kind of frustrating at some point because you, like, spend a great deal of time, like, saving a few bytes somewhere else in the object, and then you have, like, to use that raw string somewhere. So I was wondering, like, is there gonna be some kind of recommendation document maybe for the the working group, like, as as was said, like, noticing the pitfalls and so on. And maybe so that we could stop doing that and actually embrace the conciseness of CBOR and stuff and stop, like, actually mapping one for one what we have in JSON.
[01:20:54] Carsten Bormann: Yeah. I think that that's that's a pretty good idea. There there are several RFCs out there that actually have CDDL for both the JSON rendition and the receiver rendition. And there's now essentially something like an idiom for how to do that. So this JC generic that you find in some places, and we could document that a little better. Of course, we could at some point decide, okay, this is important enough that it needs to be part of the CTDL mechanism and not just something you can pull together with CTDL generics. But right now, the generics have served us pretty well.
[01:21:36] Joseph Heenan: Thank you.
[01:21:39] Warren Kumari: Hello. I'm Chuang Feng here from China Telecom. Thank you for your introduction of Zebra. Yeah. And you mentioned that a new beta format will be defined nearly every ten years over the past history. Yeah. But right now, CBOW is has been around for several years, maybe seven years. Okay. And my question about the relation of Cboe with AI, you know that AI is hot topic right now, and AI agent into communications for data exchange. So I'd like to know from your perspective, Siebel can be used for the data change between AI agents or a new data format should be defined for AI, period. What's your point?
[01:22:31] Carsten Bormann: Yeah. Interesting question. So all those new representation formats came out of left field. So nobody had guessed five years ago, five years before they appeared that that they would be interesting. The question is, what's the pain point? And the pain point that led the transition from XML to JSON for most purposes was that the data model was too complicated. The pain point from Jason to Siva was simply we need a binary representation. So the question would be, what is the pain point for AI? And one of the things we look at in the Thing2Thing research group is, can the IATF IoT protocols actually interact with various things that we call AI today? And for instance, one thing I saw that hasn't been submitted yet, but maybe I can spill the beans a little bit. How do you actually represent a CBOR data item as a sequence of LLM tokens? So that would be something that is probably quite interesting for people who want to run this efficiently.
[01:23:44] Warren Kumari: Okay. Thank you.
[01:23:47] Hannes Tschofenig: Hi, Carsten. Thanks for the presentation. I want to respond to the questions about Cboe web token and in context compare it with JSON web token because it was the Cboe web token work was inspired by the JSON web token because we wanted in the Ace Working Group, we wanted to have something that is equivalent but obviously more compact. And second point was to reuse and consequently a different security mechanism, namely COSI. The problems that you described have nothing to do with the Cboe web token. They are, in words of Karsten's, one of the earlier slide, the when you remember this layers information model, when if you have an application, the application needs to describe, like, it expects certain values in that token, for example, for authorization. It needs to say something. It's not just the basic fields like not before or expired fields, etcetera, or whether it's a has a proof of possession, public key in there, and so on. But it's also there's there's more. Like, you may expect five other fields in a specific application. And those fields need to have a specific semantics, and that comes from the information model. And those exact issues exist with the JSON Web Token as well. And so you need to find out, like, what does the entity that creates that object and and signs it? What does it need to put in there? And what is the recipient going to accept? What what it's in in general expecting and and how does it react? And that's a higher layer specification, if you will. But it's, of course, important for the overall. But there's not an obvious place where that would go because it's so domain specific. In an IoT door lock application, it may be a different situation than for some wallet based application that cares about compressing the data exchanges.
[01:25:58] Carsten Bormann: So XML has exactly if you were looking for something like that. Yeah. So it's not sufficient that a token, be it CBOE or be it JSON web token, validates. That's absolutely not sufficient. It must fit with a security policy that you are trying to achieve. And in particular, the key type you use for validating the thing must fit the key type that is actually in the web token. And that's one of the things that people consistently got wrong and why this BCP was written. I think we have time for one more.
[01:26:35] Andy Newton: We have one more. Julius?
[01:26:37] Julius Ruberg: Yeah. Thanks for the presentation. Julius Ruberg, I had just a small question. So you said that C was always self describing. Is there work done on something similar that is not self describing, like portal path or something like this in the ITF?
[01:26:57] Carsten Bormann: I'm not aware of something. So I think the the most interesting development in the last two years was that we now have a JSON schema working group. So that's solving a different problem. But, yeah, it would be fun to look at that Protobarf or Captain Proto or one of these other representation formats and see how we can maximize the synergy between these things. Protobivas may be a weird example because the the data definition language they have is interesting. So, yeah, maybe I wouldn't start with that. But efforts, initiatives to get something like this going would be very interesting.
[01:27:45] Andy Newton: I think we now have a protobuf mime type though, so or media type, sorry.
[01:27:51] Carsten Bormann: Thank you. Thank you.
[01:27:53] Andy Newton: Alright. Thanks everyone. Thanks to Carsten that you were up there talking for a long time. Carson, there were also some questions in the chat which were quite lengthy and involved, and I didn't wanna relay them at the mic. But if you wanna if you have time at some point, you if you could go into Zulip and and maybe answer them, that would be appreciated. So alright. Thanks, everyone.