Markdown Version | Session Recording

Session Date/Time: 20 Oct 2023 12:00

NMRG

Summary

The NMRG session focused on refining the group's research agenda, specifically for Artificial Intelligence (AI) for Network Management, Data-Driven Networking, and Network Digital Twins (NDT). The co-chair, Jerome Farinas, highlighted NMRG's role as a research group, not a standards-developing body, emphasizing long-term research and informational RFCs. Discussions covered the challenges, criteria, and future directions for each topic, including the need for use-case agnostic considerations for AI, data quality guidelines, and clarifying the definition and architectural role of NDTs. There was also a call for renewed contributions and a clear path forward for Intent-Based Networking (IBN) topics.

Key Discussion Points

General Introduction & Research Agenda Refinement

• Jerome Farinas welcomed participants, reminding them of IETF IPR, privacy, and code of conduct policies.
• NMRG operates as a research group, focusing on long-term research rather than standards development, though informational RFCs are welcome.
• The primary goal of the session was to refine the group's research agenda, particularly for AI, Network Digital Twin, and Data topics, based on previous meetings and bilateral discussions.
• It was noted that research questions should be clearly formulated, and priorities identified for collective group efforts. Outputs can include drafts, implementations, or joint publications, considering other communities' work.

AI for Network Management

• Diverse Use Cases & Architecture: The group acknowledged many propositions for AI/ML in network management but recognized that a common, rigid architecture might not be feasible due to the variety of use cases.
• Use-Case Agnostic Criteria: A key point was the need for common criteria for evaluating AI solutions, independent of specific use cases. These include:
  • Explainability/Interpretability: Understanding how AI makes decisions.
  • Deployability: Ease of integration with existing non-AI solutions and deployment in diverse environments.
  • Configuration Complexity: Simplicity of setup for networking professionals.
  • Generalizability: Ability of solutions to work across different networks and environments.
  • Data Requirements: Clear characterization of input data needs (quantity, quality, characteristics) for successful application and expected performance.
  • Real Deployment Success: Validation of solutions in real systems vs. laboratory environments.
• AI and Network Interaction: Discussion included AI applications for network management and the network's role in supporting distributed AI (e.g., Federated Learning). The question of whether the network is a bottleneck for AI in typical deployments (e.g., data centers) was raised.
• NMRG's Role in AI: The potential for NMRG to act as an "AI advisor" for the broader IETF community was considered, perhaps by proposing AI considerations or criteria for any IETF work involving AI. Inviting AI experts to present advancements in AI technologies (hardware, libraries) was also suggested as a workshop format, though past attempts were not very successful.
• Distinction Between Use-Case Agnostic and Specific Aspects: Christopher suggested separating generic AI technology questions (use-case agnostic) from the utility of AI in specific network management problems (use-case specific). This could lead to a common template for specifying AI use.
• Broad Scope of AI: Ken cautioned that AI's applicability extends beyond network management (e.g., congestion control, routing), raising the question of whether a dedicated AI group is needed or if NMRG should focus strictly on AI for network management.
• Challenges of Generalization and Training Data:
  • Jose highlighted that "conditions to use" should include conditions for high performance, requiring specific data collection efforts and acknowledging that AI algorithms may not perform well with inadequate data. The concept of generalization (models working across different setups) was deemed critical.
  • Albero strongly emphasized generalization as a major challenge for applying AI to networks. He drew an analogy to self-driving cars, where users expect pre-trained solutions. Training AI often requires "bad examples" (e.g., network breakdowns), which is typically impossible in production networks, making generalization crucial for vendors.
• Purpose-Driven AI Use: Parisa suggested that for each AI use case, the purpose of using AI should be clearly articulated, explaining what problem it solves better than non-AI methods. This clarity would help prioritize which AI challenges (like training difficulty) are critical for a given application (e.g., offline insight vs. real-time protocol functions).

Data-Driven Networking

• The Data Problem: Acknowledged that a core challenge is the lack of sufficient and qualitative data for both AI and non-AI solutions (for training, testing, validation).
• Data Quality: Emphasis on "quality by design" for data collection and the need for metrics to assess the quality of data sets used in research. It was noted that historical network security data sets often have quality issues. This is seen as a broader community effort.
• Data Set Description: The importance of providing clear metadata and detailed descriptions of how data sets were built, including format, collection methodology, environmental conditions, and intended use. This would help users understand applicability and limitations.
• Data Formats & Ontologies: Questions were raised about the adequacy of existing network data formats and the potential need for new representations like knowledge graphs.
• Data Volume & Real-time Management: Christopher highlighted the significant challenge of managing the volume of real-time data flowing between networks and management planes. This requires mechanisms to limit and manage data flow, and potential standardization.
• NMRG's Role in Data: NMRG's role is not to produce data sets but to provide guidelines and guidance on data set quality for the community. The work should link with other community initiatives, such as the IAB Data workshop.

Network Digital Twin (NDT)

• Existing Foundation: Unlike AI, NMRG already has an adopted group document on NDT concepts and a reference architecture.
• Flexibility & Use Cases: The reference architecture needs to be flexible enough to accommodate diverse use cases and different levels of network abstraction.
• Definition & Criteria for Assessment:
  • It was noted that there is still no universally agreed-upon definition of NDT.
  • Instead of a binary "digital twin or not," the idea of criteria for assessing NDTs was proposed, including: technology used, level of accuracy, and real-time vs. non-real-time capabilities.
  • Questions about combining/comparing different NDTs and their interfaces/interconnections were raised as potential research topics.
• NDT and AI Relationship:
  • Albero suggested NDT is a more concrete effort. He believes NDT has gained clarity in industry and research, noting common patterns. He also posited a strong correlation between NDT and AI, arguing that NDT emerged with AI despite older simulation/emulation technologies. He offered to present his view at a future meeting.
  • Jerome expressed disagreement that NDT is necessarily tied to AI or that its definition is fully agreed upon, suggesting that advancements in scalability and granularity might explain the term's emergence.
• Revisiting the NDT Architecture: Christopher suggested revisiting the adopted NDT architecture draft, leveraging industry learnings, and clarifying its role. He expressed concern about NDT being viewed as a complete management control system rather than a component. He also sought clarity on methodologies (simulation, emulation, AI), emphasizing that AI is a way of constructing a model but not the only one. Simulation can cover physical properties not emulatable digitally.
• Multiple Methodologies for NDT: Chang stated that AI is just one tool for NDT models. Other methodologies like simulation (for functional tests) or formal methods (for reliability, energy efficiency, as discussed in the new FMRG) are also suitable depending on the task. He suggested that low-level NDTs (monitoring/replication) might not require AI, but high-level ones (autonomous/auto-drive) would.

Intent-Based Networking (IBN)

• Use Case Documents: The group has several IBN use case documents, but there's a recurrent question about their overall value and how to synthesize them.
• Lack of Momentum: A sense of those present indicated a lack of energy or contribution towards continuing IBN topics within NMRG, despite its perceived importance.
• Call for Collaboration: Authors of IBN use case documents were strongly encouraged to collaborate and work together to synthesize their contributions into a common document.
• Future Role of NMRG: The community was asked to provide input on whether NMRG still has a meaningful role in IBN and what specific contributions or directions would be desired, acknowledging that other forums also address IBN.

Decisions and Action Items

• Research Agenda: The collaborative document will be revised to reflect the discussions on AI, Data, and Network Digital Twin. Participants are encouraged to directly edit the document.
• AI for Network Management:
  • Continue to welcome presentations on AI use cases.
  • Encourage presenters of AI use cases to explicitly address use-case agnostic criteria (explainability, deployability, configuration, generalization, data requirements, real-world deployment).
  • Explore NMRG's potential roles in the broader IETF on AI (e.g., advising on considerations, inviting experts).
  • Jerome will take notes regarding the need for "AI vs. non-AI" comparisons for specific network purposes.
• Data-Driven Networking:
  • Discussions will continue on defining guidelines for data set quality and description.
  • The topic of managing high volume, real-time data flow will be further explored.
• Network Digital Twin:
  • The existing NDT reference architecture document will be reviewed, with discussions on potential revisions to sharpen its definitions and role based on industry learnings.
  • Albero will present his views on NDT commonalities and its relation to AI at a future meeting.
  • The co-chairs will arrange dedicated time at the next IETF meeting for broader discussion on NDT methods (AI, simulation, emulation, formal methods) and clarifying its definition within NMRG.
• Intent-Based Networking:
  • Authors of existing IBN use case documents are strongly encouraged to collaborate to create a common, synthesized document.
  • The community is asked to provide input to the chairs on NMRG's future role and desired contributions for IBN topics.

Next Steps

• The co-chairs will integrate feedback from this meeting into the collaborative research agenda document.
• Participants are invited to directly contribute to the collaborative document.
• Planning for the next IETF meeting will include dedicated slots for further discussions on Network Digital Twin definitions, methods, and the proposed architecture.
• Future dedicated meetings are planned to address other topics like IBN and Green Networking.