Session Date/Time: 08 Nov 2021 16:00

bmwg

Summary

The BMWG session covered updates on existing working group documents, discussions on a new YANG model adoption, and presentations on two new proposals. Key discussions included the resolution of "disgusted" ballots for the EVPN draft, the potential for BMWG to adopt a Standards Track document for a YANG model, and detailed insights into a Stateful NAT benchmarking methodology. A new draft on evaluation methodology for Integrated Space and Terrestrial Networks (ISTN) was also presented, sparking discussion on its scope and fit within the working group's charter.

Key Discussion Points

Ethernet VPN (EVPN) Benchmarking Draft Status
- The draft received five "disgusted" (blocking) IESG ballots.
- Authors have been unresponsive despite repeated attempts to contact them by the AD (Warren Kumari) and chair (Al Morton).
- A deadline of October 31st was set by the AD for progress, which passed without resolution.
- Discussion initiated on finding new volunteers within the working group to take over the document and address the IESG comments, including the Auth48 stage.
Next Generation Firewall (NGF) Benchmarking Draft Status
- Last working group call comments were resolved.
- The document underwent shepherd review (by Al Morton).
- New comments require updates to the latest draft version, specifically editorial changes and checking references for newer versions.
- An explicit mention of "obsolete 3511" needs to be included in the abstract.
Multiple Loss Ratio Search (MLRS) Draft Status
- Authors (Mossyack) received new comments from Gabor and Vladimir over the weekend.
- The authors are expected to incorporate these comments into an upcoming 02 update.
- The document is progressing towards a working group last call.
YANG Data Model for Network Interconnect (YANG-NI) Adoption Call Outcome
- The initial working group adoption call received an objection (Tom Petch on prefixes) and several comments (Jorgen Schoenwalder on status).
- Discussion centered on changing the document's status to "Standards Track," which would be a first for BMWG (historically producing informational documents).
- Concerns were raised about the working group's lack of YANG expertise.
- It was noted that the BMWG charter does not explicitly prohibit Standards Track documents.
Benchmarking Methodology for Stateful NAT (NAT-BM) Draft (Gabor Lencse)
- Updates (v01, v02): Incorporated mailing list comments and refinements based on real-world measurements using iptables as a stateful NAT44 implementation.
- Measurement Experience: Conducted scalability tests on CPU cores (1-16 cores, observed ~10x scaling) and connection tracking table size (up to 800M connections, observed performance degradation with high numbers, but good scalability over a wide range after initial L3 cache effects).
- Methodology Refinement:
  - Identified significant processing difference for new vs. existing connections.
  - Difficulties with connection timeouts influencing results.
  - Proposed focusing on two extreme situations for repeatable results:
    1. All test frames create a new connection (for maximum connection establishment rate).
    2. Test frames never create a new connection (for traditional tests like throughput, latency).
  - Ensuring these extremes involves exhausting all port combinations and setting UDP timeouts higher than the test duration.
- Pseudorandom Port Numbers: Emphasized the necessity of "pseudorandom enumeration" using algorithms like Stanford's random shuffle, as ordered enumeration yielded different results in tests.
- Discussion on Capacity/Teardown: Asked about measuring connection teardown performance (e.g., aggregate time to delete N connections, which can be slower than establishment). Acknowledged difficulties in making a repeatable benchmark for this but recognized its significance.
- Initial feedback indicated the methodology provides meaningful results in the use cases presented.
Problems and Requirements of Evaluation Methodology for Integrated Space and Terrestrial Networks (ISTN) Draft (Zeki Lai)
- Background: Renaissance in space industry (Starlink, Amazon Kuiper), mega-constellations providing global internet service, leading to Integrated Space and Terrestrial Networks (ISTN).
- Opportunities: Pervasive network accessibility (remote users, global IoT), low latency for long-haul (inter-satellite links, near-optimal space routes), high throughput.
- Challenges: High dynamics (satellites move rapidly, connectivity changes), heavy development (architecture, protocols still evolving), high operational cost (deployment, updates).
- Existing Evaluation Limitations:
  - Live Satellite Networks: Lack flexibility, limited accessibility, constraints on large-scale/fault testing.
  - Simulations (STK, NS3 extensions, Hypertier, Starperf): High abstraction level, cannot run real system codes.
  - Emulation: Lacks ability to mimic Leo dynamics and time-varying behaviors.
  - Existing IETF Benchmarks: Focus mainly on terrestrial networks.
- Proposed Requirements: Realism (constellation/network scale), flexibility (diverse experiments), low cost/easy to use, realistic data/test cases.
- Future Path: Integrated (live, sim, emu), data-driven (real data/orbits/constellations), community-driven (cross-community collaboration).
- Benefits: For satellite operators (constellation design), ISPs (reliability, latency, convergence), content providers (congestion control).
- Standardization Potential: Evaluation methodology, benchmark steps/test cases, performance metrics (e.g., constellation utilization).
- WG Discussion: Sarah Banks inquired about operator involvement. Al Morton shared personal experience with laboratory satellite testing, suggesting a scope for BMWG work might exist even if not covering the entire system. Emphasis on the reality of current Leo services and the need to design test scopes for meaningful benchmarks.

Decisions and Action Items

EVPN Benchmarking Draft:
- Action: Al Morton and Sarah Banks to revisit the document status at the end of the meeting and propose finding new volunteers/co-authors if existing authors remain unreachable. (Sarah Banks joined the meeting late, so this was deferred).
- Action: Warren Kumari will attempt to find a YANG doctor or identify someone with YANG expertise in the WG to assist.
Next Generation Firewall (NGF) Benchmarking Draft:
- Action: Brian Monkman (author) to create a new version of the draft incorporating the shepherd's comments (editorial, reference update, "obsolete 3511" in abstract).
- Action: Brian Monkman to send the updated draft to Warren Kumari (AD) for publication review.
YANG Data Model for Network Interconnect (YANG-NI) Draft:
- Decision: BMWG will pursue this as a Standards Track document.
- Action: Al Morton to initiate a new, short working group adoption call explicitly stating the intent to adopt the document as Standards Track.
- Action: Al Morton to cross-post this adoption call to relevant mailing lists (e.g., [email protected], [email protected], [email protected]) to recruit YANG expertise.
Benchmarking Methodology for Stateful NAT (NAT-BM) Draft:
- Action: Gabor Lencse (author) to share the questions raised during the presentation (e.g., connection teardown performance) and measurement results on the mailing list to prompt further review.
- Action: Working group members are encouraged to review the draft in November/December.
Integrated Space and Terrestrial Networks (ISTN) Draft:
- Action: Zeki Lai (author) to continue working on the draft, clarifying problem statements, requirements, and potential performance metrics.
- Action: Zeki Lai and co-authors are encouraged to seek feedback directly from satellite operators and industry, and to try to recruit new audience members from these providers to BMWG.

Next Steps

EVPN: Sarah Banks and Al Morton will coordinate on next steps, potentially seeking new authors.
NGF: Authors to complete updates and submit for AD review.
MLRS: Authors to incorporate recent comments and prepare for an updated version (02).
YANG-NI: Al Morton to start the new adoption call, involving YANG doctors.
NAT-BM: Working group members to review the draft, aiming for a potential working group adoption call in January 2022.
ISTN: Authors to refine the draft and engage with the industry to further define the scope and need for benchmarking in this domain. BMWG chairs will monitor progress and industry interest.