Markdown Version | Session Recording

Session Date/Time: 10 May 2022 18:00

TOOLS

Summary

The TOOLS working group reviewed significant progress on core IETF infrastructure projects. The transition of tools.ietf.org is nearing completion, including a decision to rename the bibxml service to bib.ietf.org. The Data Tracker has been successfully upgraded, yielding substantial performance gains, and its development is accelerating with planned updates to Python 3.9, asynchronous operations, and critical bug fixes. WikiJS integration is ready, and an initial open-editing model (requiring a Data Tracker account) was favored for working group wikis. Preparations for IETF 114 include Mediaco's Zulip integration for chat, with future discussions planned regarding the role of Jabber post-114. Future-looking discussions covered Mailman 3, RFC Editor tooling, Data Tracker UI improvements, and the IETF's strategy for standardizing Markdown across its various services.

Key Discussion Points

• RFC Production: An XML RFC release is currently underway to enable proper rendering of non-latin script characters in author names, addressing a critical production requirement.
• tools.ietf.org Transition:
  • The transition from tools.ietf.org to its replacement services is expected to conclude this week with the creation of final redirects, to be deployed in the next one to two weeks.
  • The IETF's own instance of the bibxml service is now operational.
  • Following a suggestion from Mark Nottingham to decouple the service name from its specific output format, and noting that the service already provides formats beyond BibXML (e.g., Ribose's format) and may offer BibTeX in the future, the group discussed and agreed to rename the service.
  • Coordination with Ribose is ongoing to ensure their legacy service scraping stops before tools.ietf.org is delegated from the IETF infrastructure, to prevent operational loops.
• IETF 114 Mediaco/Zulip Integration:
  • All working groups will be provided with Zulip channels for Mediaco's chat backend during IETF 114. All Zulip streams have already been created, and Jabber bridges are being set up.
  • These channels are expected to be fully configured by the end of May.
  • A conversation needs to be initiated with the ISG to determine whether Jabber services and bridges will continue to be offered post-IETF 114, contingent on the success of the Zulip integration.
  • It was noted that rich text, images, and emojis used in Zulip will not translate to Jabber.
• Data Tracker (ietfa.ietf.org) Upgrade:
  • The upgrade to a newer version of the Data Tracker on new infrastructure was completed smoothly, with minimal issues.
  • Performance has significantly improved, with the Data Tracker now running approximately three times faster due to the new infrastructure and an updated MariaDB version.
  • A significant issue arose with Microsoft 365 services temporarily blocking high-volume mail from the Data Tracker's new IP address, which was resolved through direct outreach. This highlighted the need for broader community communication and considering dual-server mail infrastructure for future migrations.
• Mailing List Whitelist:
  • The ISG is discussing issues where some long-standing mailing lists lack the global whitelist configuration. Options for automatically configuring this, and the appropriate timing (now versus after the Mailman 3 transition), are under review.
  • Concerns were raised about the current whitelist mechanism, which is monotonically increasing (inactive addresses are not removed) and automatically adds all active emails from any Data Tracker account, potentially leading to an overly permissive list.
  • Discussion reinforced the need for "tiers of reputation" for Data Tracker accounts, with some lists (e.g., IOT Directorate, OSCAR 48) explicitly not having a global allow list configured.
• WikiJS Integration:
  • Significant progress has been made, and the integration of WikiJS with the Data Tracker is now considered complete.
  • Efforts to migrate existing Track wikis to WikiJS are ready to begin, with a planning phase involving staff and secretariat.
  • Discussion on Wiki Access: The permissions model for WikiJS will leverage Data Tracker roles. A sense of those present indicated a preference for an initial open-editing model (requiring a Data Tracker account) relying on community self-policing (reverting undesirable changes) before considering more restricted permissions.
• Data Tracker Development:
  • The Data Tracker has seen accelerated development velocity since its migration to GitHub, with multiple deployments from GitHub.
  • Upcoming releases include upgrading the Data Tracker to Python 3.9 (a step towards future containerization) and testing a proof-of-concept for asynchronous document submission.
  • Near-term development priorities are: completing work on non-compliance eligibility issues, adding support for the new RFC Editor model, and tackling the complex project of time zone-aware timestamp transitions. The latter is a significant undertaking, requiring a re-architecture to minimize service downtime.
  • New Priority: XML RFC Library Date Issue: A critical architectural issue was identified in the xmlrfc library where dates are loaded once upon worker startup, leading to incorrect dates on draft submissions if workers run for extended periods (now days, instead of hours). While a short-term workaround might be daily Data Tracker restarts, a permanent fix is prioritized soon after the time zone work.
  • Draft Submission Requirements: The group discussed requiring all draft submissions to have fully expanded references and includes, ensuring content stability and enabling offline use. This necessitates authoring tools that facilitate easy expansion and a streamlined submission process for authors.
  • Reference Graph Bug: A self-inflicted bug in Data Tracker's XML parsing logic for references was identified. It failed to expand includes, resulting in incorrect "referenced by" graphs for many documents. A code fix and a rebuild of existing reference relationships are required.
  • No higher priority items for Data Tracker development were identified by participants.
• Author Tools: The requirement for a Datatracker API key has been removed, and support for WD and WDF formats has been added. Community members are encouraged to test these tools ahead of upcoming redirects.
• XML RFC: Recent releases include support for non-latin characters in author names for RFCs. Upcoming work involves reviewing and closing over 100 v2-only issues on GitHub and migrating other tools' tickets from the XML RFC repository to their respective projects.
• Website (Wagtail): The Wagtail-based IETF website is in the process of moving to Python 3.10 (already containerized), while the Data Tracker is moving to Python 3.9 (on bare metal), prompting discussion on potential developer inconsistencies.
• YANG Catalog: Optimization of Matomo analytics is ongoing. The next major step for YANG Catalog is to incorporate SID.
• Web Analytics (Matomo): A configuration issue preventing Matomo JavaScript from loading on the Data Tracker due to Content Security Policy restrictions will be resolved in the next release.
• Common Bootstrap Theme: The theme has not yet been deployed to the Data Tracker. Following user feedback on contrast and accessibility, Springload will be asked to perform accessibility tests on the Data Tracker.

Decisions and Action Items

• Decision: The bibxml.ietf.org service will be renamed to bib.ietf.org. bibxml.ietf.org will be CNAME'd to the new name for a transitional period.
  • Action Item: Maddy to facilitate the necessary domain name changes.
  • Action Item: Coordinate with Ribose to ensure they cease scraping the legacy bibxml.ietf.org service to avoid operational loops during the transition.
• Action Item: Greg and the Secretariat will develop a plan for migrating existing working group wikis from Track to WikiJS.
• Action Item: Nick to ensure the wiki.ietf.org WikiJS instance's configuration supports the agreed-upon initial open-editing model (requiring Data Tracker accounts) with self-policing capabilities.
• Action Item: Springload is requested to conduct accessibility testing on the Data Tracker (either in its current state or with the common bootstrap theme applied) and provide a report.
• Action Item: For the Data Tracker, fix the XML parsing code that previously failed to expand includes for references, and subsequently rebuild the reference relationships for all affected drafts.
• Action Item: Community members are encouraged to test the updated Author Tools functionality, particularly before the planned redirects for xml2rfc.ietf.org and exp.ietf.org.
• Action Item: Review the over 100 open issues on the XML RFC GitHub repository to close those specific to v2 formats only and move issues related to other tools to their correct repositories.

Next Steps

• IETF 114 Mediaco/Zulip: Initiate discussions with the ISG regarding the future of Jabber services and bridges within the IETF ecosystem beyond IETF 114.
• Long-term Technical Focus (Next Year Outlook):
  • Deployment of Mailman 3.
  • Collaboration or close coordination with RFC Editor tooling and website development efforts.
  • Further Data Tracker UI revamp, focusing on redesigning outdated dialogues (e.g., phone editing).
  • Markdown Standardization: Address the current fragmentation of Markdown flavors used across IETF services (draft authoring, notes, wiki, buff requests, charters). The group expressed a preference for defining an IETF-specific Markdown variant that sufficiently covers all internal use cases, rather than attempting to solve global Markdown standardization.
    • Consider conducting an updated author survey to gather fresh data on authoring format preferences.
    • Explore leveraging John Levine's monthly author surveys to track changes in authoring format usage.
    • Further investigate the potential for Markdown to become the primary authoring format for the IETF review stage, with XML serving as the archival and publication format, and the necessary adjustments this would entail for the Auth48 process.