Session Date/Time: 11 Jan 2022 19:00

TOOLS

Summary

This interim meeting of the TOOLS team focused on updates and coordination across various ongoing infrastructure projects, including the IETFA server rebuild, Subversion/Trac migration, Author Tools production rollout, and DataTracker improvements. Key discussions included the strategy for ingesting RFC Editor mail lists, the potential use of Zulip by IAB/IESG, and the future of web analytics for IETF properties. Decisions were made regarding mail list migration and providing test Zulip environments. The meeting also touched on preparations for IETF 113, including a new registration requirement and implications for meeting conversions. Significant work remains for the Mailman 3 and DataTracker Bootstrap 5 transitions.

Key Discussion Points

IETFA Rebuild: Glenn reported that the IETFA server rebuild is underway but may slip to late February for readiness due to time constraints. The team hopes to start testing sooner, with a critical cut-over window around IETF 113. The preference is to deploy before IETF 113 to benefit from newer OS and hardware improvements.
RFC Editor Mail Lists Ingestion: Discussion centered on ingesting mail lists from rfc-editor.org into the IETF Mail Archive. Miriam noted this becomes more relevant with the new RFC Editor model. A proposal was made to move these lists into the current IETF mail processing setup before the Mailman 3 transition, then subsequently migrate them along with other lists to Mailman 3. Glenn identified no major problems with this approach.
Subversion/Trac Conversion: The migration is nearing completion, with several simple projects ready to move this week. Three working groups using Trac for document development are being transitioned, with one group requesting a delay until after January 21st, which was accommodated. The larger tools like XML2RFC, DataTracker, and Mail Archive will follow.
tools.ietf.org Transition: The plan to de-allocate tools.ietf.org and replace services with redirects is progressing. Author Tools is being prepared for production deployment before IETF 113, while replacement for Daily Dose and BAP might extend beyond this timeline, with an interim plan for BAP.
Zulip Server Deployment: The Secretariat is working on procedures for using the Zulip server. A discussion was held regarding IAB and IESG potentially adopting Zulip. Both bodies expressed commitment to Slack for the current term and requested test environments to explore Zulip before making a decision with the next slate of members in March, highlighting the importance of community feeling in a virtual environment.
FTP Services Shutdown: The shutdown of FTP services is scheduled for April 18th, with redirects and page updates already in place. Roman has volunteered to assist users with migration.
Web Analytics (Matomo): Greg reported on over a year of Matomo analytics on www.ietf.org, finding no unexpected issues and noting its utility for content observations. Eric advocated for extending analytics to other IETF properties like the Yang Catalog, especially for API usage, which would require weblog analysis beyond Matomo's JavaScript tracking. Concern was raised about filtering out significant robot traffic (80-90% of visits being less than 10 seconds). Aggregate reports are public, but detailed access is limited to operators due to privacy considerations.
DataTracker Status & Improvements:
- Restart Stability: Persistent issues with DataTracker restarts causing contention and low request serving rates were discussed. Efforts are underway to prioritize resolution and improve stability with careful restart procedures.
- Infrastructure Modernization: A sandbox instance of DataTracker is running behind Cloudflare, Nginx, and Gunicorn (instead of Apache/mod_wsgi). If successful, this stack will be adopted for production to separate listen queues and improve insight into TCP handling.
- Performance & Caching: A prototype for caching menu content by login is in queue for deployment. A move to a newer MySQL version with the new server is expected to significantly boost query performance.
- Nom-Com Eligibility Fix: A bug in registration data counting hackathon attendees as meeting attendees, affecting nom-com eligibility, was identified. A change to model usage (not schema) with a unique index over person, meeting, and ticket type is being implemented. Long-term, there's a plan to integrate directly with Mitiko's API for session attendance.
- GitHub Transition: The move to GitHub for DataTracker development will cause a temporary disruption as new processes are established.
- Major Upcoming Work: Two significant projects are the deployment of timezone-aware timestamps and the transition to Bootstrap 5.
Mailman 3 Transition: This transition will require a heavy rewrite of the mail processing chain, including Python 2 components like postconfirm. It is considered a development effort, not just configuration, and will necessarily happen after the new server is online. The possibility of transitioning lists incrementally rather than in bulk was raised.
IETF 113 Registration System: A new requirement for IETF 113 is that users must log in with DataTracker credentials before registering. This change will be communicated, and the system will guide newcomers. Alexey noted that a last-minute conversion from in-person to online for European meetings would likely necessitate refunding and re-registration due to VAT complexities.
DataTracker Bootstrap 5 Transition: Lars provided an update, noting the extensive work involved in updating templates and fixing JavaScript test breakage. He emphasized the need for a dedicated development branch for this massive change to allow broad community testing and feedback, especially for UI elements and theming decisions (vanilla Bootstrap 5 vs. mimicking old DataTracker look). Nick will be building tests using Cypress.

Decisions and Action Items

RFC Editor Mail Lists: The RFC Editor mail lists will be moved into the current IETF mail processing system before the Mailman 3 transition, and then subsequently migrated to Mailman 3 along with other lists once the new server is operational.
Zulip Exploration: IAB and IESG will be provided with dedicated spaces on the Zulip server to explore its functionality as a test instance.
Web Analytics Expansion: The team will work towards extending Matomo analytics to most IETF web properties (e.g., DataTracker, Author Tools), with a general presumption to add analytics unless there are clear reasons not to. Additionally, efforts will be made to establish a uniform approach to weblog analysis for API accesses and direct downloads across sites.
Subversion/Trac Migration: Alexey's working group repository migration will be delayed until after January 21st.
IETF 113 Registration System: The Secretariat will discuss with Ryan the process for handling registration conversions (e.g., in-person to virtual) for IETF 113, particularly regarding refund/re-registration due to VAT implications.
DataTracker Bootstrap 5 Development: The Bootstrap 5 transition will proceed using a dedicated development branch and container builds, facilitating early and broad testing by the community.

Next Steps

IETFA Rebuild: Continue work on the IETFA rebuild, aiming to start testing as soon as possible and carefully plan the cut-over date around IETF 113.
RFC Editor Mail Lists: Implement the migration of RFC Editor mail lists into the IETF mail processing chain once the new server is up.
Subversion/Trac: Complete the remaining simple project migrations and then proceed with XML2RFC, DataTracker, and Mail Archive transitions.
tools.ietf.org: Continue efforts to de-allocate the domain before IETF 113 and complete the Author Tools production rollout.
Author Tools: Encourage community testing of Author Tools as it moves towards production.
Zulip: Set up test instances for IAB/IESG and gather feedback.
Web Analytics: Investigate methods to detect and filter web spider traffic from analytics. Initiate work on a uniform weblog analysis for API access and direct downloads.
DataTracker:
- Continue work on the Cloudflare/Nginx/Gunicorn transition for stability.
- Deploy the prototype for caching menu content.
- Implement the fix for nom-com eligibility calculation.
- Complete the timezone-aware timestamp work.
- Continue the Bootstrap 5 transition, engaging the community for testing and feedback on theming.
Mailman 3: Plan the Mailman 3 transition as a significant development effort to occur after the new server is online.
Communication: Ensure clear communication to the community about the new DataTracker login requirement for IETF 113 registration.