**Session Date/Time:** 07 Nov 2023 16:00

# mlcodec

## Summary

The mlcodec meeting covered updates on several drafts, including redundancy for the Opus codec, speech coding enhancement for Opus, and Opus extension mechanisms. Discussions focused on technical details, standardization approaches, and potential future features. Decisions were made to address splicing considerations in the redundancy draft and gather further feedback on the Opus extension draft.

## Key Discussion Points

*   **Opus Redundancy:**
    *   Jean-Marc presented an update on using machine learning to achieve robustness against packet loss by encoding highly compressed redundant audio in Opus.
    *   The discussion covered the current proposed packet format including the offset field, quantizer select and slope, initial state, and latent vectors, and how it will use extension code 32.
    *   Jonathan Lennox raised concerns about splicing two streams with redundancy.
    *   Mark Harris raised security considerations about potentially including earlier audio, not attended to be sent.
*   **Speech Coding Enhancement for Opus:**
    *   Jan Buter presented an update on speech coding enhancement methods for Opus.
    *   They discussed low-complexity algorithms, quality evaluation metrics, and the standardization approach.
    *   Discussion focused on comparing different objective metrics (PESQ, WAPQ, modified OPUS compare, NOMET) and selecting appropriate thresholds for evaluating enhancement models.
    *   Tim Terrybury discussed concerns about changes that Opus encoder already makes.
*   **Opus Extension Mechanisms:**
    *   Tim Terrybury presented the status of the Opus extension draft.
    *   The discussion covered the structure of Opus packets and how extensions can be added using code 3 packets.
    *   Discussion covered the different ways of using ID 0 and ID 127.
    *   Concerns were raised about multiple frames.
    *   Discussion focused on whether to reserve extension IDs with zero and one byte payloads seperately.

## Decisions and Action Items

*   **Opus Redundancy (Jean-Marc):** Add a "Splicing Considerations" section to the draft, addressing both Opus splicing and the deep redundancy issue. Update the mailing list to try to address the issue of splicing different talkers and merging the redundancy.
*   **Opus Redundancy (Jean-Marc):** Add a "Security Considerations" section to the draft, addressing the dangers of potentially including earlier audio, not intended to be sent.
*   **Opus Extension Mechanisms (Tim):** Gather more feedback on the Opus extension draft and address concerns before submitting it for IETF Last Call.

## Next Steps

*   Jean-Marc will update the Opus redundancy draft with sections on splicing and security considerations and update the mailing list.
*   Tim will incorporate feedback into the Opus extension draft and prepare for potential IETF Last Call after further review.
*   Jan will continue the work on integration and complexity optimization of the new models in Focus, on the Opus Energy branch next generation and will investigate noisy speech performance and add outputs extension.