englishm

Yes, you certainly could use MoQ to send multiple streams (WebRTC simulcast style).

There are implementations of traditional receiver-side adaptive bitrate switching with MoQ already today (mostly switching between tracks at group boundaries). There has been interest in exploring sender-side adaptation as well, but it's not clear what that might require and if it's something worth trying to support in the first version of the spec we take all the way to RFC status. Subgroups are something that can be used today, though less experimentation (at least public experimentation) has been done on fully utilizing subgroups at this point. They could be used for independently decodable tracks, though that wastes bandwidth relative to layered codecs. Even layered codecs have some overhead that may not always be worth it. If RTT is low enough and switching is straightforward, it may be that having the original publisher publish multiple tracks into a relay and allowing the end subscriber to switch between them is sufficient.

We could really use more public experimentation with all of these approaches, so if anyone is looking to do that type of research, definitely let us know how we can help support it!

Yes, Safari's WebTransport implementation is still a work in progress.

Safari is still working to add full support for WebTransport.

Good question! I can't speak concretely to our plans for optimizations at that level of the stack at this stage, but it's true that speaking broadly QUIC does currently lag behind some of the performance optimizations that TCP has developed over the years, particularly in the area of crypto where hardware offload capabilities can have a major impact.

The good news is that there are strong incentives for the industry to develop performance optimizations for HTTP/3, and by also building atop QUIC, MoQ stands to benefit when such QUIC-stack optimizations come along.

Regarding GSO/GRO - I recently attended an ANRW presentation of a paper[1] which reached similar conclusions regarding kernel bypass. Given the topic of your thesis, I'd be curious to hear your thoughts on this paper's other conclusions.

[1] https://dl.acm.org/doi/10.1145/3744200.3744780

I plan to cover more of the internal implementation details at a future date, possibly at a conference this fall..

But I can at least say that we use anycast to route to a network-proximal colo.

Looks like it might be a happy eyeballs issue? I'll pass it along to folks who would know more about what that might be, thanks.

Thank you dang!

Synchronized playback is usually primarily a player responsibility, not something you should (solely) rely on your transport to provide. We have had some talk about extensions to allow for synchronizing multiple tracks by group boundaries at each hop through a relay system, but it's not clear if that's really needed yet.

Essentially though, there are typically some small jitter buffers at the receiver and the player knows how draw from those buffers, syncing audio and video. Someone who works more on the player side could probably go into a lot more interesting detail about approaches to doing that, especially at low latencies. I know it can also get complicated with hardware details of how long it takes an audio sample vs. a video frame to actually be reproduced once the application sinks it into the playback queue.

QUIC is already quite widely used! We see close to 10% of HTTP requests using HTTP/3: https://radar.cloudflare.com/adoption-and-usage

As for the NAT problem, that's mainly an issue for peer-to-peer scenarios. If you have a publicly addressable server at one end, you don't need all of the complications of a full ICE stack, even for WebRTC. For cases where you do need TURN (e.g. for WebRTC with clients that may be on networks where UDP is completely blocked), you can use hosted services, see https://iceperf.com/ for some options.

And as for MoQ - the main thing it requires from browsers is a WebTransport implementation. Chrome and Firefox already have support and Safari has started shipping an early version behind a feature flag. To make everything "just work" we'll need to finish some "streaming format" standards, but the good news is that you don't need to wait for that to be standardized if you control the original publisher and the end subscriber - you can make up your own and the fan out infrastructure in the middle (like the MoQ relay network we've deployed) doesn't care at all what you do at that layer.

So... MoQ represents a bit of a moving away from the all-in-one "black box" of web APIs like WebRTC. From the browser perspective, the main thing that matters is the WebTransport API. Using MoQT in conjunction with that WebTransport API, you now have various options for rendering the video as a player, for example: WebCodecs. But, if you can afford a bit more latency, you can also use APIs like MSE for playback and be able to use DRM.

And yeah, being able to publish from something like OBS is something I worked on before joining Cloudflare, but it depends a lot on what you do at the "streaming format" layer which is where all the media-aware details live. That layer is still evolving and developing with WARP being the leading spec so far. As that gels more, it'll make sense to bake things into OBS, etc. Already today though you can use Norsk (https://norsk.video/) to publish video using a rudimentary fMP4-based format similar to early versions of the WARP draft.

As for YouTube, Google has some folks who have been very active contributors to MoQT, but I'm not certain exactly how/where/when they plan to deploy it in products like YouTube.

Yes, exactly! I mention that in the post. Streaming formats are where a lot of interesting decisions can be made about how best to optimize QoE for different use cases. MoQT is designed have enough levers to pull to enable a lot of clever tricks across a wide gamut of latency targets, while also being decoupled from all of the media details so we can get good economies of scale sharing fan out infrastructure.

WARP's development (at the IETF) up until now has been largely spearheaded by Will Law, but it's an IETF spec so anyone can participate in the working group and help shape what the final standard looks like. WARP is a streaming format designed mainly for live streaming use cases, and builds on a lot of experience with other standards like DASH. If that doesn't fit your use case, you can also develop your own streaming format, and if it's something you think others could benefit from, too, you could bring it to the IETF to standardize.

(OP of the Cloudflare blog & submission here) I think kixelated is saying enough stuff beyond our post that he deserves the credit for and this shouldn't be treated as a dupe. (emailed to say as much also)

I e-mailed to ask they swap it back - I like your post, too!

Also, putting it out here, we're building on a _lot_ of kixelated's work with what we've deployed, so he really deserves a lot of credit.

You might be interested in looking at this RTC example for ideas about how to make bi-directional data flows for arbitrary groups of participants (or players) work through a relay.

https://hang.live/

It uses a feature we haven't yet implemented, but we're thinking about how we might implement it at our scale, SUBSCRIBE_ANNOUNCES[1].

[1] https://www.ietf.org/archive/id/draft-ietf-moq-transport-12....

It could! We've mainly been focused on using it for audio and video for live streaming and RTC use cases, but the MoQT layer is very intentionally decoupled from the media details so the fan out infrastructure could actually be used for a lot of different things. You'd need to decide how you want to map your data to MoQT objects, groups, tracks, etc.

Firefox also has WebTransport support and Safari has a work-in-progress implementation behind a developer mode feature-flag. Safari used to not work at all, but I know they've been putting more effort into it lately, so hopefully we'll be able to use MoQ in all three soon!

Hi! Cloudflare MoQ dev here, happy to answer questions!

Thanks for the award, kixelated. xD

The main thing we're waiting on is WebTransport in Safari.

Apple has committed to adding it, and there's already been some work on it in WebKit, but we don't know when it will make its way into a Safari release yet.

https://2023.demuxed.com/ has the day-long uncut videos available behind a $20 "VoD-only" post-event "ticket" paywall.

Eventually https://www.youtube.com/@Demuxed will have the edited VoDs for Demuxed 2023 available for free. Usually those go up around December-ish.

I also posted my slides on LinkedIn: https://www.linkedin.com/feed/update/urn:li:activity:7124766...

I very much disagree with the characterization of WebRTC being "designed to be overcomplicated and garbage on purpose" but I do think your point about needing to not open the door to DDoS botnet capabilities is something worth highlighting.

There are a number of very challenging constraints placed on the design of browser APIs and this is one of them that is often grappled with when trying to expose more powerful networking capabilities.

Another that's particularly challenging to deal with in the media space is the need to avoid adding too much additional fingerprinting surface area.

For each, the appropriate balance can be very difficult to strike and often requires a fair bit of creativity that can look like "complication" without the context of the constraints being solved for.

I hope P2P stays around in browsers.

I do, too.

There was a W3C draft [1] ~pthatcherg was working on for P2P support for QUIC in the browser that could have maybe become a path to WebRTC using QUIC as a transport, but I think it may have been dropped somewhere along the line to the current WebTransport spec and implementations. (If Peter sees this, I'd love to learn more about how that transpired and what the current status of those ideas might be.)

A more recent IETF individual draft [2] defines a way to do ICE/STUN-like address discovery using an extension to QUIC itself, so maybe that discussion indicates some revived interest in P2P use cases.

[1] https://w3c.github.io/p2p-webtransport/ [2]: https://datatracker.ietf.org/doc/html/draft-seemann-quic-add...

Here are a couple resources which may be helpful:

- https://www.mux.com/video-glossary

- https://howvideo.works/

Also, if you do want to take the time to read a longer and denser article, but come away with an understanding of much of the breadth of modern streaming tech, there's a really great survey paper available in pre-print here:

https://doi.org/10.48550/arXiv.2310.03256

The current moq-pub implementation only requires valid fMP4 (a la CMAF) to be provided over stdin. I haven't tested, but I imagine you can do the same with gstreamer.

Separately, I've been working on a wrapper library for moq-rs that I've been calling 'libmoq'. The intent there is to provide a C FFI that non-Rust code can link against. The first integration target for libmoq is ffmpeg. (I have a few bugs to work out before I clean up and shout about that code, but it does mostly work already.)

I gave a presentation about some of this work last week at Demuxed, but the VoDs probably won't be available on YouTube until Decemberish.

Also, I understand the gstreamer project has better support for Rust so I'll be looking at that soon, too.

I don't know the timeline, but Apple has committed [1] to adding WebTransport support to WebKit.

[1] https://github.com/WebKit/standards-positions/issues/18#issu...

That sounds like it was an interesting project! Jean-Baptiste Kempf also presented something at Demuxed last week using FEC and QUIC datagrams to get similarly low latency delivery over WebTransport into a browser. There's also a draft[1] for adding FEC to QUIC itself so it's quite possible Media over QUIC (MoQ) could benefit from this approach as well.

I'm not sure why you say "it is not." We have a working MoQ demo running already on https://quic.video that includes a Rust-based relay server and a TypeScript player. The BBB demo video is being ingested using a Rust-based CLI tool I wrote called moq-pub which takes fragmented MP4 input from ffmpeg and sends it to a MoQ relay.

You can also "go live" from your own browser if you'd like to test the latency and quality that way, too.

[1] https://www.ietf.org/archive/id/draft-michel-quic-fec-01.htm...

Here's where you can request your cached SERVFAILs be flushed from Google's public DNS (i.e. 8.8.8.8): https://developers.google.com/speed/public-dns/cache

I agree that these are separate disciplines, but I think that one reason we're not seeing anyone who does security usability work well right now is that that the best practitioners in each field tend to be silo'd by their specialization.

A single assessment is not necessarily a single metric, and an assessment comprised of audits in each domain seems like a good first step toward building understanding of a common goal and measuring progress toward it. Putting these audits together will hopefully start to expose not only the tradeoffs, but also the synergies at play in designing secure systems.

It seems to me that at the root of the security usability problem is a failure in collaboration between developers with solid software engineering practices and designers with solid UX design practices. Talent on both sides is in high demand, but there are few organizations that are able to get both working together effectively on these hard problems.