padenot

We're implementing it though: about:keyboard in a Nightly build does what you expect, this is tracked in https://bugzilla.mozilla.org/show_bug.cgi?id=2000731 and dependencies.

Browsers have been doing (sometimes tiered) AOT compilation since wasm inception.

Debian has stopped supporting x86 32bits recently, Chrome did so 9 years or so ago.

We've carefully ran some numbers before doing this, and this affects a few hundreds to a few thousand people (hard to say, ballpark), and most of those people are on 64bits CPUs, but are using a 32bits Firefox or 32bits userspace.

The comparatively high ratio of 32bits users on Windows is not naively applicable to the Linux Desktop population, that has migrated ages ago.

Those methods are sub-sample accurate, granted you call them a bit in advance to account for the cross-thread communication, as you say. But yes, in general this was designed (prior to me becoming an editor) with scheduling in mind, not with low-latency interactivity. That said, it goes quite far.

Other systems go further, such as Web Audio Modules (that builds on top of AudioWorklet) implement sample-accurate parameter change from within the rendering thread, using wait-free ring-buffers. That requires `SharedArrayBuffer` but works great, and is the lowest latency possible (since it uses atomic loads and stores from e.g. the main thread to the rendering thread).

Sorry, I'll implement it, I had forgotten we didn't do it for erm... 9 years.

https://bugzilla.mozilla.org/show_bug.cgi?id=1308431

`jj commit -i` (or a lot of commands `-i`) and maybe `snapshot.auto-track="none()"` in the config, to a certain extent is what I use. I used to do the same with mercurial. In practice, I also use `absorb` a lot, that leaves unrelated files and chunks alone.

He did last year, alongside mayor Hidalgo and others, such as Tony Estanguet, who is a former athlete and was overseeing the olympics and the minister for sports.

Article in French: https://www.franceinfo.fr/les-jeux-olympiques/paris-2024/bai...

Hi, https://blog.paul.cx/post/audio-video-synchronization-with-t... (I'm the author) will have some info about what is happenning and what to do.

You're out of luck on Safari because it seems that the important APIs aren't implemented yet: https://developer.mozilla.org/en-US/docs/Web/API/AudioContex... (scroll down). This means this cannot be made correct, e.g. there will always be a desynchronization when using high latency audio output devices such as anything wireless. Their handhelds (on which non Safari is not allowed) and their MacBooks, in wired/built-in speaker mode have excellent latency figures and we can get away with not doing anything explicit there, granted the audio is not happening somehow before the visuals, that is jarring for a human. A bit late is a lot more natural if it can't be exact.

lmk if you need further details on tight synchronization of real time audio and visuals, padenot@mozilla.com, happy to help, and congratulations on the delightful websites!

This is planned, and important, and we'll fix it hopefully soon, it's long overdue. I'm sorry this hasn't happened yet, it's always a game of priorities that can never satisfy everybody on time. It however ranks fairly high on my personal list.

As one could imagine it's a bit (read: a lot) more complicated than just pausing the AudioContext after some time of silence, but we'll get it fixed regardless, it's possible because others did it. There are tradeoffs we're willing to do.

Source: Firefox implementer of a lot of things around this, editor of the Web Audio API standard.

It does not, it serves no purpose for only listening to music.

Producing, broadcasting, or any sort of scenario that require reliable low latency can benefit from a real time kernel, but not merely listening to music.

Indeed. I work on the Firefox media stack and we have been grabbing wake locks when video playback is happening for a long time. Occasionally e.g. on some linux desktop variant this has malfunctioned and we're alerted in no time and fix it.

The Wake Lock API is for other use cases, such as recipe websites, or other document for which you don't want the screen to go away / dim, the kind where you happen to need to look at the screen for long period of time without touching it/interacting w/ mouse and keyboard.

Prior to this API being introduced, websites used to play an inaudible/almost invisible looping media file to keep the screen awake. This has power usage implication, a small single digit number of watts (1 to 3.5 depending on os, hardware, mobile or not) is required to keep audio running (because of high priority threads, frequent wakeups, and simply because the audio hardware itself needs power).

One of those libraries source, for illustration: https://github.com/richtr/NoSleep.js/blob/master/src/index.j...

This is also a technique that was used more than 100 years ago for the line 4 of the Paris métro: https://www.ratp.fr/en/discover/out-and-about/culture/histor...

Yes, I use rr all day every day (to record Firefox executions) on a rather recent Threaripper Pro 7950, and also with Pernosco. The rr wiki on GitHub explains how to make it work. Once the small workaround is in place it works very reliably.

The way this works (and I'm obviously taking a high level view here) is by comparing what is being played to what is being captured. There is an inherent latency in between what is called the capture stream (the mic) and the reverse stream (what is being output to the speakers, be it people taking or music or whatever), and by finding this latency and comparing, one can cancel the music from the speech captured.

Within a single process, or tree of processes that can cooperate, this is straightforward (modulo the actual audio signal processing which isn't) to do: keep what you're playing for a few hundreds milliseconds around, compare to what you're getting in the microphone, find correlations, cancel.

If the process aren't related there are multiple ways to do this. Either the OS provides a capture API that does the cancellation, this is what happens e.g. on macOS for Firefox and Safari, you can use this. The OS knows what is being output. This is often available on mobile as well.

Sometimes (Linux desktop, Windows) the OS provides a loopback stream: a way to capture the audio that is being played back, and that can similarly be used for cancellation.

If none of this is available, you mix the audio output and perform cancellation yourself, and the behaviour your observe happens.

Source: I do that, but at Mozilla and we unsurprisingly have the same problems and solutions.

We've been working on the Web Codecs API for a few years now. It only handles encoding and decoding media codecs, ffmpeg does much more (container handling, filters and everything needed really).

Web Codecs can take a compressed media packet and get you the decoded image or audio buffer it corresponds to. Conversely it can take audio or images with timestamps and get you a series of encoded media packets you can then containerize (we say mux) and get you e.g. an mp4 file.

https://w3c.github.io/webcodecs/

I (Firefox developer working on anything media related) got in contact with the dev on Twitter, and he told me that Web Codecs was missing (and we're shipping this in a month or so, it's been in Nightly for some time), and something to save project file to disk (https://developer.mozilla.org/en-US/docs/Web/API/Window/show...).

So I spoofed the user-agent in a nightly build here on my Linux desktop workstation, then had to alias one method that we should have implemented years ago but only have with a `moz` prefix (`HTMLMediaElement.mozCaptureStream`). This is on us to fix.

Then it looks like a worker script is served with the `Content-Type` `text/html` instead of `application/javascript` or something like that. We also have a pref flip to bypass that check, so I did that, but this is on the dev to fix.

When you do this it works, I've loaded project demos containing videos, audio, various things composited on top, scrubbed the timeline aggressively in a debug build, moved things around in various bits of the interface and also in the rendering frame, etc., things seem to work as they should, perf is as I'd expect it to be (and again, I'm running it in a debug build with optimizations disabled for anything media related, enabled for other parts of the browser).

What's missing is `window.showSaveFilePicker` and file system related stuff. It's possible to use https://developer.mozilla.org/en-US/docs/Web/API/File_System... instead (that we ship, e.g. Photoshop on the Web uses it). We think that it's much less scary than giving access to the file system to a content process of a Web browser. Maybe because videos can sometimes be extremely big files, direct access to the FS could be of use there. Thankfully, we also ship extremely modern video encoders to make them tiny instead, but that's currently a limitation Firefox has, for better or worse.

https://paul.cx/public/pikimov-firefox-nightly.webm

ShazamKit, from that fetch the BPM, BPM detection from the mic, compare ?

We're aiming to release this half, so in a month or so, on all desktop platforms, mobile will follow shortly after.

Then we'll gradually optimize (e.g. enable the use of more hardware encoders, decoding being generally in hardware at launch of supported) release after release, but generally almost everything will be supported at launch.

Please open a bug at https://bugzilla.mozilla.org/enter_bug.cgi?product=Core&comp... if that's the case, and CC padenot@mozilla.com, I'll have a look next week.

about:preferences#privacy, scroll down a bit, it's under "Permissions". You can also adjust it when on the site, using the icon at the left of the URL bar.

We've put a dedicated section called Media in about:support, it has decoding capabilities and other things such as audio IO informations.

If you find that it is not accurate, e.g. by cross-checking via other means, please open a ticket at https://bugzilla.mozilla.org/enter_bug.cgi, component "Audio/Video".

If you want to know the media decoding capabilities of a Web Browser, you can use the MediaCapabilities API. In Firefox, when flipping the "resist fingerprinting" flag, it's spoofed appropriately based on a study of what the most common results are. This is available on all engines, desktop and mobile.

Depending on the implementation, what it returns can be based on the presence of optimized software decoders for a platform, presence of hardware, resolution and other characteristics of the video, it can also be based on a decoding benchmark that the web browser runs, etc.

https://w3c.github.io/media-capabilities/

https://developer.mozilla.org/en-US/docs/Web/API/Media_Capab...

Raspberry Pi have had hardware decoder for h264 for as long as they've existed (I think?), but dropped in the most recent version. I don't understand why.

They've recently contributed non-trivial patches to Firefox to use the embedded Linux API for video hardware acceleration (V4L2, vs. VAAPI on desktop that we also support), and are shipping the h264ify extension with their Firefox build to get that codec often for their users so that the experience is good on older devices.

Maybe the 5 is that much faster than it's not needed as much, but h264 represent so much content that it feels a bit surprising anyways.

But I'm just a software person, hardware is complicated differently.

taskkill /F firefox.exe or something like that.

Firefox developer here. There are a few ways to diagnose this, depending on your setup. If everything is frozen on Linux, it's probably the parent process that is having an issue. The "parent process" is the process that oversees all the tabs, it's the top process in a Firefox process tree, the one that has lots of children.

If our crash reporter is enabled, you can also just kill the parent process with `SIGABRT`, this will produce an entry in `about:crashes` that you can look up on restart, and submit. This will then give a link to our crash reporting front-end.

If you have `gdb` and `debuginfod` on your distro, and you're using your distro's Firefox, attach `gdb` to the parent process of Firefox, let it download symbols, and do `thread all apply bt`, this will dump the stacks of all the threads of your parent process.

If you're using one of our Firefox builds, you can use https://firefox-source-docs.mozilla.org/toolkit/crashreporte..., that will integrate with our own symbol servers. Depending on your distro this might or might not work on your distro's Firefox (some distros submit their symbols to our infra to help diagnosing crashes).

Once you have some info about the crash, you can open a ticket on your distro bug tracker, or https://bugzilla.mozilla.org/enter_bug.cgi?product=Core to reach us directly, attaching the stacks, Firefox version, crash report, about:support, or anything else that you think is relevant.

Oftentimes, those freezes (on Linux) are caused by a bug between Firefox and the desktop environment, or by graphics drivers, that kind of thing, and stacks allow narrowing down the issue greatly, sometimes it's just a package update or a configuration flip away!

Just use getUserMedia as usual and it will just work, nothing special to do.

https://blog.paul.cx/post/audio-video-synchronization-with-t... has some background, https://github.com/w3c/webcodecs/blob/main/samples/lib/web_a... is part of a full example that you can run using web codecs, web audio, audioworklet SharedArrayBuffer, and does A/V sync.

If it doesn't answer your question let me know because I wrote both (and part of the web audio spec, and part of the webcodecs spec).

To paint broad strokes, the layout phase (~= take the HTML, take the CSS, determine the position and size of boxes) is largely sequential in production browser engine today. Selector matching (~= what CSS applies to what element) is parallel in Firefox today, via the Stylo Rust crate originally developed in the research browser engine Servo. Servo can do parallel layout in some capacity (but doesn't implement everything), https://github.com/servo/servo/wiki/Servo-Layout-Engines-Rep... is an interesting and recent document on the matter.

Parallel layout is generally considered to be a complex engineering problem by domain experts.

Rendering the page, as in deciding the colour of each pixel and putting them on the screen, based on the layout, style, and various other things, can be done with lots of parallelism, on the CPU or on the GPU (that is preferred on most platforms in production browser engines, these days).

https://hacks.mozilla.org/2017/08/inside-a-super-fast-css-en... is a really cool article that is related, that is a few years old but what it says is largely correct today.

It works for other code bases than Firefox, e.g. we index LLVM and webkit. It has a special understanding of the Firefox code, though.

Here's a cool class diagram with emojis: https://asuth.searchfox.org/mozilla-central/query/default?q=...

https://firefox-source-docs.mozilla.org/contributing/stack_q.... It's then two clicks to diff two arbitrary versions of a particular patch. For complex patch sets that take a few rounds of review, it's nice to understand what changed since last time you looked at it. When reviewing code from e.g. a beginner, it helps to quickly check that all comments have been addressed and nothing else has changed, etc.

Then you merge ("land") the patch stack in one unit, and it's tested in one unit, and bisection tooling knows that it is the case and it's nice.