GeneticGenesis

Congrats to the team at Stainless, it's a great team to be joining over at Anthropic.

We were an ealy adopter of their Node SDK generator at Mux (and latterly their Typescript and other generators), and the product worked great, and I'm sad to see it be shut down.

At the same time, it's easy to understand why this is a complciated product/market to be in at the moment - it's very tempting and easy to vibe code SDKs from a OpenAPI spec files right now. I would think a lot of teams will just go in that direction (for better or worse), using the same toolchain that the product developers are using today for the product, for effectively no extra cost.

In the interests of transparent disclosure on such a positive blog post, It might be worth calling out that all the links on the page are all linked to the Bunny Affiliate Program. [1]

[1] https://bunny.net/affiliate/

I know it's not public domain per-say, but for me, the thing that's most exciting is that in 2025, the last remaining patents on the h.264 (AVC) video codec will expire [1].

Now if only HEVC wasn't such a hot patent / licensing mess.

[1] https://meta.wikimedia.org/wiki/Have_the_patents_for_H.264_M...

Disclaimer: We're an early adopter of Stainless at Mux.

I've spent more of my time than I'd like to admit managing both OpenAPi spec files [1] and fighting with openapi-generator [2] than any sane person should have to. While it's great having the freedom to change the templates an thus generated SDKs you get with using that sort of approach, it's also super time consuming, and when you have a lot of SDKs (we have 6 generated SDKs), in my experience it needs someone devoted to managing the process, staying up with template changes etc.

Excited to see more SDK languages come to Stainless!

[1] https://www.mux.com/blog/an-adventure-in-openapi-v3-api-code...

[2] https://github.com/OpenAPITools/openapi-generator

A more energy efficient dishwasher would be nice though...

Super interesting, it really depends on what you mean by "choppy".

Are you seeing the video re-buffer (a loading spinner re-appearing), or just the video looking stuttery?

My video engineer gut tells me that this is a frame rate conversion issue - do you know what frame rate your graphics card is outputting to your display, and do you happen to know the frame rate of the content that you're seeing issues with (or more generically, what type of content is it - film? TV) ?

Great question! The real answer is it varies, but for H.264, most just encode on software right now, because GPUs are expensive (especially in the cloud), and the failure rates are really high (if you try to build your own). ffmpeg and lib264 is really fast on modern hardware with decent X86 extensions.

It's also worth noting that YouTube also now builds its own transcoding chips [1], and AWS just launched a dedicated transcoding instances based on Xilinx chips:

[1] https://arstechnica.com/gadgets/2021/04/youtube-is-now-build... [2] https://aws.amazon.com/ec2/instance-types/vt1/

Hey, I work in the Product team at Mux, and worked on the LL-HLS spec and our implementation, I own our real-time video strategy too.

We do offer LL-HLS in an open beta today [1], which in the best case will get you around 4-5 seconds of latency on a good player implementation, but this does vary with latency to our service's origin and edge. We have some tuning to do here, but best case, the LL-HLS protocol will get to 2.5-3 seconds.

We're obviously interested in using WebRTC for use cases that require more real-time interactions, but I don't have anything I can publicly share right now. For sub-second streaming using WebRTC, there are a lot of options out there at the moment though, including Millicast [2] and Red5Pro [3] to name a couple.

Two big questions comes up when I talk to customers about WebRTC at scale:

The first is how much reliability and perceptual quality people are willing to sacrifice to get to that magic 1 second latency number. WebRTC implementations today are optimised for latency over quality, and have a limited amount of customisability - my personal hope is that the client side of the WebRTC will become more unable for PQ and reliability, allowing target latencies of ~1s rather than <= 200ms.

The second is cost. HLS, LL-HLS etc. can still be served on commodity CDN infrastructure, which can't currently serve WebRTC traffic, making it an order of magnitude cheaper than WebRTC.

[1] https://mux.com/blog/introducing-low-latency-live-streaming/ [2] https://www.millicast.com/ [3] https://www.red5pro.com/

We actually built How Video Works as a side project at Mux [1] (inspired by How DNS Works [2]) - there's a note about it at the top of the page. We have contributions from our own team as well as others in the industry.

Our main motivation is to try to educate on the complexities and intracies of streaming video. Despite streaming video representing 80+% of the internet, it's all underpinned by a fairly small community of engineers, which we're eager to help grow through tools like this, and the Demuxed community [3].

Edit: I should also mention that Leandro was kind enough to adapt a this content from his amazing Digital Video Introduction [4]

[1] https://mux.com/ [2] https://howdns.works/ [3] https://2021.demuxed.com/ [4] https://github.com/leandromoreira/digital_video_introduction

DRM'd video content uses the same video codecs and containers, but introduces segment encryption during the packaging phase. In most cases, this encryption is within the audio and video samples rather than on the entire segment. Most content is encrypted using MPEG Common Encryption (CENC) - though there are a couple of variants.

Decryption keys are then exchanged using one of the common proprietary DRM protocols, usually Widevine (Google), Playready (Microsoft), or Fairplay (Apple). The CDM (Content Decryption Module) in the browser is then passed the decryption key, so the browser. Can decrypt the content for playback.

While HLS isn't the cleanest protocol (Yay for extensions of the M3U playlist format...), it's actually really good at what it's designed to do - provide reliable video streaming while using HTTP/S over variable networks.

Ultimately, HLS isn't designed for downloading and storing videos, it's designed for streaming.

Do you have a marketing site or landing page for Padka? It sounds interesting, but just linking to a .pkg file on S3 really isn't the way to educate HN users about your offering!

Looks like this is the homepage: https://www.padka.com/ - why not just link there?

So many permissions, but meanwhile autoplaying video and audio is still hidden behind a horrible heuristic model in Chrome that fundamentally prioritises internet giants like YouTube, Netflix and alike.

I work in the video streaming industry, and we continue to support TLS 1.1 extensively for a wide range of "smart" TVs and set-top boxes, very frustrating, there was even a long period where large CDNs were trying to shut down 1.1 and realised they'd lose a lot of business in the streaming space if they did...

This was briefly interesting and then I quickly started to see blatant racism, threats and rape accusations.

I don't realistically see this as being a killer feature for most users, but it's super interesting none the less.

Sony are saying the bitrate will be "Up to 80Mbps", which is around 3x that of most UHD streaming service profiles. However there will be significant diminishing returns on the perceptual quality at this bitrate - 20Mbps to 40Mbps for example will only increase perceptual quality by a couple of percentage points. Much of this "Up to 80Mbps" will be wasted perceptually speaking.

It's also worth keeping in mind that the encoding on BluRays is often lazy because there's lot of space available, where as a huge amount of work goes into encoding content for streaming delivery in the lowest bandwidth possible for a given target quality, using technologies like per-title and per-scene encoding. [1] [2] [3]

[1] https://netflixtechblog.com/per-title-encode-optimization-7e... [2] https://bitmovin.com/per-scene-adaptation-going-beyond-bitra...

This isn't accurate at all. HLML 5 video, MSE etc. have a world of improvements over the mess of FLV / Flash / RTMP streaming.

Can you give some examples of what's wrong with HTML5 video streaming and these implementation issues? There's a huge community of video engineers working to make streaming video better, and we'd love to hear feedback.

Great question.

There's a few approaches depending on your target latency. If you need to get below 2 seconds, yes, you'll likely want to use a WebRTC based technology to deliver the video. There's a couple of open source approaches out there, including Pion [1], which is a go implementation of a WebRTC stack. You could also build something on top of Jitsi [2]. Commercially there's also a few solutions, including Milicast [3], Red5Pro [4] and others.

The biggest problem with WebRTC based stacks is that the cross device compatibility is still generally poor, and the cost of operation is generally very high, as you can't use commodity CDNs for delivery.

If you're comfortable around the 2-5 seconds latency mark, there's more traditional HTTP based technologies available. MPEG-DASH has a Low Latency mode which uses chunk transferred HTTP fragments, and Apple has recently introduced a Low Latency HLS mode, which works in much the same way [5] [6]. You can build LL-HLS and DASH-LL solutions on top of open source toolchains like Streamline [7], and use commodity CDNs for delivery to reduce cost.

[1] https://github.com/pion [2] https://jitsi.org/ [3] https://www.millicast.com/ [4] https://www.red5pro.com/ [5] https://mux.com/blog/low-latency-hls-part-2/ [6] https://mux.com/blog/the-low-latency-live-streaming-landscap... [7] https://github.com/streamlinevideo/streamline

No worries!

While some of the larger UGC platforms do use VP9 for live streaming, it is only for a limited subset of high concurrent viewer streams, and yes, in many cases those aren't running the encodes on commodity hardware.

As for AV1, there really aren't any live implementations ready right now, a few have been demo'd, but I'm not aware of any deployments today.

Sure, but this was my point, x264 is very fast on commodity hardware, the overwhelming amount of video delivered is still h.264, especially on live video, and in particular for low-viewership, live user generated content.

If you want/need to transcode to VP9 or AV1 at scale with good quality, yes, you'll absolutely need GPU or ASIC accelerated encoders, of which there are a couple for VP9, and none for AV1 today.

h.264 will get you to 95-99% market penetration on the device landscape.

Great question, this is a large, complicated topic, but here's a quick overview. For context I've been building live streaming video infrastructure for a little over 10 years now.

At a fundamental level, yes, the encoding is one of the most expensive components of a live-streaming system (at low scale), and honestly, your guess of 2 bitrates for each video is very much on the low end - generally on average most platforms create about 5 different qualities created for any one stream, ranging from ~500kbps to ~5+mbps.

If you look at the pricing of modern video platforms, you can see the high cost of ingest and transcode captured in their pricing:

AWS IVS - $2.00 per input hour. API.video - $2.40 per input hour. Mux - $4.20 per input hour.

Generally there isn't too much use of hardware acceleration on ASIC or GPU for h.264 processing today. FFmpeg (x264) is plenty "fast enough" when tuned on commodity X86 hardware, when you have things like AVX extensions. Generally transcoding a 4-6 second segment of video should only be taking a couple of hundred milliseconds at a maximum.

As for how the larger platforms deal with keeping live streaming cost competitive, the number of qualities (and often codecs used) varies depending on the number of viewers you have, so that they're not wasting resources encoding for a couple of viewers. Many platforms also implement just-in-time encoding, to limit the amount of content that's transcoded when it isn't being viewed.

Some platforms also drop all the way down to a transmux for streams with very low viewer numbers - transmux in this context just changes the packaging of the inbound stream without changing the actual encoded picture data.

It's also worth considering from a business perspective that many UGC live platforms will also be taking a loss on small streamers with a low number of viewers, and covering that with the revenue from larger streamers with ad revenue / subscribers.

I hope that helps!

Great question, it's actually not quite that simple. HTML5 was indeed supposed to bring trivial video to the browser, and it almost did.

If you want a good video streaming experience for all your viewers, you need something called Adaptive Bitrate (ABR). Some protocols that you may have heard of the implement ABR include HTTP Live Streaming (HLS) and MPEG Dynamic Adaptive streaming over HTTP (DASH).

ABR allows the stream to adapt to the bandwidth capabilities of the viewer. If you don't do this, some users will experience buffering, and some will have to sacrifice on quality.

Unfortunately most browsers don't support an ABR format natively, so you need to use Media Source Extensions, and a more complex player to support these protocols in a browser.

Please correct me if I'm wrong, but my understanding was that Cloudflare should not be used to deliver video files unless using Cloudflare's "stream" product, IE specifically this [1].

[1] https://community.cloudflare.com/t/cloudflare-how-not-to-vio...

I don't really understand why this is being voted up.

There's zero documentation on what formats this supports, how to use it in another project etc.

It feels a little like a voting ring here to be honest.

Mozilla's choice to add their own pip button is a massive pain for anyone developing video applications - anywhere over the player can't be considered a safe space to add our own buttons anymore, it really feels like Mozilla didn't talk to any video service developers when deciding how to approach this problem.

Pip needs to be common API driven (as Chrome is) to make building portable video applications possible.

A terrible misstep from Mozilla. (And yes, this has been repeatedly raised on the bug tracker with no answer)

Hey, I've worked in streaming video for 10 years now, and I co-organise the biggest video streaming conference in the world. Here's my insights.

If you want to build a video platform from scratch, your cost models are going to be dominated by the combination of data egress and CDN costs. Transcode and storage costs are non-trivial, but do quickly become noise against delivery costs.

As CyberFonic mentioned, AWS has a variety of media products, which for transcode and storage work well, but then to deliver you'll need a CDN to deliver the content. The simplest way of doing this is to just hook up CloudFront, but the list prices for CloudFront are very high ($0.085/GB in the US), you'll be able to negotiate this down, but Cloudfront isn't actually a great CDN performance wise (particularly outside the US), and eventually you'll want to use a multi-CDN strategy for performance and reliability reasons.

Then comes the second hit, if you store your content in AWS and need to egress to a third party CDN, you can be paying very high data rates on that data transfer, particularly in regions such as Asia and Australia / NZ.

Now, honestly, if you're small and have a lot of time to invest, go for it, it's an amazing learning experience, you can learn a lot about how streaming video works on these sites and playlists:

https://howvideo.works/ https://awesome.video/ https://www.youtube.com/channel/UCIc_DkRxo9UgUSTvWVNCmpA

Honestly, my recommendation is to focus on content, and user experience, and utilise modern off-the-shelf video services, like:

Mux - https://mux.com - An API first video infrastructure provider (Disclaimer: I work here) Cloufdlare Stream - https://www.cloudflare.com/en-gb/products/cloudflare-stream/

As JB said, this is a pretty poorly cut together video and article.

We had JB on the Demuxed Podcast last year, and he told a lot of the early stories around VLC, it's much less edited than the version in this article. Here it is, with a full transcript: https://www.heavybit.com/library/podcasts/demuxed/ep-8-video...

TikTok is a strange one - It has some great content, but if you scroll through the home feed, you still stumble on pictures of kids dancing and lipsyncing in... inappropriate ways.

I think this is a pretty limiting barrier to entry for a lot of users.

Author here.

You should have read the first draft

Wow what a terrible website. The GDPR advertising opt-out:

1) Renders at the top of the page when the page is loaded, then because this link contains an anchor scrolls you to that anchor, while the entire screen is greyed out

2) Your behavior is actively being tracked by default, every click, mouse move is sent to a third party.

3) The GDPR opt out process fails every time.

Solid violations of GDPR, well done.