High Quality Video Encoding at Scale

http://techblog.netflix.com/2015/12/high-quality-video-encoding-at-scale.html

by hepha1979 • 11 years ago

129 66 11 years ago

techblog.netflix.com

spdustin• 11 years ago

It's a fascinating look into how they produce multiple encoded formats from source video, but something stuck out at me... One feature in their ingestion system is meant to reject source inputs that would lead to a poor viewer experience. This doesn't seem to include interlaced NTSC video as a "poor experience" metric. I'm willing to bet that the majority of the screens on which Star Trek: Deep Space Nine is played are progressive scan, yet that show is interlaced at NTSC resolution.

In cases like this, where the source material is analog and, I would have to assume, not available in progressive scan, is there a technical reason why Netflix doesn't de-interlace the source before encoding? DS9 seems big enough of a catalog to be worth encoding in a less jarring scan rate.

Not complaining, mind you. The DVDs are interlaced too. Seems that the original recordings were NTSC only. Honestly curious if anyone has any insight.

castell• 11 years ago

The probably used DS9 DVDs as source. In fact DVD rental was their original core business.

jfb6• 11 years ago

It's possible, but the way the rights are assigned, they more than likely went back to tape and had them recaptured.

dbcooper2• 11 years ago

Are you sure they aren't 24p that's been flagged as NTSC interlaced but is actually converted to 30p via 3:2 telecine?

speakeron• 11 years ago

TNG was like this (sourced from a video version and with a very nasty interlace look) on Netflix up to a couple of months ago. Now they stream the magnificent remastered blu-ray HD version which is sourced from the original film with fresh CGI since the original CGI was tied to the video.

It's not clear whether DS9 will get the same treatment.

dbcooper2• 11 years ago

I prefer to watch SD rips from the TNG blu-rays. The merciless clarity of 1080p just kills the magic.

tertius• 11 years ago

So now I have to watch TNG all over again?!

jfb6• 11 years ago

If size of catalog trumps video quality, you roll with NTSC if NTSC is the best you can get. Deinterlacing is extremely tricky, and if you can deliver interlaced video to playback devices, you're often better off punting.

lmm2• 11 years ago

If you deinterlace an interlaced source before encoding you either discard information or store twice as much. And potentially people watching on an interlaced screen lose data - not all deinterlace/interlace pairings roundtrip cleanly. Better to store it in the original format, and then it can go through one round of deinterlacing on playback for those screens that need it, and none for those that don't.

(Also deinterlacing approaches get better over time - if a particular episode entered their catalogue 10 years ago and was deinterlaced using the state of the art approach at the time, it would look much worse than a modern deinterlace)

jfb6• 11 years ago

One of the overriding constraints on these sorts of video pipeline tasks is touching the input pixels as few times as possible. Every lossy transformation you do (cropping, scaling, color correction, transcoding) potentially introduces defects; and that's assuming that your lossless transformations (repackaging container formats, for instance) are bug-free.

I'd love to see some figures from Netflix' QC team; I bet at their scale they see all kinds of insane edge case problems.

izacus• 11 years ago

Uhm, no, due to the way modern video formats work, you don't store twice as much data - on the contrary, H.264 and similar modern formats are significantly more efficient at storing progressive (including deinterlaced) video than equivalent interlaced stream.

Gelob• 11 years ago

That isn't Netflix's job. A post house delievers the raw .mov files to Netflix. The post house gets the files or tapes from the studio or whomever manages the libary for them (aka another post house). The post house has to comply to a set of standard's set forth by Netflix or it is automatically rejected. Most of the time things like no interlacing is in those requirements but also there can be exceptions if no other source material is available. iTunes works the same way. So say the post house that delievers to Netflix is only getting digital raw .movs and not the actual tapes, they may not be allowed to re-encode those or have access to the tapes to do the correct conversion.

n0us• 11 years ago

Fascinating article. Forgive me for being ignorant about this, but how often do they need to encode video?

I would think that it's only done when new titles are added to streaming, and once the video has been encoded into all the required formats they would be done with it. Sure, there is a lot of video content out there to be encoded but it isn't unlimited. Is serving the content and providing the recommendation engine to users at scale not a greater challenge than encoding the video?

kyriakos• 11 years ago

Its another one of their challenges. Time is an important factor, encoding high quality video is time consuming, multiply that by the many bitrates and codecs it could mean that they need to delay their content availability by days.

jfb6• 11 years ago

When I built a similar system (for a large consumer electronics company) we built parallel paths for high-priority content; we'd intelligently split the incoming mezzanine and distribute the 90-120s chunks to a farm of systems, while also completing the multi-pass encodes. When the latter finished, the system would swap them. Because of the business model, we never ran this in full production mode, but it was built and ready to go.

mkagenius4• 11 years ago

Yes, encoding needs to be done only once.

Generally it would take 2-3x of original duration of video to encode a source into a 1080p, so I am not sure why they take full 1 day? unless they do each bitrate serially which I think is not as hard to parallelize as it is to parallalize single bit rate by chunking.

Yes, I believe serving is lot harder, but serving is almost a solved problem since people are dealing with for long time.

mkaufmann• 11 years ago

For x264 that is true, HEVC which is also mentioned is much slower. For a 4k source transcoding can take more than a second per frame. For a normal movie this can quickly result in encoding times of more than a day.

Another problem is that you have to encode the movie for each codec profile times the number of different bitrates per profile. The article mentions four profiles (VC1, H.264/AVC Baseline, H.264/AVC Main and HEVC) and bitrates ranging from 100 kbps to 16 Mbps. Assuming now there are 20 different bitrates per code you already get 4*20 => 80 encoded copies per source. But of course this can be solved by parallelism.

ksec• 11 years ago

Well do they not get re encode once in a while? I am pretty sure the x264 encoder now is significantly better then the one 3 - 4 years ago. Same goes to HEVC.

garblegarble4• 11 years ago

I am not sure why they take full 1 day? unless they do each bitrate serially which I think is not as hard to parallelize as it is to parallalize single bit rate by chunking.

I think the talk I know this from is https://www.youtube.com/watch?v=tQrsz3BrfwU - they chunk not only for encode but also for QC (and QC validation on the resulting transcoded asset).

If memory serves the talk also discussed the long transcode time, because their transcoder (EyeIO at the time and I have not heard differently since) is optimised for efficient packing over performance

jedberg• 11 years ago

There are lots of reasons a reencode might be needed. For example, a new compression algorithm is developed, a new device is supported with a new codec, a new way of giving users a faster startup is developed, etc.

Basically any change to the way video is delivered over the internet could trigger a full or partial reencode of the entire library.

mkagenius4• 11 years ago

Interestingly, not even a single mention of ffmpeg.

doppel• 11 years ago

I was wondering the same thing. I find it highly unlikely that they are doing this processing with in-house created tools, although their limited set of input and output codecs shrinks this down from "practically impossible" to "improbable".

legulere• 11 years ago

I guess using ffmpeg commercially is hard because of patent issues.

MichaelGG• 11 years ago

I would be surprised if Netflix cannot pay the patent fees. The ffmpeg page mentions that commercial ffmpeg users end up paying the groups like MPEG LA. In general, I'm guessing patent owners don't like saying "no, we don't want to accept your money".

https://www.ffmpeg.org/legal.html

astrange2• 11 years ago

It's possible their AVC encoder is not x264, and their VC1 encoder might be from Microsoft, in which case it would be a pretty modified ffmpeg. And some of the input formats might not be using it either.

But you can be sure it's involved somewhere, since there's no other ProRes decoder that works on Linux!

coalescence• 11 years ago

FFmpeg supports Prores decode and encode? FFMBC for sure last time I checked

contingencies2• 11 years ago

there's no other ProRes decoder that works on Linux!

Lightworks supports ProRes: http://www.lwks.com/index.php?option=com_content&view=articl...

garblegarble4• 11 years ago

It's my understanding they use a transcoder from EyeIO rather than FFmpeg

mkagenius4• 11 years ago

I haven't yet watched your linked video fully but at 8:00 he mentions they used ffmpeg in 2013:

https://youtu.be/tQrsz3BrfwU?t=480

Keyframe3• 11 years ago

Has Netflix started with HEVC h.265 support? I see they mention HEVC briefly.

virtuallynathan• 11 years ago

I think their 4K streaming to TVs uses HEVC.

JustSomeNobody2• 11 years ago

Interesting post. Netflix's posts usually are.

And then you scroll to the comments on the page...

camperman2• 11 years ago

I'm envious of these capabilities. I wrote the backend for a streaming service that streams about forty channels to a quarter of a million mobile users in Africa. We get source video from our content providers that has given me grey hair. The problem is that I simply can't spin off the encoding and source quality checks to the cloud because of bandwidth costs here. So I do the quality checking and compression on local servers and then upload the compressed output to the servers.

I'd kill for a 100 megabit line at a decent price.

noir_lord• 11 years ago

That sounds fascinating, what is the video used for?

camperman2• 11 years ago

It's literally a bunch of TV channels: sport, soaps, educational, news and music videos.

whatever_dude• 11 years ago

As someone who encodes video for work with a ton of ffmpeg-based batch files, this article makes me feel like a child.

Splines• 11 years ago

I've done a bunch of playing with ffmpeg at home, and I imagine the tech stack is probably similar at Netflix, at least for the Source-->Chunk, Chunk-->Assemble, and Assembled-->Encode steps.

The validation done during all of these is interesting. Netflix's early years are probably exactly like what you're doing - single file in, transcode, single file out and deploy.

Chunking the pieces up is clever. Getting it right must have been challenging. How do you write an oracle for something that complex?

jfb6• 11 years ago

How do you write an oracle for something that complex?

You start with a much smaller problem and incrementally build up.

contingencies2• 11 years ago

I personally worked on a similar system for a major mobile video solutions provider six years ago: none of this is particularly new.

High Quality Video Encoding at Scale

Related Stories