Be your own video hoster with Docker and Nginx

https://datarhei.github.io/restreamer/

by betablocker • 11 years ago

118 30 11 years ago

datarhei.github.io

unnikked• 11 years ago

Awesome!

smegel4• 11 years ago

Does docker run on FreeBSD? Because it is crazy to run NGINX on anything other than FreeBSD because of Linux's appalling bad async IO support. For all I know Windows might be a better choice.

corobo2• 11 years ago

Out of interest not challenge, do you have any benchmarks for this at all? I'd be interested in seeing the end results

smegel4• 11 years ago

Well when NGINX introduced thread-pools on Linux to handle async tasks (mainly waiting for block device IO) I think the speed-up was around 8x. Thread-pools make NGINX on Linux tolerable, but it just works flawlessly and beautifully on FreeBSD.

davexunit• 11 years ago

What a bold, unsubstantiated claim.

skreuzer• 11 years ago

Docker does run of FreeBSD but it heavily relies on ZFS and the 64bit Linux compatibility layer.

https://wiki.freebsd.org/Docker

rubiquity• 11 years ago

Aside from the thousands of applications using Nginx on Linux with fantastic results. Yes, yes we know *BSD has a better networking stack than Linux. Now that you're done letting everyone know how smart you are, please sit down.

smegel4• 11 years ago

Aside from the thousands of applications using Nginx on Linux with fantastic results.

Citation needed. Name a big company using nginx on Linux to serve up lots of static content.

Yes, yes we know *BSD has a better networking stack than Linux.

Who mentioned anything about a network stack?

Now that you're done letting everyone know how smart you are

Well given some people like you clearly don't understand the point I was making I think justifies the comment in the first place.

nl2• 11 years ago

There are millions of Ngnix on Linux production implementations.

And your advice is to use experimental[1] Docker support in FreeBSD, when Nginx would be running inside Docker on top of FreBSD's Linux compatibility layer.

Finally - and most critically - Ngnix introduced thread pools to specifically work around a limitation of Linux's async IO implementation:

Although Linux provides a kind of asynchronous interface for reading files, it has a couple of significant drawbacks. One of them is alignment requirements for file access and buffers, but NGINX handles that well. But the second problem is worse. The asynchronous interface requires the O_DIRECT flag to be set on the file descriptor, which means that any access to the file will bypass the cache in memory and increase load on the hard disks. That definitely doesn’t make it optimal for many cases.

To solve this problem in particular, thread pools were introduced in NGINX 1.7.11.[2].

[1] https://wiki.freebsd.org/Docker

[2] https://www.nginx.com/blog/thread-pools-boost-performance-9x...

jkldotio• 11 years ago

I heard a while back - not sure where, it could have been on Slashdot but it escapes me - that FreeBSD was dying. Looking into the statistics just now it seems that's the case. FreeBSD is basically dead on the desktop[0] and is dying, no longer even holding a single percent, in the server market.[1] In the High Performance Computing (HPC) market, Linux holds over 99.8% of the top supercomputers.[2]

[0] https://www.netmarketshare.com/operating-system-market-share...

[1] http://w3techs.com/technologies/details/os-freebsd/all/all

[2] http://www.top500.org/statistics/details/osfam/1

Animats• 11 years ago

This isn't being your own video hoster. This is the input side for feeding the live streaming feature of YouTube, etc. It's not a playout or distribution server, where viewers get their video direct from your server. That takes more bandwidth than you can pump through a Raspberry Pi or up most DSL lines. Also, most ISPs don't like you running a server, and streaming video servers are rather noticeable.

Being your own video hoster usually means renting a server in a data center with serious bandwidth, or buying streaming service from some cloud provider.[1][2]

[1] http://www.streamingvideoprovider.com/prices.html [2] https://aws.amazon.com/cloudfront/streaming/

toomuchtodo2• 11 years ago

True, but you can use this to pipe your video source to a CDN for wide distribution.

EDIT: If you're a CDN, it'd behoove you to extend this project to allow for one-click endpoint configuration. Just add API key/secret.

dogma11383• 11 years ago

I'm streaming from my home plexn box over my internet line even at 1080p the limit is more often than not the internet I have on my portable device at the moment.

Within Europe 1080p is mostly manageable sub 720 is doable even in Asia.

In have 40ish mbit upload capacity.

Never had complaints from my ISP.

frik• 11 years ago

40Mbit upload is common?

sandworm101• 11 years ago

Many sub-pewdipie youtube celebs have expressed a desire to host their own content. They want to break free of youtube, but the bandwidth and legal hassles are too much. A turnkey setup read to go on AWS or azure would probably suit them. This isn't that.

z3t4• 11 years ago

By the looks of it, it actually IS a fully featured live video stack including "your own video hoster". But sure, if you want netflix scale, a Raspery Pi on ADSL wont do, at least not for now.

m00dy• 11 years ago

Hej, What is the revenue model ?

toomuchtodo2• 11 years ago

Perhaps its just a cool project not trying to make money?

bambax• 11 years ago

There's a typo on those links in the FAQ (restreamer is repeated):

https://datarhei.github.io/restreamer/restreamer/wiki/dynami...

should be

https://datarhei.github.io/restreamer/wiki/dynamicdns_en.htm...

and

https://datarhei.github.io/restreamer/restreamer/wiki/guide-...

should be

https://datarhei.github.io/restreamer/wiki/guide-buy-hardwar...

(but on that page, all links point to a 404).

dschiptsov2• 11 years ago

Why should one put anything under Docker for production?

dschiptsov2• 11 years ago

Perhaps, chrooted environment is required for effeciently serving static files, I don't know.

And of all apps - nginx, which has been designed to be self-contained (process-isolation is good enough) and has almost zero external dependencies (openssl and pcre, which are parts of any modern OS). I could understand the idea to "containerize" that pile of Java crap with its 100500 maven dependencies just in order not to touch it again, but nginx...

BTW, does Docker's "port mapping", which is, of course, has zero overhead, is what one needs in front of nginx in production?

kevan• 11 years ago

Nginx is only one piece of the puzzle for this app, check out the Dockerfile[1].

[1] https://github.com/datarhei/restreamer/blob/master/Dockerfil...

sandGorgon• 11 years ago

Does anyone know if the nginx+Netflix sendfile changes in Freebsd can be ported to Linux at some point?

Be your own video hoster with Docker and Nginx

Related Stories