HN user

ende42

38 karma

[ my public key: https://keybase.io/nikod; my proof: https://keybase.io/nikod/sigs/LxHBrxuIuJ57F5VJTemrX5jTZzmeNDdh-S0DJ29bHyM ]

Posts3
Comments37
View on HN

We recently implemented a simple streaming server based on Openresty and Redis. Redis is used as buffer. We load tested it up to 2000 mountpoints (128kbit mp3 streams) with 15.000 clients.

The architecture has some nice properties. For example the buffer being not in-process means (besides an obvious performance hit) that the streaming server can be seamlessly redeployed without loosing the buffer.

The Openresty Lua implementation is amazing.

I just moved from a late 2014 MBP running Arch to a Precision 5520 (more or less the business variant of the XPS15) running Alpine. Both feature an i7 HQ, 16GB RAM, 512GB SSD. Both where about 2200€.

The Precision is a good laptop but it's my first that isn't better in every way than the one I had before.

On Par:

* Fast (for a laptop) processor.

* Fast SSD. 900MB/sec on the MPB, 400MB/sec on the Precision. Anyway fast enough.

* 16GB RAM. Just enough for me.

Good:

* Linux support. Very important for me. Suspend to RAM on lidclose works reliably. WLAN works. (Display dimming just works (as with the MBP), no coil whining here).

* Display. Native HD (had to use Linux' crappy scaling on the MBP). Non-glare. (And the MBP had the display stain issue)

* 3 years on site guaranty, if on site isn't possible, I can keep the SSD.

* Can change RAM, SSH, battery without voiding guaranty.

Bad:

* Display. I'd prefer 16:10.

* Sound is slightly worse when laptop is used on a table. Sound is dismal when laptop is used... on the lap.

* Keyboard on the new Precision is worse than the one on the 3 years old MBP. Will use the UHK in the office anyway.

3. is the reason why NGinX is the recommended proxy in front of webapps with scarce parallelism (for example Ruby with Unicorn; see http://unicorn.bogomips.org/PHILOSOPHY.html for an explanation) when "slow clients" are to be expected. NGinX is protecting the webapp from blocked workers by slow clients and Outlook Webmail seems to behave just like one. I don't know by heart how to tune this behavior if one wants to avoid it but this property is the main reason we use NGinX.

Thanks for your work!

Actually we (laut.fm) are using the HTTP Push Module for our public API for a live stream of tracks which get played on our ~ 1500 icecast stations. We offer 3 formats: http://api.laut.fm/song_change.stream.json for a line separated JSON HTTP stream, ws://api.laut.fm/song_change.ws.json for the same as websocket endpoint and http://api.laut.fm/song_change.chunk.json for the last x songs. It's not really high volume, just 6 to 7 per second. But it runs basically unattended for years now and I'm pretty happy with it.

Is there any reason to update to the new one (other than new features; admittedly I haven't really looked into NCHAN)?

We're using a home baked solution using our own config server (https://github.com/niko/goconfd) which evaluates templates POSTed to it. Combined with some trivial shell loops over a blocking POST and a HAProxy restart this does the job sufficiently well. If anybody is interrested in the details I'm happy to elaborate.

Nice article. Thanks. Does proxying icecast streams with NGINX work when the protocol is not HTTP but ICY?

For scenarios where I felt etcd/gconf/consul is overkill I wrote goconfd (https://github.com/niko/goconfd). Features:

* reads config from a json file * GET requests with deep links into the json tree * POST request with template as body and deep links (using the go template engine, too) * long polling (response has to be triggered explicitly by a PUT request)

I also made a simple docker container running HAProxy - configured by goconfd - to orchestrate docker containers.

Introducing Carrot 12 years ago

Two years ago I implemented something different, but in green: https://github.com/niko/green

I had a juice version in the pipe but I didn't like that I had to use yellow for that. It's hard to use yellow once you get used to green.

But I'm really interested in orange now so I might give carrot a try.

(For us) Docker is not about security or scalability but about (good enough) isolation, separation of concerns and reproducability. Let me elaborate.

* Isolation: Docker enables us to pack any software and let it run with close to none side effects. Different versions of packages, libs and gems needed by apps don't interfere. It's like bundler but for all kind of dependencies not only gems.

* Separation of concerns: For our operations it doesn't matter what's inside a Docker container. We have mid sized Ruby web apps, small Go demons, NGINX with Lua support compiled in, legacy PHP apps neatly packed in Docker containers. They have a well defined interface:

The build script which consistently sums up dependencies and the build process. `docker run` wrappers which clearly state the interface of the running container like exposed ports and mounted volumes.

* Reproducability: We are able to run the same containers in development, staging and production. A dozen containers will easily run on a developers laptop.

As a side effect the Docker architecture makes us think harder about app architecture like which services are stateless and which are not and for what reason.

The fact that containers share a kernel and thus are not 100% isolated or reproducable as with virtualization hasn't been an issue for us (so far).

There are still issues and features we're missing. For example private Docker repos are PITA and building instead of pulling from a repo means you might get fooled by the build cache. And we'd love to have build in support (or at least a common standard or best practices) for orchestration. But all together for our needs it's already pretty useful.

You're right. It's more like a docker philosophy or best practice, not a restriction. And a best practice not everybody agrees on that is [1][2]. I'm using some docker containers based on Phusions base image myself. It works fine. Others of my containers are just running one process. I found it feels like a cleaner architecture when I manage to bundle concerns in a way so the resulting containers are stateless. But when I don't achieve that I don't pull my hair out over it. Whatever floats your boat.

[1] http://phusion.github.io/baseimage-docker/ [2] https://news.ycombinator.com/item?id=7258009

I tried it. Compiled fine. The example conf file has some chinese comments in it. Most of the basic configuration is pretty obvious.

I started the server and it serves an index.html with an upload form which is nice. The upload response is an HTML page, too which states in the body where the uploaded file is accessible.

There I got curious… how would one use zimg uploads from app? I think the common use case would be: User upload images to a web app or an API app which then POSTs the image to zimg and puts the image location into a DB.

I tried POSTing images using curl having the vague hope the response would be something more machine readable than an HTML body. At first that seemed to be true because I used the wrong curl parameters and zimg couldn't find the image within the POST body and responded {"ret":false,"error":{"code":1,"message":"File type not support."}}. But once I got the curl parameters right (curl -XPOST --form userfile=@9393.jpg -i http://localhost:4869/upload.json) zimg again responded with HTML. Adding an accept header and/or a ".json" postfix to the URL didn't make a difference. Am I missing something?

Is there a possibility to restrict upload to local network only but allowing public download?

Good read, thanks. Is it possible to run an own, modified version of an open source app on sandstorm.io? As a non-techie user? Like - say - a small modification of Wordpress?

Thank you for your honest response. Mistakes happen all the time.

The thing that concerned us most is that as a user of Facebook we had the impression that all efforts of reporting the issue where unheard. There was literally no feedback what so ever. I totally get that Facebook can't respond individually to error reports, it's just got too many users for that. None the less we felt really helpless in that situation.

I would never have imagined anyone of Facebook would ever answer this HN post. So again: Kudos for this statement and thanks for resolving this issue.

Niko.