Triton: Docker and the “best of all worlds”

https://www.joyent.com/blog/triton-docker-and-the-best-of-all-worlds

by bithavoc • 11 years ago

202 100 11 years ago

kordless3• 11 years ago

Here's the signup link for Joyent's hosted container service: https://www.joyent.com/lp/preview.

We're seeing the initial explosion of the microservices ecosystem right now. I've been spending most of my time over the last few years thinking about trusted decentralized infrastructure and have decided that microservices, including hosted ones, could be one possible solution for instantiating trust as a proper 'knob' of the cloud. This Intracloud, if I'm allowed to use a new buzz term, will be smeared across 100s (1000s?) of datacenters world-wide. Have a high-trust use-case? Run it in a German or Dutch datacenter. Have a high-efficiency use case? Run it on a friend's cluster for free.

I believe in this idea of the 'trust knob' enough that I sought out a job with the folks at https://giantswarm.io/, where I am now a dev evangelist. We are a German hosted microservices stack which provides a Docker platform. Alpha signup is here: https://giantswarm.io/request-invite/. Demo of it in action here: https://github.com/kordless/swarm-ngrok#ngrokn-giant-swarm

I also hacked together a SF Microservices meetup last Friday: http://www.meetup.com/SF-Microservices/. 139 people have already joined. Would like some feedback for content! Planning on mid-April for the first event.

I'm excited to see what happens with this market in the next 6 months!

SandB0x• 11 years ago

Can someone explain this in words that a time traveller from 2010 would understand?

simtel20• 11 years ago

Well, since 2010 Sun has died, and opensolaris is no more. It lives on as an open source effort called Illumos. At one point, solaris could run Linux binaries unmodified under a solaris kernel and userspace. That became un-maintained, but it is now working again.

One of the reasons this feature fell out of use was because it could only run 32-bit linux binaries. Now it can run 64-bit binaries too.

If you want to try it out there is a company you may not be familiar with, dear time-traveler from 2010, called Joyent, that will let you use this.

What is very interesting dear time traveler is that there is a format for containing programs and their environments called docker that allows you to use features, as you could in a zone, but which is easier to use than zones. You can deploy your containers built on linux to solaris zones with this feature, and Joyent lets you do it.

pyre2• 11 years ago

Oracle bought Sun in 2009 and finalized in Jan. 2010, so this time traveller would have to be from a very specific slice of 2010 to not know about the demise of Sun. :P

kordless3• 11 years ago

Remember when virtual machines came out? That technology gave us tools to create a 'layered' effect for compute related resources on a single, managed machine. Microservice technologies, like Docker containers, are a logical extension of that layering into the process space on the machine, and in many cases multiple virtual machines running on a single machine.

It's all about the layers, man.

steeef• 11 years ago

Joyent actually extends this and allows you to treat an entire datacenter as your docker host, similar to how you can use a product like VMware to cluster ESX hosts together.

mrfusion• 11 years ago

Love the question. This should be the new ELI5. I think a lot of articles on here would benefit from answering that question.

evanspa• 11 years ago

Based on your username, I'm not surprised you like the time-traveling angle.

tomcam• 11 years ago

Ditto. It was a boon to me too.

raus22• 11 years ago

The Explain it like i have been in a coma for 5 years... Elihbiacf5y or just 5 years in coma, explain! 5YICE....

Or ETTT5YA or ET5... explain to time traveler from 5 years ago

pgeorgi• 11 years ago

I assume that the time traveller knows Open Solaris up to 2010.

Someone used Linux facilities that are somewhat similar to Solaris Zones and ZFS snapshots (you'll know those, since Solaris 10 from 2005 shipped with that and wasn't even the first) to build an ecosystem for lightweight virtual systems with standardized interfaces. That system is called Docker (and runs on Linux, obviously)

OpenSolaris is all but dead, but a community succeeded it, called IllumOS. People from that community decided to attempt to use Solaris Zones, with Linux branded zones (that capability was dropped in the meantime so they had to invest some effort to bring it up again) to reimplement Docker on an IllumOS (ie. OpenSolaris) foundation.

jimpick2• 11 years ago

Nice!

thinkingkong4• 11 years ago

Tldr.

Triton lets you run your docker containers in the cloud without worrying about virtual machines or setting up your own paas.

Signup here: https://www.joyent.com/lp/preview

hueving• 11 years ago

So how is this different from the container service from google?

thinkingkong4• 11 years ago

Frankly, the technical implementation.

brianshaler• 11 years ago

One interesting bit from the article: "Does this mean that I'll be able to DTrace my Linux apps in a Docker container?" (yes)

Technically you can run dtrace on various platforms, and I don't know enough to tell you specific reasons why you would rather use dtrace on smartos than linux. I got the impression a few years ago the ports weren't as good. (perhaps relating to the linux port not yet being a kernel module?)

mcescalante• 11 years ago

Thanks, I spent a minute skimming through this post and ended up coming here hoping someone wrote a tl;dr like this. Must be challenging to write effective copy for something like this (not being sarcastic)

m_mueller• 11 years ago

What does this provide over running a Docker host running on a dedicated server?

lojack• 11 years ago

I haven't heard of this before today, but it seems as though with this you could scale out to multiple servers and it'll continue to be treated as a single docker host. It also isn't linux.

phildougherty2• 11 years ago

"You can run your Docker containers across entire data centers without ever creating a "cluster" as other IaaS providers would have you do."

This is a little unclear. I get that you don't have to create a "cluster" because you're going to use the API to launch containers on existing dedicated hardware, but how is that a WIN for your finance team? I guess I'm missing something.. but wouldn't you have to already have systems online that you're paying for that these containers can be launched on?

From: https://www.joyent.com/blog/docker-bake-off-aws-vs-joyent

thinkingkong4• 11 years ago

The difference for the operator is: Baremetal -> smartos -> container vs Baremetal -> hypervisor -> virtual machine -> container.

Theres root safety in the smartos implementation of docker. So they can do multi-tenancy.

For the customer, you dont have to provision entire virtual machines to run docker containers.

phildougherty2• 11 years ago

Is it multi-tenancy of various Joyent customers on the same hardware? So you get containers placed somewhere in their data center on shared hardware with other customers? Or do you already need to have dedicated hardware provisioned at Joyent to launch containers on? I'm still unclear on that point, which makes me question how it's saving the end user money.

redwood631• 11 years ago

This is running Linux binaries on Oracle Solaris (which they call SmartOS), and not the real thing that Oracle is still developing, but a fork from 2010 that is maintained by 1/10th of the developers it once had. https://www.openhub.net/p/illumos.

This would have been pretty rad 10 years ago, when the world still cared about Solaris.

On a minor note: the post doesn't credit Oracle or Solaris, from where more than 90% of their SmartOS code comes from, until Oracle closed their code in 2010.

thinkingkong4• 11 years ago

Thats because Oracle deserves no credit. The illumos project was created as a derivative of opensolaris which was created by Sun. The lineage is actually sort of mind boggling. If we want to be pedantic, we could complain about nobody is giving credit to BSD.

jimpick2• 11 years ago

And Bell Labs.

pyre2• 11 years ago

until Oracle closed their code in 2010

At that point, why credit Oracle instead of Sun? Oracle finalized their deal to buy Sun in January 2010. How much did Oracle contribute between then and the announcement that they were closing their code?

Edit: Oracle announced closing OpenSolaris on August 13, 2010. The last "release" of OpenSolaris was June 2009.

bahamat4• 11 years ago

Oracle never publicly announced it. The last commit that I see copyrighted by Oracle is https://github.com/illumos/illumos-gate/commit/ea10ff14a02f7... on August 18, 2010. After that, updates from Oracle silently stop.

But your point stands well. Oracle only had a paltry 7-8 months of contributions.

In my opinion, and I really don't know who would share this view, since Sun opened Solaris, illumos is now what Solaris was intended to be; it's not the illumos community that forked Solaris, it was Oracle.

CamTin2• 11 years ago

This is cool RIGHT NOW because WHO THE HELL CARES WHAT THE UNDERLYING OS IS? For a user trying to bring something to market, here's what matters:

1) it's cheaper (theoretically, due to performance savings from not dealing with hardware virtualization), and 2) it looks like a docker host

I honestly expected Docker Hub to expand into this kind of operation before anyone else, but Joyent, from out of nowhere, has done it instead. Why should anybody care that they're doing it with a Solaris-based kernel, Linux, on NetBSD, as MAME machines, VIC-20 cartridges, or with 1402 machine images and druidic spells?

To the user, it's just a magical world where your Docker host always expands to whatever size you need it to (and you of course get a bill for the usage).

The real test will be whether or not this is cost-competitive to deploying Docker swarms on other public cloud providers. If the performance and tenancy claims are as they claim (or even the same order of magnitude, which seems reasonable), and they're not insanely greedy with pricing, this could be the biggest tech since Linux itself.

jgrowl3• 11 years ago

I was wondering how this would compete with docker swarm. Is my understanding right that docker swarm basically lets you run an agent on individual docker hosts whereas triton will let you run a single virtual docker host?

geofft4• 11 years ago

The author worked for the Solaris team in Sun back when it was actually Sun and back when Solaris was actually being developed. Among other things, he was behind the invention of DTrace. He left the company shortly after the Oracle acquisition, after working for Sun for 14 years.

I sorta feel like that gives him the right to credit or not credit Sun (and especially Oracle) as he sees fit.

binarycrusader2• 11 years ago

Solaris is actually still being developed; I have no idea why you think otherwise. If anything, Oracle's been on a hiring spree since the acquisition and has grown the systems business considerably compared to Sun.

If you think Solaris isn't still in development, just wait for the Solaris 12 announcement details.

josh26002• 11 years ago

Joyent has solved some really hard problems to make this work. I was lucky enough to get the chance to sit down with Bryan last week and talk with him about what this took. It's essentially reimplementing Linux on top of Illumnos, which is no tiny feat.

There are still messy hurdles to running Docker in production, but it's clear that Joyent has really tried to make something awesome here.

As someone who works deeply in the future-container space, I applaud folks who are taking us deeper down the rabbit hole. I think it's clear that this whole 'back-to-the-future' isolation technology is really cool stuff. Jails/zones/containers have been around forever and I think it's great that we're finally taking advantage of this technology.

Edit: at Terminal.com, we want people to push Linux forward, and this is a great example of taking Linux to new and intriguing heights. I did not think we would have Linux on Illumnos in quite this way in 2015 and it's delightful to see. We are all standing on the shoulders of giants and it's great to reach new vistas.

kordless3• 11 years ago

+1 on terminal.com. Love it.

justincormack4• 11 years ago

Isn't it an example of taking Solaris to new and intriguing heights? There is no actual Linux code in the implementation.

josh26002• 11 years ago

That's a great point, but I think the point of what Bryan has been doing is to make Linux work with Zones (and dtrace).

That's a primitive that Joyent has wanted to upstream into the Linux kernel for a long time and has never been able to get the necessary consensus around it (similar to OpenVZ's troubles getting their work upstreamed).

In short, this is sort of a hack to give you zones on Linux without needing to get zones into the upstream. Yes, there's no linux code, but there is a lot of required understanding of Linux code to make something like this work.

It's kinda amazing that they got 64-bit linux to run on top of Illumnos, right? I did not see that coming and maybe that's because I'm ignorant in some capacity, but it's been a pleasant surprise.

mrinterweb2• 11 years ago

Is there any pricing information for Triton yet?

jholmes83• 11 years ago

Pricing here: https://www.joyent.com/blog/expanded-container-service-previ...

mrinterweb2• 11 years ago

Thanks for finding that link. That is a very attractive pricing model. I am currently running about 10 VPS running about 25 containers. The costs of the VPS hosting is affordable, but managing those VPS hosts, deployment, balancing, discovery, and scaling are the hard parts. I'm looking forward to learning more about Triton.

themgt3• 11 years ago

Didn't see the actual repo linked: https://github.com/joyent/sdc-docker

"A Docker Engine for SmartDataCenter, where the whole DC is exposed as a single docker host. The Docker remote API is served from a 'docker' core SDC zone (built from this repo). ... Disclaimer: This is still very much alpha. Use at your own risk!"

trentmick• 11 years ago

Thanks for the link. I've updated the "current state" notes to better reflect reality. https://github.com/joyent/sdc-docker#current-state

ochoseis• 11 years ago

To me it seems like Triton is about shoehorning Linux into Solaris/SmartOS containers. While this seems great for accelerating adoption, why not just use SmartOS-specific Dockerfiles/repos in conjunction with sdc-docker (Not directly addressing the parent, more of a general question)? Imagine all of the crazy bugs and edge cases that will arise when trying to run Linux binaries on UNIX.

I'm surely making some assumptions, but the primary languages/developers that something like this targets are probably like Java, Javascript/Node.js, Ruby, Python, C...which are probably all portable enough to run on SmartOS without much/any modification.

wmf• 11 years ago

Docker multiarch support isn't finished yet, and getting all the world's Dockerfiles rebuilt on SmartOS is a big task.

There are also different philosophies about whether container images should be distributed as source (Dockerfiles) or as binaries. It looks like Joyent decided to run Linux/x86-64 binaries to be on the safe side.

crashoverdrive• 11 years ago

I feel like perhaps I'm the only one concerned about this: (Somehow, “SmartOS + LX-branded zones + SmartDataCenter + sdc-portolan + sdc-docker” was a bit of a mouthful.)

Not only is it a mouthful, it means there is a much wider space for things to go wrong.

tracker1• 11 years ago

Compared to say "hypervisor + Linux + kubernetes + weave + etcd + lxc-docker" ...? Not to mention all the pieces that work together for lxc-docker.

23david2• 11 years ago

Excited to see this. Congrats to the team on the release!

bau5• 11 years ago

I saw the name and my desire to read it plummeted. The same person wrote http://www.joyent.com/blog/the-power-of-a-pronoun which led to one of the most talented developers of node core and libuv leaving the project for a while.

rubiquity• 11 years ago

What relevance does that have to this? It's sad that your post is even being upvoted.

stuaxo• 11 years ago

Wow, bloody hell - hadn't read that before.

cookiecaper• 11 years ago

Me either. Wondering how I missed all that drama but not particularly sad I did.

codygman• 11 years ago

The power of a pronoun is a great article, if anything him writing it should be a plus.

The only way it is a negative is if you feel gendered pronouns don't further sexism, but I don't see how you could support that argument.

crazy1van• 11 years ago

Does Joyent refuse to hire speakers of Spanish, French, Italian, Hindi, or any of the other dozens of languages that have gender throughout the entire grammar?

nosequel2• 11 years ago

That "most talented developers of node core and libuv" took time out of his day to revert a commit to take a gender neutral pronoun and make it back into a male one. Talented people can be shitbirds also you know.

geofft4• 11 years ago

Just because someone is "one of the most talented developers" doesn't mean that it's good for them to be part of the community.

One of the best things that ever happened to glibc was Ulrich Drepper, far and away glibc's most talented developer, leaving. glibc is a better project for having multiple good people run it instead of one, brilliant, impossible-to-work-with developer in charge.

pencilcode• 11 years ago

The fact that he wrote it actually wins him points in my book but it's not relevant to this discussion. Kudos to the Joyent team, i've been curious about dtrace for ages but never looked into it thinking i couldn't use it anyway (only use debian based distros at work) - maybe now is the time :)

DannoHung3• 11 years ago

Uh, so I've never really heard of SmartOS or Joyent's SDC before now...

This stuff looks pretty fucking unreal. What's the catch? It looks like it solves so many problems with clustered container deployment. Exposing a whole datacenter as a single Docker host seems like the end-game to me.

Am I missing something? I'm a little low on sleep today.

edit: To answer my own question a little: I guess the docker-sdc isn't quite fully baked yet. That's not really a huge issue since it's still a preview, I think?

bananaoomarang• 11 years ago

Nope, SmartOS is pretty neat.

ssmoot3• 11 years ago

Ditto. We were early adopters at work. KVM + ZFS and practically no setup is ace.

jgrowl3• 11 years ago

I'm right there with you. It feels like the beginning of skynet

pannon• 11 years ago

No catch, it is a truly innovative/disruptive technology.

_delirium• 11 years ago

The main catch (which may also be a plus) is that the Joyent set of technologies is quite opinionated about how to manage a cloud, so you more or less have to "buy in" and do things its way. Your node OS will be SmartOS, nodes will be PXE-booted, your filesystem will be ZFS, the network topology will work in a specific way, etc. More practically, it also takes some time to be proficient in operating it (or even just get it working), because there are quite a few moving parts. Getting a SmartDataCenter deployment running, and then things like Manta and now sdc-docker running on top of it, is more than an afternoon's work. But if you're willing to buy in to the system and its choices fit your needs, it's really well engineered, imo much more "done right" than e.g. OpenStack is.

kstenerud• 11 years ago

Is there any further work going on to support AMD in the KVM driver? I remember the codebase having sat idle for a few years last time I checked...

snw• 11 years ago

While sadly not merged upstream yet, it works very well. I've been running custom SmartOS "eait" builds from http://imgapi.uqcloud.net/builds (which include AMD support) for some time now without any issues. The source can be found at https://github.com/arekinath/smartos-live

themgt3• 11 years ago

I find these to be pretty weasel words: Is Triton open source?

Yes. Triton Elastic Container Infrastructure is the commercial offering built on the top of the open-source SmartDataCenter cloud infrastructure management platform. In addition to the open source components, Triton includes support, a DevOps portal, and intellectual property indemnification only available to commercial customers.

In more plain English, "Yes. Triton is open source just like Mac OS X"

https://www.joyent.com/developers/triton-faq#os

lloydde6• 11 years ago

That's a good thing to clarify. It's not an open core system like Mac OS X. SmartDataCenter is open source and Triton is our product branding. https://github.com/joyent/sdc

There is work in progress to clean up and open source the self-serve portal and its dependencies.

The propriety bits are not ours: "The Joyent SDC USB images offered to our customers contain certain firmware binaries for use with Joyent-branded hardware. We cannot make that software generally available in the open source builds; however, it is available to all who have purchased Joyent-branded hardware and is of no use without that hardware." https://docs.joyent.com/sdc7/obtaining-software

themgt3• 11 years ago

But what you're branding Triton in fact appears to be a PaaS GUI on top of SDC? i.e. the way people manage their containers/hw in Triton the service, is not available in SDC which seems to be all CLI tools?

DannoHung3• 11 years ago

What does the bit about security patches and minor and major updates mean? The Product version includes automatic updates and the open source version you manage yourself? Or something else?

yonran• 11 years ago

Very cool. Is the next step integration with Manta so you can specify an environment of Linux executables to map over your data?

dap• 11 years ago

Yes, that's definitely the direction that we're headed.

Manta allows users to spin up large numbers of transient containers around their data (i.e., without moving data around) in order to do map-reduce operations, and without having to think about the container management. Today, these compute tasks are specified as scripts that run inside a well-stocked SmartOS container, plus some optional assets to download (for custom binaries or other data). The obvious next step would be to let people specify they want to run an LX zone (rather than a SmartOS zone) so that they can run GNU/Linux binaries. Then we can consider whether it makes sense to phrase the task as a Docker image. That might well be a more natural way to incorporate a larger bunch of binaries and other shared data. Architecturally, all of this should be relatively straightforward (famous last words!).

batbomb• 11 years ago

    > When Docker first rocketed

I thought this was an interesting way to start off the post.

Bedon292• 11 years ago

This looks like a very nice way to deploy containers. I see the per hour prices (https://www.joyent.com/blog/expanded-container-service-previ...) but is there any pricing on bandwidth? I imagine there will be eventually, right?

adamtaro• 11 years ago

I'd suggest looking at this page as an indicator (https://www.joyent.com/public-cloud/pricing -- Bandwidth tab): all inbound traffic free, outbound is 12 cents/GB after the first GB, cheaper at higher tiers.

Triton: Docker and the “best of all worlds”

Related Stories