florianleibert

hi! open sourced a serving config for Kimi K2.6 from 90 tok/s 508 tok/s on 8xMI300X. Same weights / 0 quality loss.

Scaling is linear @15.8 tok/s per slot latency is constant. REpo has command launcher, Dockerfile, benchmark tool. Known limitations: BF16 KV only (FP8 crashes due to an AITER 384-expert constraint)

This is pretty cool.

There is also a tutorial on the Spinnaker website. https://www.spinnaker.io/guides/tutorials/codelabs/dcos-sour...

Here is a tutorial on the easiest way to run Tensorflow: https://dcos.io/blog/2017/tutorial-deep-learning-with-tensor...

Here is another good tutorial - about Tensorflow on DC/OS: https://mesosphere.com/blog/2017/05/11/deep-learning-tensorf...

Super impressive performance!

Here is more information about the underlying technology, which also powers e.g. Twitter (Apache Mesos and DC/OS). https://azure.microsoft.com/en-us/blog/microsoft-joins-the-n...

Here is a reference to the original mesos (core of DC/OS) paper by Ben Hindman, Matei Zaharia and Andy Konwinsky: https://www.cs.berkeley.edu/~alig/papers/mesos.pdf

The 0.7X releases contained a number of bugs. 0.8X has been significantly better!

I'd love to hear about the bugs that you encountered. Did you file any github issues for them?

Great post!

Here is the Github link: https://github.com/mesos/myriad

Here is the link to the Github repo: https://github.com/mesosphere/mesos-dns

You can try it on either Digital Ocean or GCE.

Related articles:

GigaOM Jonathan Vanian https://gigaom.com/2014/12/07/mesospheres-new-data-center-mo...

TechCrunch Ron Miller http://techcrunch.com/2014/12/07/mesosphere-releases-first-d...

VentureBeat Jordan Novet http://venturebeat.com/2014/12/07/mesosphere-funding/

WSJ Deborgah Gage http://blogs.wsj.com/venturecapital/2014/12/08/mesosphere-ra...

The New Stack Alex Williams http://thenewstack.io/mesosphere-develops-a-data-center-oper...

You can try this out on https://google.mesosphere.io

YARN is still in its infancy and doesn't run in production for user-facing applications at any large site. Also, there are big limitations when it comes to scaling YARN. YARN also has a lot of overhead on each machine the worker processes run on.

Twitter runs mesos in a multi 10k node cluster for user facing, production applications.

Well that changes everything b/c Marathon is just a piece of the puzzle. This turns the statement into "init.d isn't an operating system".

Mesos itself doesn't do scheduling - it merely does allocation and allows applications built on top of it to schedule to workload.

Twitter recently stated they had roughly 1 SRE / 10K nodes.

Arguably, most systems are distributed systems today. Even if you're just connecting a Rails app with a MySQL DB.

If we had a fabric (which is called Mesos ;) which allowed you to write elastic, distributed systems without the need for worrying about interconnecting hosts and segmenting hosts into static partitions, etc. wouldn't that be a big win?

Spark is also a great example for an app that was built directly on top of Mesos - the authors could focus on implementing the actual logic rather than worrying about interconnection. Same is true by the way for systems like Chronos and Marathon - all of which run on top of Mesos.

In light of all the excitement around Docker & containers, it's important to note that Mesos has been using cgroups / LXC (for isolation) since its inception.

Mesos paper: http://people.csail.mit.edu/matei/papers/2011/nsdi_mesos.pdf

We're working on some video tutorials. How can we make the content easier to understand?

Mesosphere is based on Apache Mesos, Marathon and other technologies. Mesos is an abstraction of the underlying hardware - exposing APIs to allow running tasks in a cluster ~ think of just launching threads on your linux box - with Mesos it's similar, you can launch processes on your servers across your datacenter or cloud. It's production software that powers Twitter, Airbnb and other companies at scale. Mesos is more than just orchestration - it's really a fabric for writing distributed systems. Apache Spark was a sample app written for Mesos. Marathon is another application that runs on top of Mesos and orchestrates containers. It can handle tens of thousands of containers and more in a single cluster. Chronos and Storm are other systems that can run on top of Mesos.

Marathon: https://mesosphere.github.io/marathon/ Chronos: https://airbnb.github.io/chronos/

Should be public now!

Looking awesome!

Great job, ross, brenden, harry & matt!

If you have specific requests on how to make the docs for these frameworks better, it'd be great if you filed a ticket on their respective github page.

I'll outline the specific goals of both Marathon / Chronos to clear any confusion:

Marathon: Execute a long running job in the cluster and make sure it keeps on running. You can specify resource requirements as well as how many instances of this job you want to run. Examples of jobs: Rails App, Jetty Service, JBoss Service.

Chronos: You can specify a repeating & finite job based on a schedule or another job completing. You specify resource requirements. Examples of jobs: Mysqldump (e.g. daily dump prod DB), Hadoop Job (cascading, pig, cascalog, scalding...), bash script using ImageMagick to create thumbnails.

Both of these systems are Mesos frameworks - Mesos does the heavy lifting and offers resources to these frameworks which the frameworks can accept or reject.

You can find more info here:

http://mesos.apache.org/

Mesos cannot do "cron-like" things. Mesos is a resource manager for your cluster. You need a framework to actually schedule tasks. Marathon / Chronos are both frameworks.