steveb

There are a lot of developments around using Kubernetes as an IaC platform for the reasons in your comment. The combination of a standard API model in CRDs + the controller model maps nicely to managing infrastructure and exposing resources to developers.

<https://crossplane.io> just graduated to CNCF Incubation and each of the cloud providers are working on K8s controllers and code generators (like Amazon Controllers for Kubernetes, Google Config Connector, and the Azure service operator).

Before you leave, I'd love to talk to you. I'm in STL and looking for engineers to work in a professional, supportive, and inclusive environment. steve@aster.is

Yes, we have designed it to deploy things like Kubernetes and Mesos clusters via integrations with Terraform and Packer.

We've been working on a directed graph execution engine called Converge https://github.com/asteris-llc/converge.

In this case the task resource http://converge.aster.is/0.5.0/resources/task/ might help, as it allows you to create a directed graph using any kind of interpreter (for example, Python or Ruby) instead of having to use the DSL.

We have started work on exposing Hashicorp Vault secrets via FUSE and Docker volumes. The expectation is your containers will just mount secrets via a mount like /secret in the container.

The project is brand new and we'd love to hear your feedback: https://github.com/asteris-llc/vaultfs

Thanks for sharing this!

With the mantl project https://github.com/CiscoCloud/microservices-infrastructure we feed mesos task information into consul.

We've looked a lot a load balancing and feel that rewriting haproxy files dynamically can lead to brittle behavior.

Our current setup is we use traefik https://github.com/emilevauge/traefik to proxy marathon tasks.

Haproxy 1.6 includes some dynamic dns lookups http://blog.haproxy.com/2015/10/14/whats-new-in-haproxy-1-6/

tl;dr: mesos/marathon run the tasks, consul exposes the tasks in DNS.

Both Mesos and Marathon store different views of task state on the cluster.

How do we use this data to make it easy for jobs to find one another? Can we use this information to automatically configure things like Load balancers?

This is where tools like consul and mesos-dns come in. They populate a DNS store, so that task-name becomes something like task-name.example.com. If you are running 10 copies of a service across different hosts, DNS will have 10 entries.

If a container moves to another system, DNS is updated on the fly.

We can embed health checks with consul, so that if a service is unhealthy, it gets pulled out of DNS.

Consul also is nice because the edges perform the checks (instead of a central server), so the load is distributed.

Thanks for bringing this up. We opened an issue to investigate: https://github.com/CiscoCloud/microservices-infrastructure/i...

Why you should care about this project: You wouldn't spend months building your own Linux distribution, so why spend time reinventing the wheel with chef/puppet/ansible?

We're not building a PaaS. Like a Linux distribution, we're integrating open source building blocks (logging, service discovery, scheduling) so that you can focus on app development and analytics instead of sysadmin tasks.

I'll be giving a talk to the NYC Mesos user group this Wednesday June 17th: http://www.meetup.com/Apache-Mesos-NYC-Meetup/events/2229328...

KONG looks amazing, I haven't heard of it before.

Here's how you would deploy it in microservices-infrastructure:

1. You'd deploy the cassandra mesos framework, giving you HA cassandra. Instead of setting IPs, you'd connect to this as cassandra.service.consul

2. You'd launch a Kong container in marathon. It would show up in dns as kong.service.consul for other apps to find, so you don't have to hard code IPs in your config.

Edit: We've opened an issue to make this an example app https://github.com/CiscoCloud/microservices-infrastructure/i...

We like Kubernetes (and are looking to add it to our project), but our goal is to integrate building blocks that allows us to run many different types of workloads. Think of our project more like an Ubuntu for distributed systems.

Kubernetes may eventually spread out beyond Docker, but for today we need to support things like Kafka and Spark.

As others have noted, we've had things like CloudFoundry, OpenShift and Heroku, and these all-in-one frameworks tend not to extend outside their original domain.

The real problem is going from tutorial to something you would use in production. Throw in logging, security and service discovery and you can have a few engineers hacking away for months.

So I want to plug a project I've been contributing to: https://github.com/CiscoCloud/microservices-infrastructure

We're trying to make it super easy to deploy these tools. For example every time you launch a docker container, it will register with consul and be added to haproxy. The nice thing about using Mesos is we can support data like workloads Cassandra, HDFS, and Kafka on the same cluster your run Docker images on.

We use terraform to deploy to multiple clouds so you don't get locked in to something like cloudformation.

Our goals for this tool is to make it dead simple to deploy and run. The binary and a json config file is all you need to check a host. When you combine it with consul, we can integrate service discovery with host monitoring and push checks out to the edge nodes instead of relying on something like Nagios.

We're using this tool to monitor cluster configurations and test deployments of https://github.com/CiscoCloud/microservices-infrastructure.

I'm pretty excited about this release. We just added deploying Vault on top of consul in the microservice/realtime data platform we're building:

https://github.com/CiscoCloud/microservices-infrastructure

We were already using consul as our service discovery engine and as a data store for dynamic configurations via consul template. We feel Vault will help us manage secrets on our infrastructure and inside containers.

As someone who is contributing to this project, what we are focusing is making it easy to run and connect multiple containers and applications. We do this by:

1. Building in service discovery. This means that containers register automatically with DNS (and mesos tasks will register with consul in our next version)

2. Distributed scheduling. We use mesos to manage the running of containers and other processes. This means you can just use a an API call to launch a container, and that container will register itself in DNS.

3. Framework support. One of the coolest features of Mesos is the ability to launch containers that easily run things like Kafka, HDFS and Cassandra clusters without having to use tools like chef or ansible.

Please let me know if you have any questions. I'm really excited about this project!

I gave a 30-minute overview of Spark yesterday at StampedeCon. Spark is generating a lot of excitement in the big data community:

https://speakerdeck.com/stevendborrelli/introduction-to-apac...

There are a few efforts to support docker (and other) containers in Mesos under (https://issues.apache.org/jira/browse/MESOS-816)

As part of this, a couple of new docker efforts were released today:

https://github.com/tarnfeld/mesos-docker-containerizer

https://github.com/mesosphere/medea

I'm pretty excited to see fleet. These types of distributed systems are a natural complement for containerized infrastructure.

I've been using gitblit for a few months, and have been very pleased with how easy it is to set up and manage. It's great to set up http sharing of git repositories inside an organization.

It has nice features like ACL's and federation for backup.

Other links:

Video: http://vimeo.com/42675696 Font Download: http://dl.dropbox.com/u/20517415/ZXX.zip

Each card (not core) is running an embedded version of Linux. Binaries are almost all linked to busybox. You can mount NFS shares to share data.

The embedded systems need specially compiled binaries. With the Intel compiler, this is usually achieved by turning on a flag at compile time. So yes, you can install and run your own software on the nodes.

St. Louis, Missouri - 2 Software Engineers - Fulltime

Major global financial corporation.

We are seeking 2 software engineers with strong C++/Java/Python skills to develop grid software and implement visualization of financial data.

The roles are as follows: 1) Help develop a multithreaded C++/MPI application to simulate the behaviors of mortgage portfolios. The application runs a cluster of Linux nodes. We're looking to scale to thousands of cores.

2) Develop visualization tools using Paraview or other technologies for financial data. Mine data sets and work with analysts. We are open to big data technologies and techniques.

Experience with quantitative finance, HPC or scientific computing is a plus. Our target platforms are both Linux and Windows.

email me at steve@borrelli.org if interested.

There are several billion people using many billions of devices every day.

From the code in your microwave to massive computing clusters, virtually all of our software can trace its ancestry back to this man's intellectual output.

I'm eternally grateful for his life and contributions to humanity.

NeXT sold maybe 50,000 systems in 5 years, less than the number of PCs that a typical large company has stuffed in cubicles.

And from that small install base, emerged the first Web browser, Doom, and Mathematica.

They were amazing, if flawed products. Every system came with Mathematica, a full dictionary and the complete works of Shakespeare.

I remember having to go use Windows NT after NeXT failed in the market. And I remember, ten years later, holding the first iPhone in my hand and knowing where it came from.

Fiscal years don't always match calendar years. Microsoft's Q4 ends on June 30th.

http://www.microsoft.com/investor/EarningsAndFinancials/Earn...

I don't think you can say that tablets are bought in addition to PCs, as we've seen PC sales flatline and contract in the U.S. and Europe since the iPad was released. Nobody really knows what the long-term impact will be.

If we estimate around 80MM PCs sold in a quarter, MS generates about $60 Windows revenue for every PC sold, but their costs are about $22 (from Q411 results Rev $4.7B, Profit $2.9B). It will be hard for them to the cut license fee unless they can significantly increase unit sales.

This is probably why they didn't push WP7 on tablets.

For almost all the OEMs, the average margin on a PC is single digits.

Tag search results:

Metro 81

HTML5 25

c# 9

c++ 12

javascript 15

Silverlight 0

WPF 0

I find it hard to reconcile the term free market with the monopoly status granted in a patent, but this is the system we have and MS would be wrong to not respond in this way. The goal here is to make Android uneconomic for handset makers, not to generate revenue.

Google can dump their OS on the market because their business model is different than Microsoft's. They make 98% of their revenue on advertising, so it is in their interest to commoditize the OS level and increase the adoption of internet-connected devices. MS makes virtually all their profits on the licensing of software, they need to protect that business and their core platforms.

Jobs failed multiple times early in his career!

1980: Apple III released, bombs and is killed in 1984

1982: Jobs is kicked out of the Lisa Project, joins the Macintosh project

1984: Mac released. It is overpriced and has tepid sales.

1985: Kicked out of Apple

1989: First NeXT cubes released. They are overpriced and see slow sales

1991: NeXTStation released. NeXT still has tepid sales.

1993: NeXT hardware cancelled, OS ported to multiple other platforms with little success. Only the Intel port survives. Releases OpenStep for Windows.

1996: NeXT is bought by Apple.

For over a decade he had a string of public failures to match every success.

Nowadays Apple hits home run after home run and disrupts multiple industries. Imagine how much he learned from each failure.

To me slashdot was critical in galvanizing the geek community and bringing free software to the mainstream in the late 90's. The comments added incredible value and created a community that I had not come across anywhere else.

I think the high point of that period was the announcement of the open-sourcing of the Netscape code base. Nowadays, itt is hard to imagine the need for all the stories on how to convince your boss to use this software some dude in Finland wrote.

Slashdot also championed everything2.com, kind of a proto-wiki.

The low point was all the trolling in the article about death of W. Richard Stevens, which lead to much of the moderation code that needed to be put in place.

Rob's run at slashdot was pioneering and hugely influential. I look forward to his next project.

This is the OS that is supposed to carry MS until at least 2015. If they are going to unveil the cool stuff during BUILD, why just not wait a couple of weeks?

- The examples seem dated, like people copying pictures over USB drives in the era of facebook and dropbox. He runs the example on a netbook, which I guess means after his sister leaves he needs to get another USB drive and copy those files again to his PC.

- Why did he copy files, edit them, copy them back and then need to figure out what version he wanted to use? Now he's got two copies of the same filename in different locations and they may get out of sync.

- Why in the case of conflict, does a number have to be added to the file if you want to keep both versions? Why not have some type of versioning and conflict resolution built into the filesystem? Look at a user's computer, directories are littered with filenames like "copy of Myfile (2) 2011_may.doc".

- Where is the touch interface? Those buttons are small and all the dialogs assume a mouse.

I do like the blog and I like that they introduce different members of the Windows8 team, but they need to focus on more compelling features.