Ramiro

Very cool! This is a great example of how ephemeral environments can help for a lot more than just fast inner loops or manual verification.

XMPP was supposed to be this. There was a boom in the early to mid-2010s (I worked at HipChat back then, and it was built on top of XMPP). IMO, it works well; it scales, but none of the commercial solutions picked up for whatever reason. In the end, as you say, proprietary protocols like WhatsApp or Slack won the market.

I've heard great things about Matrix. Peeble's founder, Eric, was building Beeper, a chat app with a similar purpose to what you describe (it recently got acquired by Wordpress). I believe that the core was built on top of Matrix, so you might be up to something there.

I'd highly discourage you from using SMS; it's very insecure. I'd go as far as to recommend you not to implement your own auth and instead use something like Auth0, WorkOS, SuperToken, or SSOReady (https://github.com/ssoready/ssoready), among others.

Building auth stacks is not trivial and is not what will make your SaaS successful. The more you can leverage experts to focus on what makes your SaaS special, the better.

I second this, the SF Public Libraries are a great place to work. I used to work out of the Potrero Hill branch a lot.

The main library (the one by the Civic Center) closes way later than the rest. Good spot if you're working alone.

I love reading these "reports from the field"; I always pick up a thing or two. Thanks for sharing @ianvonseggern!

Every time I read about Nomad, I wonder the same. I swear I'm not trolling here, I honestly don't get how running Nomad is simpler than Kubernetes. Especially considering that there are substantially more resources and help on Kubernetes than Nomad.

I agree with @metaltyphoon on this. Even for small teams, a managed version of Kubernetes takes away most of the pain. I've used both ECS+Fargate and Kubernetes, but these days, I prefer Kubernetes mainly because the ecosystem is way bigger, both vendor and open source. Most of the problems we run into are always one search or open source project away.

I really like the "remote"/ "on-site" / "hybrid" filter. Good job!

The most significant value I get from Costco is that the quality is consistently high and the options are well curated. I like not having to go through 100s of brands of basic staples like yogurt, toilet paper, milk, etc...

I agree is early, but the same thought around WASM crossed my mind. Interesting approach though

@blakesterz what kind of pain are you feeling when developing with containers on a Mac vs using your Linux box? I know perf is worse, but are you hitting other things?

This is very cool. Running tests on browsers has always being a pain. Is this something that could be run from within a container?

I think this something that people miss a lot when trying to optimize their Docker builds. Is the whole optimizing for most of your builds vs optimizing for a specific build. Not easy.

I never really thought about this, it's a good point. What do you suggest it's used instead of ["npm", "run", "start"]?

Distroless are tiny, but sometimes the fact that don't have anything on them other than the application binary makes them harder to interact with, specially when troubleshooting or profiling. We recently moved a lot of our stuff back to vanilla debian for this reason. We figured that the extra 100MB wouldn't make that big of a difference when pulling for our Kubernetes clusters. YMMV.

Fig's integration for Makefiles makes my life so much easier!

I've played with shell autocompletion before, but this way easier to use. Good luck!

Not yet. We are trying different approaches to curb this since we do want to keep our free tier.

We have the same problem in Okteto. We've been investing a lot on building tech to prevent this (I gave a talk on this during the last eBPF community days -> https://www.youtube.com/watch?v=tplv3Hjjv2Q), but it's tough. We spend a LOT of resources fighting it.

There's a couple of approaches we've tried:

1. Register each dev env with the provider. Dev Envs can have predictable URLs (e.g in okteto cloud it's the name + namespace + domain), so you can directly register it. This works well with self-service webhooks like Github.

2. Have a proxy that routes to the right dev env. This works if there's a key you can use for the routing, like a user ID or the subscription.

3. Have a proxy that duplicates traffic and send it to all the dev envs.

I'm curious wow do you solve this on the DTAP env. Are you registering a single endpoint with the webhook? Then how does it reach the separate dev envs? (or do you have a single, shared dev env?)

For debuggers, we wrote a few guides like this one: https://okteto.com/blog/how-to-develop-node-apps-in-kubernet....

We need to maintain separate profiles for running apps in debug mode, to expose the java debug port to the host and supply the right JVM options

Yes, you should. This is something you can do in the 'okteto.yml' file. You can specify different environment variables, different commands and even a different image (e.g. an image with your dev tools and debugger, which you don't need in prod).

Latency

This can be a challenge, for sure, specially if you have a bad network connection. In our experience, debuggers and incremental file sync don't require too much bandwidth.

Don't leave your debugger paused for too long

This has happened a lot to me. In the latest version we introduced a switch to disable healthchecks during dev time so avoid this.

And forget about trying to debug something that happens on application boot

This is the trickiest definitely. I've had luck by pausing the debugger on attach (on go and node you can do this), but it's not the most optimal.

There's two parts to Okteto. The open source project (https://github.com/okteto/okteto), which launches development environments and works with any kubernetes distribution.

And Okteto Cloud (https://okteto.com) our development platform for Kubernetes Applications, that integrates everything you need to build cloud native apps (development environments, a build service, a private registry, SSL endpoints, etc...)

If your kubernetes cluster can support a 16GB pod, it will work with okteto.

That said, the pattern we recommend to follow is to split it into smaller dev envs. That makes it easier to manage, and allows you to have dedicated dev envs for different parts of your app (e.g. your API dev env and your frontend dev env).

There are challenges on this model for sure. A big advantage is that, by routing to the dev environments, you can have the dev interact directly with the webhook when they code, instead of having to wait for a shared environment to test integrations. This is pretty useful, specially if your application depends a lot on webhooks for it's main functioning.

This is an area where I'm particularly interested in. We are constantly experimenting in ways to better integrated it into our idea of remote development environments.

Ramiro here (co-founder of Okteto)

I agree that "developers should be sympathetic to the environment there code runs in". But that's about where the code runs in production, not where the developer runs it while coding. That's why we are pushing towards using remote clusters, so that you are developing on an environment that's as close to prod as possible.

In my opinion, Minikube, Docker for Desktop, Kind and others present two big challenges: First, it's not trivial to run them locally. They require extra memory, cpu and battery, they have issues starting/stopping (I don't use my local cluster that often, and I've spent way more than 5 minutes just starting/stopping/troubleshooting it). And, at the end, they have different components (load balancers, disk, network) and configuration from a remote kubernetes clusters, creating more dev-prod skew.

Second, the feedback loop is not that fast. You still need to either mount volumes (which is slow if you handle multiple files, like node's node_modules folder) or end up building containers to validate your changes. Sure, you don't have to push and pull them, but what if you didn't have to even build them?

Even npm watch takes several seconds, let alone docker builds almost always take minutes

That's where caching, container layers and build stages can help a lot. Specially for development, where you might be rebuilding the same container over and over.

How do you build containers safely in a multi tenant setup?

We have a fork of buildkit where we are implementing multitenant protections, authentication, etc ...

Hey, I'm Ramiro, one of the maintainers. We do several things to speed the whole cycle up.

When you are using Okteto Cloud to run your apps, you get all this:

- Build is run in big servers in the Okteto Cloud infra. It's based on buildkit, and it's heavily cached and optimized towards development scenarios. The build context is sent from your local machine to the service and everything else happens there. - The registry in the same infrastructure as the build services. The image is pushed from the build service to the registry, instead of from your machine. - The deployment runs in the same cluster as well, to speed up the time it takes to pull the image.

You can also use "okteto push" with your own infra, by providing your own buildkit service, registry and k8s cluster. Depending on where they are located with respect to each other this might be slower or faster (e.g using dockerhub for registry will be slower than using a dedicated one in your own cluster, mainly due to network speed and proximity).

The biggest benefits of this approach are being able to move all this to the k8s cluster, where you can leverage more compute power and faster network transmission. That way you are not pulling and pushing images across the internet.

I'm one of the creators of Okteto, thanks for the shout out!

We recently wrote a post on how to integrate Okteto with NodeJS apps, in case anybody wants to try it out: https://okteto.com/blog/how-to-develop-node-apps-in-kubernet...

We started building Okteto last year because we found that the existing tools were focused on speeding up the kubernetes cycle, and not on the end to end flow. Our goal with Okteto and Okteto Cloud is for most developers to be able to work on Kubernetes-based applications without even realizing they are on Kubernetes.

Skaffold's focus in on automating the build, push, apply part of the development cycle. Okteto's focus is on moving development to a Kubernetes cluster, abstracting away Kubernetes and the container from your inner developing loop. (I'm one of the creators of Okteto).