jordanbeiber

Exactly this. Having spent almost three decades in enterprise context I see a lot of reinvention of something like a poor mans, unstructured, enterprise architecture - because AI agents.

I keep repeating ”what is good for humans in an organization is also good, or even required, for AI agents”.

Imagine every new instance of an AI agent as a new employee. With humans its ok to slowly accumulate knowledge through word of mouth, trail and error and the general inertia of larger orgs almost seem structured (or unstructured) knowledge-wise for this.

AI agents will never be useful in high value operations in a larger orgs without organizational knowledge available and reliable.

There’s a place for everything.

Most coding tasks take place outside of pure tech companies, if I’d venture a guess.

And let’s be honest, enterprises in general do not value that quality - and they face very little in terms of technical challenges that can’t be solved by code on stack-overflow or github.

What most enterprises lack is knowledge about themselves though - this is more a business problem than a technical one however.

This is most likely due to the fact that it is really bad at resetting blinker when the steering wheel is straight’ish again. Extremely annoying as any other car is much more sensitive (and sensible).

In a tesla an on-ramp to straight highway is rarely enough to stop the blinker, something I’ve never experienced in any other car.

Couple this with, IMO, the best baseline speaker system of any manufacturer… I’ve been driving with the blinker on for several kilometers at times!

Even if we still make a mess I think centralization of the mess is better than distributing it - what I mean is that polluting cities where millions sleep, eat, drink and breathe will probably be worse, net effect, than containing energy pollution to select places.

Running EVs in densely populated regions is probably a lot better for the population on the whole even if the net pollution would stay the same, IMO.

Still no EV is even better, but we’ve created a world where transport is often required so, one step at a time I guess.

Using AI doesn’t really change the fact that keeping ones and zeroes in check is like trying to keep quicksand in your hands and shape it.

Shaping of a codebase is the name of the game - this has always been, and still, is difficult. Build something, add to it, refactor, abstraction doesn’t sit right, refactor, semantics change, refactor, etc, etc.

I’m surprised at how so few seem to get this. Working enterprise code, many codebases 10-20 years old could just as well have been produced by LLMs.

We’ve never been good at paying debt and you kind of need a bit of OCD to keep a code base in check. LLM exacerbates a lack of continuous moulding as iterations can be massive and quick.

The problem is most likely not writing the actual code, but rather understanding an old, fairly large codebase and how it’s stitched together.

SO is (was?) great when you where thinking about how nice a recursive reduce function could replace the mess you’ve just cobbled together, but language x just didn’t yet flow naturally for you.

The argument is perhaps ”enshittification”, and that becoming reliant on a specific provider or even set of providers for ”important thing” will become problematic over time.

As go feels like a straight-jacket compared to many other popular languages, it’s probably very suitable for an LLM in general.

Thinking about it - was this not the idea of go from the start? Nothing fancy to keep non-rocket scientist away from foot-guns, and have everyone produce code that everyone else can understand.

Diving in to a go project you almost always know what to expect, which is a great thing for a business.

”Same here” meaning moving to Hetzner, but from Azure - could’ve made it less ambiguous!

Might throw together a post on it eventually:

https://news.ycombinator.com/context?id=43216847

Same here, but Azure. About 90% saved, with a very similar stack.

It is a great big cloud play to make enterprises reliant on the competency in their weird service abstractions, which is slowly draining the quite simple ops story an enterprise usually needs.

Cursor have a nice ”docs” feature for this, that have saved me from battles with constant version reversing actions from our dear LLM overlords.

We’ve gone full-on full-code.

Although we’re using temporal to schedule the workflows, we have a full-code typescript CI/CD setup.

We’ve been through them all starting with Jenkins ending with drone, until we realized that full-code makes it so much easier to maintain and share the work over the whole dev org.

No more yaml, code generating yaml, product quirk, groovy or DSLs!

Of all the paas providers Azure have the worst abstractions and services.

In general I think it’s sad that most buy in to consuming these ”weird” services and that there’s jobs to be had as cloud architects and specialists. It feeds bad design and loose threads as partners have to be kept relevant.

This is my take on the whole enterprise IT field though!

At my little shop of 30 so developers, we inherited an Azure mess, built abstractions for the services we need in a more ”industry standard” way in our dev tooling, and moved to Hetzner after a couple of years.

A developer here knows no different, basically - our tooling deals with our workflows and service abstractions, and these shouldn’t change just because new provider.

1/10-th of the monthly bill, and money partly spent on building the best DX one can imagine.

Great trade-off, IMO!

Only two cases come to mind for using big cloud:

- really small scale: mvp style

- massive global distribution with elasticity requirements.

Two outliers looking at the vast majority of companies out there.

Writing the code is NOT the problem with these enterprise project failures.

Usually decades of problem-solving have led to an absolute mess of blurry ownership and accountability.

This in turn leads to corner cutting and a road completely covered in Chesterton fences…

Tearing arbitrary fence down leads to consequences out of project scope, no one can answer questions, and no one can prioritize - this is a business problem, and no amount of fancy code (lo/hi/full/lo/left or right) will help.

If you run a bigger company and rely on IT and ERP flows, well, it’s a part of your core and you’d better treat it as such!

CRUD applications are not what most enterprises build though - they need interconnected flows of data & state-machines often supporting rather complex workflows and integration patterns.

Not realizing this is a mistake many enterprise IT shops make.

The boring crud-thingie that someone hacks together will sooner rather than later have to be integrated in a distributed system of state - this is where it gets hairy and most enterprises get stuck.

Just take the foot off the accelerator gradually.

The kia I drive still roll at a couple of km/h when accelerator is fully released and will not hit 0 unless you use a paddle by the steering wheel (or use the breaks).

But you rarely have to brake an EV though.

I’ve had two EV’s over the last 6 years, and I have to remember to occasionally hit the brakes to keep them from rusting.

One pedal driving is just money - saves power and is just much smoother over all than old school breaking. IMO.

In my experience people usually spend to much time trying to figure out what is most important instead of just doing an important thing.

But you wrote the docs and documented it’s place in the enterprise knowledge graph, because you’re a professional, right?

As an eng manager at an enterprise (not SV style tech) I differentiate between coders and developers.

Developers help our business by actively taking part in shaping software, automated workflows and integrations that makes us improve as a whole.

At lot of time is spent documenting and figuring out how that piece of software someone wrote 7 years ago, before us, works and what integrates.

This is why I firmly believe you need to tie documentation to running code in some way - at the very least a basic service graph that contains all services, dependencies and some metadata.

Tie this database to automatic firewall and proxy configuration and you’re on a good track.

We’ve on the other hand taken the strategic decision that tech and IT will help us win (is it the 90’s all over again?!) and we staff as such. A team is at least two, and no individual can own a service.

As an engineering manager I see problems with authority and process as something usually positive.

This usually leads to more things getting done “right” than “wrong”. IME.

Having the same issues/traits I’m not sure how that gets formed - my upbringing was limitless in many ways.

Question is - what are you using airflow for? My experience with airflow have been in data ETL, and if so you are not the target for something like windmill.

The target would most likely be automating HR, Finance and IT workflows and tearing down the shadow IT web of crazy integrations taking place at every larger organization I’ve ever experienced.

We’re talking “new hire” workflow for example, which at my current employer is about 25 activities in a workflow.

All assets have to be lifecycle managed in an enterprise and automated workflows will help you scale that. Far too many enterprises have a lot of people shuffle excel files and emails around to fulfill processes and workflows.

Airflow is a beast imho and usually not used in the same niche IME.

I’ve worked for years implementing tools like this in enterprises.

My most fond memory was introducing a workflow platform to an enterprise with a fully outsourced IT-ops department - it was ruining everyone else in terms of cost, speed and quality.

The security dept (this was a large bank) was gridlocked in this setup and wanted the ability to automate their way out of the sourcing mess.

I spent roughly three months building a few “hot path” workflows important to them which enabled them to take the ownership back of the processes and save an incredible amount of time and money.

Encapsulating these integrations as workflows makes them observable and measurable. The customer had in the first quarter after deployment 10’s of thousands runs and avg time to completion went from 2 weeks to 2 days. It also cut out an rather expensive middle man.

And this is not the worst enterprise customer I’ve worked with. One hade 4000 Windows servers manually provisioned and managed. There’s low hanging fruit out there!

You basically trade agility and quality for competence, unfortunately a lot of enterprise IT shops are not willing or capable to do so.

They have “not only one way or the other” - they have “all the ways” and this is exactly the problem you want a solution like this to fill.

We’re talking backoffice/cost-center workflows in IT, finance and HR - absolutely not business profit processes mind you.

A tool such as this can help an IT department take ownership of integrations and workflows, building them in a framework that can improve speed and quality. It will help with organizational scalability.

The alternative is in my experience a giant mess of integrations lacking ownership and observability.

Every large enterprise needs “something” like windmill, they might just not know it.

In a way it is (and also a humanity problem).

It would be great if a bunch of car parkings could be converted to secure bike storage spots.

And I've had several bike stolen over the years (include being mugged with my bike)

So, let me get this straight - it’s quite likely a bike gets stolen and this experience, as shared with me, has led you to take precautions.

My precaution being not taking the bike some times and to some places.

How many times have you have a car stolen over that same time period?

I can emphasize “feels”, but if you’ve ever parked a moderately expensive bike in a dense city center for a few hours you know the feeling. Especially if you’ve had a few stolen. If not the bike, the saddle. If not the saddle a wheel. If not the wheel the battery. Etc etc. It’s all really annoying trying to get home not matter the insurance.

I had two strollers stolen from my backyard while at home last year… it’s kinda crazy over here!

Bike will get stolen anyway which is really annoying no matter the insurance. Not saying bike isn’t preferred to car, it’s just something I always consider.

I’ve got a bike AND a car, I still take the car on certain trips where the bike would make more sense logistically because it feels like 50/50 it’ll get stolen.

That’s usually comp for simply being on-call - all agreements I’ve ever had includes increased compensation for actual on-call work a.k.a overtime compensation.

If you have drift in actual state using Ansible you have to account for all eventualities and you simply can’t declare “make my system(s) look like this”.

I’m talking “desired state” here and how to fulfill it - not “various states to switch between”.

With nix(os) I feel you treat a machine more like an appliance!

Biggest drawback of ansible is that it is stateless and as such you can’t simply declare a desired system state and apply it.

Nix is absolutely awesome in that regard.

Possibly this is Ansibles biggest strength as well, as it feels incredibly simple to get going.

Salt has a much steeper learning curve due to both weird nomenclature and the infrastructure.

Nix is in many ways easy to use “on the surface” but quickly becomes hairy as you dive deeper.

Just my 2c having spent quite some time with all three.