ownagefool

Honestly, I think you're boiling the ocean there.

Internal hosting teams ( cloud or on-prem ) should work more like public hosting environments and should focus on:

- Segregating workloads. If amazon can host a hacked service on their hosting stack without every tenant immediately being compromised, you can too.

- Self-Service. Teams should be able to just get accounts and do stuff without asking anyone.

- Building assurance. Now you've made it easy for teams to do small things, you're more likely to get more users and have less shadow-IT. You can now try and build organizational assurance without making teams suffer.

A team just needs petty cash and a cost code for a small amount of space in dev land.

What everyone does instead if ask for the planning to be done and completed upfront, which isn't realistic for a team you just stood up trying to do UR and a POC.

You also get internal cloud teams that "own cloud" taking away your ability to get accounts without going through the same all broken processes.

My take is it's actually because of poor performance of internal teams.

With a cloud provider, you can get an account by filling out a sign-up page and handing over a credit card. Internal teams will often hide behind Bureaucracy and a ticket system, with processes that make hosting a simple web app take multiple quarters.

Now there are layered reasons for that, some of them with legitimate basis, but at the end of the day they're usually low skilled teams click-ops-ing some VMWware, delivering a terrible service.

Mines was $190,594,974,587,761.20 :)

So I'm on the other side of this usually.

Most places I land there's deference to the status quo and those that came before. People assume that efforts of that past were well considered and that good work was done.

But ultimately when pressed, there's generally no evidence of the above, and when there is evidence, one can generally demonstrate the flaws in the approach, so I press on anyway and it hasn't really bitten me yet.

Of course, in many of these projects there's now deference to decisions I've made. Some of those decisions were well made, with evidence and supporting work, to make them challengeable should the understanding changes. Some of them are gut checks because they didn't matter to me at the time.

Either way the take away is good, but often practices exist simply to frustrate.

I actually don't have a problem with runners very often, but I must admit I use a combination of public and private the leans private. Are you seeing elevated errors on actions?

Whilst that's crazy, github hasn't migrated to azure yet, so it's probably not exclusively an azure problem and we've seen the same problems with amazon too ( and several other large orgs ).

Github isn't having a debate over how many 9s they have, they're having a zero 9s problem.

I think there's 3 big themes with this, thought not

1. LLM tools have added considerable load.

2. LLM used by developers to increase velocity seem to be leading more outages. This calls into question the increased velocity.

3. Roadmaps focused on pushing features that aren't reliability problems. i.e. github moving to azure, or adding AI features.

All these same problems happen to orgs with other fads that aren't AI. Following fads is not good engineering.

What's this trade off about?

I thought it was a simple 2 dims are probably better than 4, but unsure how you'd ever land on 48?

I offered self-hosting to bypass this. It did the trick and I was able to convert the enterprise customers where compliance was a red line.

I worked on a small part of one of these back in around 2013 ( specifically managing beds ).

You were talking about a team of 5 cranking this out in about 2-3 months with some longer term part time involvement, with an annual cost of less than 1m and those people mostly all dellivering several product lines ( so actual cost is half or a quater ).

This isn't true.

Our old jenkins hosts were largely forever instances with forever credentials that were just waiting to take down the org.

Modern pipelines are orchestrates that run ephemeral execution environments with ephemeral credentials that can significantly decrease the impact and timescales of getting pwned.

They're not perfect, but you can get pretty good posture by applying expertise to the subject. The problem, like always, is this expertise is neither valued nor rewarded.

Reproducible builds shows that, within a specific configuration, the code produced the binary, regardless of who signed or published it.

Indeed, this could mitigate an attacker replacing the binary with something that's not produced from the code, but it does not mitigate the tool chain or code itself containing the exploit, creating a malicious binary.

No, I was simply wrong. Thanks for pointing that out.

Both work (:

But the point was it was in a comparble situations without the microservices / k8s / whatever pet tech you want to hate on.

I no longer work in that industry.

I wrote a scheduler for VMs a long time ago. k8s is basically just the same thing but for containers.

It really confuses me how someone can argue for cloud providers over a decent open solution without realising their argument is simply they don't want to be managing the thing.

And that's fine, most teams shouldn't be neck deep in managing a platform. But that doesn't make the solution bad.

I took over tech for a POS company some years ago. They were a .net shop with about 80 developers, less than 200 concurrent connections, 6 figures spend cloud, and 0 nines uptime with a super traditional setup.

Point being, it's not the tools the causes the probem.

I ran renderapp in ECS before I ran it in k8s.

The deployment files / structure were mostly equivalent with the main differences being I can't shell into ECS and I lose kubectl in favour of looking at the AWS GUI ( which for me is a loss, for others maybe not ).

The main difference is k8s has a lot of optionality, and folks get analysis paralysis with all the potential there. You quickly hit this in k8s when you have to actually need the addon to get cloudwatch logs.

This is also where k8s has sharp edges. Since amazon takes care of the rest of the infrastructure for you in ECS, you don't really need to worry about contention and starving node resources resulting in killing your logging daemon, which you could technically do in k8s.

However, you'll note that this is a vendor choice. EKS Auto Mode does away with most of the addons you need to run yourself, simplifying k8s, moving it significantly closer to a vendor supported solution.

To be fair, I have k8s on my hetzner :p

It depends what you're doing it.

My app is fairly simple node process with some side car worker processes. k8s enables me to deploy it 30 times for 30 PRs, trivially, in a standard way, with standard cleanup.

Can I do that without k8s? Yes. To the same standard with the same amount of effort? Probably not. Here, I'd argue the k8s APIs and interfaces are better than trying to do this on AWS ( or your preferred cloud provider ).

Where things get complicated is k8s itself is borderline cloud provider software. So teams who were previously good using a managed service are now owning more of the stack, and these random devops heros aren't necessarily making good decisions everywhere.

So you really have three obvious use cases:

a) You're doing something interesting with the k8s APIs, that aren't easy to do on a cloud provider. Essentially, you're a power user. b) You want a cloud abstraction layer because you're multi-cloud or you want a lock-in bargaining chip. c) You want cloud semantics without being on a cloud provider.

However, if you're a single developer with a single machine, or a very small team and you're happy working through contended static environments, you can pretty much just put a process on a box and call it done. k8s is overkill here, though not as much as people claim until the devops heros start their work.

Sure, but you can do that in a diff after the event, rather than live.

Nobody really tries to get technical people to do the work.

Like cool, it's a great idea and would potentially produce positive results if done well, but the roles pay half the engineering roles, and the interviews are stacked towards compliance frameworks.

There's very little ability to fix a large public company when HR is involved

In more practical terms:-

1. Users - your users table may not benefit by being ordered by created_at ( or uuid7 ) index because whether or not you need to query that data is tied to the users activity rather than when they first on-boarded.

2 Orders - The majority of your queries on recent orders or historical reporting type query which should benefit for a created_at ( or uuidv7 ) index.

Obviously the argument is then you're leaking data in the key, but my personal take is this is over stated. You might not want to tell people how old a User is, but you're pretty much always going to tell them how old an Order is.

To be fair, many human tests I've read do similar.

Especially when folks are trying to push % based test metrics and have types ( and thus they tests assert types where the types can't really be wrong ).

I use AI to write tests. Many of them the e2e fell into the pointless niche, but I was able to scope my API tests well enough to get very high hit rate.

The value of said API tests aren't unlimited. If I had to hand roll them, I'm not sure I would have written as many, but they test a multitude of 400, 401, 402, 403, and 404s, and the tests themselves have absolutely caught issues such as validator not mounting correctly, or the wrong error status code due to check ordering.

I haven't looked at pfsense UI, but you can happily hand out a prefix to a device, which can then hand out its own prefixes. I do it with my k8s clusters, which means the node themseves have enough IPs addresses to launch their own routable k8s clusters.

https://renderapp.io/

A platform for digital asset management, review and workflow. Current features focus primarily on review of images aimed at automotive configurators.

The problem is generic, however, our USP is we have a couple of enterprise customers that upload packs of 60k+ assets for a round, and thus we aim to help discover what demonstrably changed.

A bit like Github, only working with images, videos, and other digital assets rather than text files.

Honestly, I think that's people reacting to the market more than it's the market reacting to people.

If your average zoomer had the ability to get a job for life that paid comparably well by a company that would look after them, I don't think loyalty would be an issue.

The problem is today, sticking with a company typically means below market reward, which is particularly acute given the ongoing cost of living crises affecting the west.

I suspect they were more referring to curmudgeons not patching.

I was engaged after one of the worlds biggest data leaks. The Security org was hyper worried about the cloud environment, which was in its infancy, despite the fact their data leak was from on-prem mainframe style system and they hadn't really improved their posture in any significant way despite spending £40m.

As an aside, I use NATs for some workloads where I've obviously spent low effort validating whether it's a great idea, and I'm pretty horrified with the report. (=

The thing about your bigco, the OPs and the post he's talking about, is it's all so abstract from money.

You have two poles here.

1. The VC route, strikes gold, and never really needs to live with the reality of asking what an ROI is, it's all talk about spotlight, impact and value, without any articulation about cash money.

2. The MBA route where you effectively can't brush your teeth without a cost/benefit analysis that itself often cost multiple times your initiative, resulting in nothing getting done until you're in some tech debt armageddon.

The reality is if you're still making bank on the abstract without being able to articulate revenue or costs, you're probably still in the good times.