swatcoder

What's funny is that humans do this too, but we don't find it irritating; we just call it a speaking style. But when a machine does it, it drives us crazy. Very interesting psychological phenomenon there.

When a human does it, it's identifying. Like the timbre and dynamics of their spoken voice itself, It distinguishes them from the dozen other people you're working with on the project and the thousands of people you encounter through your days. It's signal

But when we have a handful of popular models, and they answer every question everybody has, and get quoted and forwarded everywhere, and are used to reformat and rephrase personal communication... that signal becomes noise.

Rather than voices disinguishing sources in the cacophony of our lives, everything and everyone starts to sound the same, and we lose key information that we're biologically and culturally accustomed to relying on.

Some people are likely unbothered by this in the way that some people are face blind or colorblind, and so don't see the problem. But as we see in discussions like this, many many people do get bothered by it, even if they don't yet have the insight as to put their finger on why.

You are correct.

You can't trust the agent, let alone its harness, to oberve any particular directive you give it, so "md files" provide no meaningful protection for anything important.

But users are broadly reckless and naive and commercial vendors are exploitative and irresponsonsible, so the vendors take advantage of what they can get away with for as long as they can get away with it.

Use a tight sandbox, and join the chorus loudly when others press on vendors to be make user safety an earnest and hard-to-abandon priority.

We practice engineering. As engineers, we need to understand the realities of each project we take on. Those realities are not limited to the marketing bullet points we might apply to the imagined product someday, which is what early career folk might naively call "the requirements".

Among the most critical realities to consider exist outside the product definition entirely, and have to do with the environment in which it will exist during and after development. These are things like plausible scaling curves and limits, code lifetime, team size, deployment options and preferences, etc. Others are things like available runway, team fluency and talent, available tools and licenses, etc.

One of the big eyeroll moments of the last 15 years or so was when folks started blindly cargo culting global-scale 10,000-engineer enterprise practices on projects that would obviously only ever be touched by 1-2 technicians and deployed on a far far far more modest scale. Instead of actively considering any of those environmental requirements above, "engineers" would try to treat every project as though it would someday service 10,000,000 DAU under the hand of dozens or hundreds of churning technicians.

Akin to the joke about some Americans who see themselves as "temporarily embarassed millionares", making foolish choices at their own obvious expense, many engineers during this era would seem to see themselves as "temporarily embarassed Facebooks".

In countless cases, this was simply very dumb and very wasteful and at best amounted to inadvertent (or perhaps intentional) resume-padding for the people involved, while their projects bloated and sagged and lurched under unwarranted complexity.

Sometimes (in fact: often), just doing the thing well with simple, clear, stable tools -- and nothing more -- is the right choice.

With respect, you may not have the experience to recognize the cost tradeoff you're suggesting here.

If you're choosing to rely on a strict, exhaustive validation process to approve your work product without concern for how it's produced, you shifting focus to several critical cost centers:

1. Exhaustive validation is not free and its cost grow super-linearly as your work product gains complexity -- both on its tested surfaces and in what inefficiencies it develops internally. As you invite more frequent commits of more uncoordinated code, you're running your QA process ever more often on ever more heavy code, covering ever more work surface area. This is non-trivial cost and potentially catastrophic risk.

2. AI costs scale with token requirements and project complexity. As you allow uncoordinated code to accumulate within your product, you super-linearly increase the token consumption required by AI agents attempting future commits and analyses as they digest the increasingly chaotic/uncoordinated code base, both in raw input tokens and the reasoning tokens used to strategize plans. Even if you believe that your AI agents will be able to keep up with this expanding burden indefinitely (a questionable assumption!), you're incurring another growing, super-linear cost.

3. If you're relying on frontier models to make this workflow feasible at all, you're at risk of uncontrolled cost ratcheting if and when your code grows so speghettified that only those models can maintain it. You may have economics that work today, but the model providers you use are positioned to squeeze your margin aggressively once you trap yourself within their ecosystem.

Now, it's possible that an adept team can figure out how remediate some of these concerns by appropriately tasking agents to reduce project complexity and efficiency independent of new feature development. But the overall workflow here remains uncharted territory, where few people are going to be getting it right, and many will find themselves being painted into a very expensive dead end.

TLDR; do what you want, but all the traditional concerns of technical debt management and vendor exploitation risk are at play here. They've just shifted around in a way that makes it easy for inexperienced or myopic people to think they've somehow escaped them because the underlying technology itself is so novel and unfamiliar.

In general, what you're talking about here is something that businesses have been able to do for decades by delegating their tasks to outsourced suppliers. The lesson, then and now, is that it works until it doesn't and that once it stops working, you're screwed.

So be careful -- the revolution you've promised is not what you imagine it to be!

"a different insight"

What a generous euphemism. Depending on the topic, it's not unusual to see Google AI search referencing links that indicate the exact opposite of what it presents in its prose.

So many of these tools are presented as if they're doing something you'd intuitively expect of a human operator, like collating search results and then summarizing them in this case, but the actual operation of them is so alien to us that our intuitions don't apply and these presentations are all but fraudulent.

Because it's presentation cites references, and because those references sometimes reinforce the expression of the AI prose, we grant it trust, but it doesn't use those references in the way a human would and so the relevance of them being cited is radically different (and weaker) than we're accustomed to.

The closest example in human behavior might be the rushed, naive student who just pastes citations into an already written paper at the last minute, but even that's an anthropomorphism that misrepresents the alien ways in which the tool works.

Got it. I misunderstood you. Agreed.

Can you point to some examples of where that's happening?

These data centers are specifically being scouted for communities whose governance is too weak to negotiate for some "sizeable share of their profits" and too ill-prepared to have suitable environmental regulations on the books already. The Ivy League sharks planning these buildout initiatives are sharp people who are looking out for the interests of their employers and know how to pick locations where they have the best opportunity to exploit locals unprepared for their kind of esoteric deal-making, political lobbying, and lawfare. They'd be failing at their job if they did what you're suggesting, and that's why we don't really see that happen.

For any benefit to national or global society AI data centers might provide to someone, the buildout looks a lot like the dirty and exploitative stories of rail expansion in the 19th century. That rail infrastructure proved a good thing for the US, but that doesn't mean the process of making it happen was honest or good for the people immediately affected.

People have a dozen reasons to refuse data centers being built in their communities and zero reasons to encourage it. They're little more than post-industrial mines that take limited resources (power, land, water, quiet) from a community, sell them for profit as compute, and siphon those profits away onto the books of far-flung megacorps with no community reward.

Meanwhile, golf courses are a traditional green space where people in a community gather for both work and leisure. They're not ideal themselves, but they at least provide some benefit against which their negatives can be weighed.

If all you hear from critics of data center building is water use complaints, that's strictly because you've chosen not to listen to people.

I write the code myself. I find it hard to believe why so many engineers try to avoid it.

Gradually over its recent booming years, software work went from one of several practical engineering refuges for curious tinkers and puzzle-addicts to a career path for financially ambitious bright people akin to finance, law, or medicine.

Many people carrying a "software engineer" title now never really enjoyed that part of the work at all but were suitably clever and responsible to accomplish what modest ends they were tasked to by their very generous employers. Mostly (but not entirely), those people are the ones most eager to have AI agents shield them from rigorous design and puzzle work and enable them to leverage their innate cleverness more lazily. They never really internalized the coding and engineering principles of the industry and so can't foresee what might be down the road for them with this technique, especially when they're surrounded by peers with the same mindset.

AI isn’t always faster.

It is when coding was an extremely frustrating and high friction experience for you in the first place, as is the case for many who work among us now.

shipping 10x as many features and bugfixes sounds better

I understand you're excited about the tool, but for the sake of earnest discussion here, maybe commenters like yourself can tone the hype down to plausibility?

Claims like this are just nonsense. It's not how product development works.

How do you even have so many bugs left to fix if the tool is so fast and productive? Surely, you didn't have a backlog of tens of thousands of bugs that you're still chewing through? And of course, the volume of new bugs much be minimal since the AI-composed additions introduce "no problems so far". If it works like you say, which we'll accept in good faith per HN guidelines, you must have exhausted your backlog long ago.

And if you've indeed exhausted your bug backlog long ago (incredible!), you're left to talking about shipping "10x as many features". Yet no product has a limitless capacity for features. Nobody would want to use software so bloated and churning that was gaining features at such a pace. And who is designing and specifying them so quickly anyway? If it works like you say, which we again accept in good faith, you must have stalled out on your feature list long ago.

If the AI indeed allows you to "[ship] 10x as many features and bugfixes", and we take what you say in good faith, then one of the following seems to be implied:

* you've fixed all your bugs and blew through your mature feature designs already, leaving your AI agents sitting idle for all but a few hours a week, while you're bottlenecked on feature design and your software product is bloated beyond imagination

* your coding productivity before AI was absolutely glacial by industry standards such that "10x" productivity for you is actually much closer to "0.5-2x" for others

Any insight into which of those it might be?

We rely on subtle distinction in writing style to distinguish sources from each other and to read important cues into their communication intent, the same as with vocalized speech.

Scrubbing that away from writing with AI has the same effect as would running everyone's speech through an anonymizer.

Some people may be blind to that sort of thing entirely, much like some people can't recognize faces, but for most people it just creates a sense of noise and discomfort when everything everywhere is written in the same style. And that impression is only made worse when that increasingly uniquitous anonymized style is so painfully average and repetitive.

If there was a tool that let you edit select characteristics of your writing while preserving your voice, the way a professional editor might help a professional writer, that might be amazing for helping people communicate better and more easily. But that's not this. This is loud, articulate noise.

Probably not a ton of people in the largely-peaceful, largely-comfortable world that many people here have only ever experienced.

But history (and world-awareness) shows that those periods don't last forever, so having mature decentralized technology ready and warm for periods of crisis or devolotion is hugely valuable in the long term. It can be hard to maintain commitment to maturing and seeding that kind of technology when there's not yet a pressing need, exactly because it's hard to gain enough traction to overcome the relative inconveniences. It's admirable and important work regardless.

can better infer the user’s underlying goal and intended level of work

This is a trap.

It's the optimistic fallacy that poisons all "consumer scale" machine learning products and what's going to effectively ruin these models as they keep chasing it in the same way that web queries were ruined, social media feeds were ruined, and media recommenders were ruined.

For the vendor, optimizing metrics across their whole user base, they always see positive technological progress as their system gets better at making assumptions and accumulating user engagement scores in aggregate. But for the individual user, most of which has some weird tail intent/interest and some of whom have many weird tail intent/interests, the experience quietly but catastrophically degrades. Output/results become more generic, more divergent with the underspecified "weird tail" intent, and more stubbornly hard to ever wrangle towards that "weird tail" altogether.

We've been watching this cycle happen for 20 years now and it's proving hard for anybody to escape because it works so well for the trillion dollar company driving it forward. But while each step might feel ergonomic and welcome to individual users, there's a frog boiling enshitification at play.

In pursuit of output quality and capability (rather than simply the vendor's user count), what we need rather than "makes better guesses" is "presses for more clarity", even where it feels kind of annoying.

Even among human professionals, one of the first hurdles of breaking out of junior tier work is gaining the confidence to press your colleagues and clients to be more specific in their thoughts and expressions despite their desire to have you do it all for them. But they're often coming to you with incomplete, muddy, and conflicting ideas for which there is no safe and correct assumption that you might just run with, and it's your expertise (i.e. relevant "intelligence") that's critical to bringing attention to that. To achieve professional progression, you need to learn to do that and to not just optimize appeasing the ambiguous client/colleague today in exchange for mutual expense tomorrow. To avoid enshitification, which is probably not possible, we need these models to be learning that too.

That depends entirely on whether you want to culture a humane trust society or a transactional surveillance society.

Among others, the voices on the internet include:

* some sincere people with very extreme takes,

* some trolls that masquerade as the above, to bully others over their credulousness and lack of guile, which is distinct from sarcasm,

* some trolls that insincerely speak anything that earns engagement,

* and more and more bots that mimic the above

So sadly, the answer to your question is generally yes.

The "open" in "open source" is traditionally about respecting a user's right to modify a library/application to suit their needs. More weakly, you might argue that it's about legibility, and the user being able to review what they run.

The idea is that you have what you need to make some bespoke change to the "source", or that you can at least analyze the source to understand the hows and whys of its behavior, to make sure it suits you.

Do weights provide either of those qualities?

Using AI here is not really at fault here

Maybe reality isn't represented by the git logs, but according to them, you appear to have composed, reviewed and merged 131 commits affecting 58 files and 6000 lines -- a task that you say amounts to "replacing the engine while driving" -- in the span of one week.

If that's accurate, of course your reliance on AI is at fault here. It invites you to mistake velocity for quality and conflate test satisfaction for completion.

Without AI, this effort would likely have taken much longer and an experienced team would have approached it with due meticulousness, being especially stringent in review. Absorbed in it, the work would have weighed on their unconscious while they slept, with "Ah shoot, did I consider XYZ?" moments striking them in the shower. Ideally, familiar with the specific contributors (i.e. you) and their style of work, the reviewer who later consider merging the work would have ideas what to watch out for and would spend extra time looking at the details you might not have fully considered. It's a whole decades-matured craft practice that can do a pretty good job of making sure that a refactor of this scale doesn't land in main too broken.

And while that version still probably wouldn't have been entirely flawless either, or might never have even been attempted because of its greater calendar and attention burden, that doesn't mean that using AI wasn't responsible for why and how this version is broken.

This is exactly how AI makes things worse and why many people are wary of relying on third-party projects that embrace it too blithely. If you're going to use it, that's great, and maybe it will help you keep landing big features more quickly than otherwise -- but at least accept responsibility for the tradeoffs that you're inviting when you do so.

This tells you more about inflation methodology than it does home affordability.

It's easy for some to give away money if they have it, but it's hard to accumulate that much money if you're one of those people.

Newmark maybe representing the occasional exception, most people who accumulate great wealth through their effort and ideas are afflicted with an addiction problem and have a hard time saying "this is enough for me". They become attached to how much more wealth they might be able to gather and recognize how the wealth they already hold plays a role in making the most of that.

Even if they somehow imagine themselves as philanthropists or minimalists, they tend to put off giving away too much too soon under the rationale that they'll be able to eventually share more if they hold out and use it more practically.

And if you don't have that kind of mindset, you're probably just not going to climb your way into the level of riches we're discussing here. You might still do quite well as far as most people are concerned, but that billionaire milestone is hard to catch without some propensity for wealth addiction.

That's not a distinction that people really benefit from.

Approximately nobody can read other people's code for intent or quality, let alone to surface malware meant to be hidden in it.

For almost everyone, the only hope is that somebody else validated the code you want to use before you choose to use it and successfully interfered with its distribution upon finding an issue. That's why the culture of automatic-updating package managers and bloated dependency graphs are so dangerous and why inserting delays into package managers can make such a difference in exposure to supply chain attacks for those that are intent to use them.

It's true that open source provides the transparency that makes any kind of third-party validation possible, but closed source benefits from commercial vendors staking their brand on what they release. It's a tradeoff, not a straightforward win for one side.

I never see Google return phishing pages

Maybe you're not looking or maybe you're lucky.

Either way, many of us see it happen all the time there too. For GitHub especially, I almost never get the canonical repo for a project in my Google results. Phishing or innocuous, it's almost always some fork at the top and then a bunch of non-github.com sites.

Search is more or less "cooked" now, as they say. Google vs Bing vs DDG vs Kagi is mostly in the noise.

Early work in any trade is mostly junk, and academia no exception.

But the process of creating that work, engaged throughought that process with those purported to be more practiced, is usually pretty good at seeding enough expertise and confidence that you might be able to proceed more independently and with real novelty, or might at least be prepared to share the trade with others new to it.

That's the point of those years, and so it's more than a little ironic that AI is being used to undermine a practicing expert while simultaneously eroding the traditional process for becoming one by making it so easy to just generate slop and engage with hallucinations than to actually practice writing deep work or engaging with primary sources.

Regardless of what their hiring process screens for, it's safe to say that people able to pass screening for Meta are able to get work elsewhere. It is never an engineers only option, although it may be the only one in a certain luxurious compensation tier.

And while it's true that many organizations carry dirty laundry, especially as they scale into larger organizations with fingers in more pies, Meta's business model is specifically to maximize engagement by any means available so that it can sell their eyeballs to the most lucrative advertising opportunities.

In Facebook's early days and as Whatsapp continues to do, their products may incidentally provided a useful societal function for earnestly connecting people in the way that those people wanted to be connected. But there's no way to look at it through the lens of a socially responsible business -- a qualitative difference from an organization simply not having "clean hands".

The shifting sands of commercial models or pay-per-use managed models are just really not appealing to a lot of people.

Most come with huge privacy concerns, total costs and availability are impossible to forecast very far out, and the specific behavior of frontier models in particular is not something anybody can rely on as those are subscription products that are subject behavior on their publisher's whims (whether from changing system prompts, new "safeguards", retired models, forced "updates", new regulations, etc).

It's quite hard to put a price on all that, and as more people find local models productive enough or develop curiosity to explore models, training, or harness-crafting in their own ways, the marginal cost of buying some shop hardware just sort of disappears into the budget noise for plenty enough people.

Same with many and their shop tools in other trades.

Most hobbyists and many professionals could end up far ahead financially by leveraging makerspaces, tool rentals, and co-op shops or even by hiring out a professional to prep certain intermediates for them, but they get psychological value -- as well as flexibility, reliability, and resale opportunity -- from having their own well-outfitted shop.

And they can afford that premium, so they do. At the scale of individuals and small shops, not everything that matters gets captured in financial models.

engineering seems to matter so little to a business's success

That's because your engineering career coincides with a race to the bottom, where advertising-saturated, FOMO-afflicted consumers demonstrated a preference for accumulating as many cheap/free/subsidized things that they could over a few durable, valuable things that genuinely benefit them.

It wasn't always that way, and if the economy does encounter a strong correction, it could very well change again.

Software developers in the US are seriously expensive, using them for data labeling would be a waste of resources.

The frontier work is on labeling and training expert content, by experts. It's unglamorous work and almost certainly doesn't warrant FAANG pay, but neither did most of the work that most FAANG engineers were already doing. But it does require competent talent from the expert domain.

Like their peer companies, Meta is still sitting on a huge pool of vetted-as-competant workers from the hiring boom and expert AI training is the most ripe business opportunity in a fragile economy where pretty much every comparable opportunity has evaporated.

With its IPO, SpaceX secured its role as the vehicle for consolidating Musk's vanity businesses into one closely held public organization that can more easily convert publicity into investment and internally reallocate funds and debts based on his personal whims.

So yes, SpaceX is pivoting, but it's to no one's surprise.

The assumptions are that

* "better models" will remain so signficantly more profitable for firms that have access to them that that they're effectively a "must have" for big orgs, rather than a grossly overpriced marginal gain

* said better models will only be attainable by orgs in US jurisdiction, rather than by foreign alternatives that come to be either independently or through a legally clever "cleaving" of a US-jurisdiction business interest that wants access to an eager international market

If either of those are wrong, restricting Anthropic et al to only sell to the domestic market is effectively a poison pill that makes it much harder for them to meet growth and profitability objectives and could see them lose their market-leading position sooner and more thoroughly than if they retained access to a larger market and had more flexibility.

Sure, but it's also not really out of scale with the cost of a shop tool in other trades.

If you're a professional that's confident in a positive return on the investment (optimal or not), or just a hobbyist with the luxury budget for a "shop" that cost is well within norms.

That's not everybody, of course, but it's not some inconceivable fantasy. A lot of people in the tech community here on HN, specifically, end up with pretty high discretionary budgets that they pour into stuff like this.