walleeee

4) energy and materials scarcity and compounding ecological externalities

this of course affecting not only the Western regimes but technocratic rule everywhere

I'm not endorsing victimhood or saying that Bangladesh is poor because the US is rich. It was badly phrased if that is how it reads, for that I apologize.

I'm saying the US does what it can to keep itself richer than other countries all around the globe by immoral means. It is not unique in this. This is not the only reason the US is rich or the only reason any other is not.

I'm also saying that this is a really bad strategy if the goal is humankind flourishing on this planet. People already enrich one another in many ways. We have to stop warring on one another and nature, thoughtlessly dumping entropy where we can't see it, etc.

Yes, so many.

This may be true to some extent, but what I dislike about this idea is that to many it implies that human flourishing is impossible without suffering exported elsewhere

To some degree this is true, in the sense that human flourishing implies some degree of suffering for e.g. the ants we accidentally step on, animals we eat, bacteria in our guts, etc. But Jains do their best not to step on the ants, many people refuse to eat flesh, and so on. Plants and bacteria will have to fend for themselves for now. We can certainly do better with each other.

I am proposing no version of fatalism, besides the fact that, at least in our living substrate, organisms have not all learned to do each other no harm, if this is even possible, and even if it isn't, fatalistic hedonism is not the inevitable response to this fact.

The life possible in San Francisco is possible in large part because of the entropy it externalizes to "somewhere else", a place many in SF and the US more generally prefer to ignore, and which includes Bangladesh, alongside as much of the "global south" as can be coerced into lopsided agreements, and much more generally the possible futures available to humankind. Cell membranes externalize entropy in the same way, the difference is they are good enough at working together, and working with the world around them, which is perhaps frequently hostile but also nutritious and useful, that higher-order life emerges nonetheless.

Much work in e.g. anthropology shows the "default state" of humanity is not nearly as well-defined as "subsistence agriculture". That is recent and it is a prototype of a strange phenomenon at the limit of which is San Francisco, a truly unusual bubble of order and some degree of flourishing, for the moment. If we were wiser we would be trying to extend the cell metaphor to the planet as a whole, which would benefit people in San Francisco and Bangladesh alike. Part of that includes retiring the war against nature mentality, it does very little good imo.

Can you characterize the binary choice in your last sentence in more detail? I don't see how one excludes the other

Strongly agree. Our beginning to grasp how it works even adds to the miracle!

But what we're beginning to understand we're also destroying far more rapidly even as we devote more and more effort to digital facsimiles of our faculties

My dry pessimism was a comment on this foolish inversion of priorities. Alien visitors, if the forest is truly dark, may conclude we're well on the way to self-annihilation in pursuit of false idols, while we fail to recognize the miraculous nature of existing biology.

On what grounds do you think it likely that this phenomenon is at all related to consciousness? The latter is hardly understood. We can identify correlates in beings with constitutions very near to ours, which lend credence (but zero proof) to the claim they're conscious.

Language models are a novel/alien form of algorithmic intelligence with scant relation to biological life, except in their use of language.

This also evidences that in this case, it's more developers of handbrake just know their audience rather than a real design failure. Maybe they'd prefer to keep the user base deliberately small?

Fewer every year. If 3I/A is ET flyby, perhaps our progress trading primitive simulacra for biological miracles will satisfy it that we pose no great risk, annihilation being costly and reserved for planets with better long term odds

Way bigger and deeper than that, there was some slack in the energy situation remaining at that point. Not any more.

You don't consider an application that is designed specifically to siphon all the attention it can to be a perverted implementation?

Surprise, it's harder for people to self-govern when they are surrounded by mechanisms designed deliberately to subvert their capacity to self-govern.

A chat like this is not a solution though, it is an indicator that our societies have issues

Correct, many of which are directly, a skeptic might even argue deliberately, exacerbated by companies like OpenAI.

And yet your proposal is

a company to tackle this at scale.

What gives you the confidence that any such company will focus consistently, if at all,

on help, not profits

Given it exists in the same incentive matrix as any other startup? A matrix which is far less likely to throw one fistfuls of cash for a nice-sounding idea now than it was in recent times. This company will need to resist its investors' pressure to find returns. How exactly will it do this? Do you choose to believe someone else has thought this through, or will do so? At what point does your belief become convenient for people who don't share your admirably prosocial convictions?

legislation

Perhaps more generally phrased as governance

Yes, the answer is not some business plan by which some can dodge disaster in an untrustworthy market, the answer is to recognize that this planet is a spaceship i.e. materially closed, and we are massively soiling the nest, microplastic is in steak because it's literally everywhere on the surface of the earth, etc.

Therefore, good ecological governance is a requirement, as is the analysis, as a public service, of the resources and ecosystems, and the services they provide human beings and our dependents, i.e. a democratic and just policy, not a lucrative plan to privatize yet more of public health

If one is convinced the best vehicle for the above in the near term is a business, then it had better have a different approach than is typical of personal health tech startups

Empowering individuals isn't worthless by any means but pitting one against another with asymmetric information is worse than worthless

Painting opposition as "sour grapes" is an extraordinarily bad faith take.

"Decentralize it" isn't a bad instinct in many contexts but I am sure you are capable of conjuring a counterexample to every one of the cases you mention with a modicum of effort/imagination.

Apologies, not meant to be smug

The question in your last paragraph is not the only one that matters. Funding the technology at a material loss will not be off the table. Think about why.

whether they confuse people by accident or design.

Both, surely

And I have no idea when it's going to start nor how to fireproof my investments !

You don't, that's the idea.

It certainly does impact you if nearly everyone else is using them.

It's not only that Linus is atypical, it's also that he is reviewing other people's code, and those people are also highly competent, or they would not be kernel committers. And they all share large amounts of high-quality and hard-earned implicit context.

Reviewing well executed changesets by skilled developers on complicated and deliberate projects is not comparable to "fleet of agents" vibe engineering. One of these tasks will sharpen you regardless how lazily you approach it. The other requires extreme discipline to avoid atrophy.

Effects like these, those of our tools on ourselves which occur slowly/subtly enough that we hardly notice, underlie a great many of our greatest problems, I think

"This level" presumably being when the accumulation of progress past at last overwhelms us in all its complication?

I don't really understand yet why people don't see that they are doing this to themselves.

Maybe it has something to do with the purveyors of these products

- claiming they will take the jobs

- designing them to be habit-forming

- advertising them as digital mentats

- failing to advertise risks of using them

And energy is the air in the economy at large. The Eliza effect is not the only bias disposing people to believe AGI is right around the corner. There are deeper assumptions many cling to.

I agree about ML.

LLMs are a bubble on the back of an AI bubble in a market finally surmounting an enormous single-shot energy surfeit

we all should hope

If you're serious about anything, you do more than hope. You do diligence on your prospective employer before going to work for them. You think through a litany of contingencies and prepare a plan of action for each. Jobs in this industry are uniquely amenable to this by virtue of their relatively higher compensation and the autonomy often afforded to employees. If you spend an hour every day on HN, you can spend an hour meditating upon your conscience.

Predicting one's response to stressful and unexpected circumstances is hard. So try to anticipate circumstances and cultivate relevant virtues in advance.

Regardless whether the rest of society finds the programmers responsible, the integrity of that society depends upon programmers in such situations holding themselves accountable. Apparatchik or moral agent? That choice remains ours.

No, it's probably not that simple. His self-interested maneuvering is without doubt part of the story, but it's very unlikely to be all of it.