yannyu

If AI has a survival instinct, then we should theoretically see evidence of it if we construct the right environment for AI to express it. Animals and cellular organisms demonstrate a survival instinct under the right conditions, so we would have to find equivalent conditions for a hypothetical machine intelligence.

Conversely, we know that if we take animals that do have a survival instinct and put them into the wrong kinds of environments, they will not thrive and will degenerate or possibly commit suicide. Similarly, if AI did have a survival instinct, do we think we've created an environment where that could be reasonably tested and observed?

If an active shooter is the anticipated threat, how does a turnstile effectively stop that? Many of these turnstiles are specifically meant to allow people through in emergencies, and aren't strong enough to withstand bullets or even a sturdy kick. The elevator restrictions would be a better chokepoint, but as the article noted they didn't turn those back on.

I think you're right, but I also think it's worthwhile to look at Edward Bernays in the early 1900s and his specific influence on how companies and governments to this day shape deliberately shape public opinion in their favor. There's an argument that his work and the work of his contemporaries was a critical point in the flooding of the collective consciousness with what we would consider propaganda, misinformation, or covert advertising.

Did you read the article you cited or are you just evaluating the snapshot of numbers?

WASHINGTON, D.C. -- Forty-four percent of Americans say they are “very satisfied” with the way things are going in their personal life, the lowest by two percentage points in Gallup’s trend dating back to 2001. This also marks the continuation of a decline in personal satisfaction since January 2020, when the measure peaked at 65%.

Record-Low 44% of Americans Are 'Very Satisfied' With Their Personal Life

And then to link to your own blog post as though that were a supporting citation is strange to say the least.

It's a lot of "just stop being depressed" energy.

Learning is effortful. People can travel and not learn anything, but people can not learn from many things they should learn from. Travel is something you can learn from no matter where you go, but you typically have to put in the effort.

This is an interesting idea, I think clickbait titles are one of many problems with our engagement-based social media tools today. For the sake of experimentation and transparency, here's the suggested titles from ChatGPT 4. They seem to be more descriptive and accurate overall.

---

Possible alternative titles that better match the article’s content:

How Phishers Are Using SendGrid to Target SendGrid Users with Political Bait

– Accurately reflects the mechanism (SendGrid abuse), the audience, and the novel political/social-engineering angle.

SendGrid Account Takeovers Are Fueling a Sophisticated Phishing Ecosystem

– More technical / HN-native framing, avoids culture-war implications.

Phishception: Politically Targeted Phishing Sent Through Compromised SendGrid Accounts

– Highlights the core insight and the self-reinforcing nature of the attack.

Sure, it worked and then it stopped working. Upvotes, reviews, social media, and word-of-mouth have been co-opted by advertising and marketing.

Or maybe companies turned products that were one-time purchases into monthly subscriptions. Or companies made it incredibly hard to cancel monthly subscriptions. Or companies will continue to charge and auto-renew us even when we clearly don't use the product. Or companies will unilaterally degrade our service or push us to lower tiers in order to better serve customers who aren't you. Or companies will have us pay a monthly subscription, and then also sell our data to the highest bidder anyway.

Maybe many of these companies have never had our interest at heart, and people are tired of feeling constantly screwed over and seen as a revenue stream instead of customers.

One thing I don't understand in these conversations is why we're treating LLMs as if they are completely interchangeable with chatbots/assistants.

A car is not just an engine, it's a drivetrain, a transmission, wheels, steering, all of which affect the end-product and its usability. LLMs are no different, and focusing on alignment without even addressing all the scaffolding that intermediates the exchange between the user and the LLM in an assistant use case seems disingenuous.

If the only tactic the police knew was to pull over every Infiniti with tinted windows and no plates, the crime rate would drop to zero.

Then the question is, why don't they do that? Why do we need a surveillance state to enable police to do what residents might consider the bare minimum?

Well, then we should judge the ideas on their own merits. And it's also not a great idea.

It's a shallow, post-hoc, mystic rationalization that ignores all the work in multiple fields that actually converged to get us to this point.

I think it's even more pernicious than the paper describes as cultural outputs, art, and writing aren't done to solve a problem, they're expressions that don't have a pure utility purpose. There's no "final form" for these things, and they change constantly, like language.

All of these AI outputs are both polluting the commons where they pulled all their training data AND are alienating the creators of these cultural outputs via displacement of labor and payment, which means that general purpose models are starting to run out of contemporary, low-cost training data.

So either training data is going to get more expensive because you're going to have to pay creators, or these models will slowly drift away from the contemporary cultural reality.

We'll see where it all lands, but it seems clear that this is a circular problem with a time delay, and we're just waiting to see what the downstream effect will be.

Ah, that's helpful and something for me to learn more about. Thanks for the info.

I've thought about this a lot as I see more and more reckless driving in the areas I live in. Surveillance is generally a net negative, but it's also bad when you see people speeding around schools, rolling through stop signs, and running red lights. We seem to have a worst of all situations where traffic is getting increasingly difficult to enforce, driving is getting more dangerous year by year, and we're terrified of government overreach if we add any automation at all to enforcement.

I don't know the solution, but I do know that in the US we've lost 10-15 years of progress when it comes to traffic fatalities.

Why would someone in an Idaho suburb care about how Manhattan manages its congestion pricing? Why is this national news?

Everything you're saying has zero impact on 93-97% of the US population (New York State is 6% of the US population, NYC is 3%). None of these people have real skin in the game, because this literally has no effect on them. New Yorkers don't vote in other states.

Why is a single student's grade in OSU national news? Why is congestion pricing national news? Why is a library in the middle of nowhere California news?

None of these things are actually related to why people are stretched thin and getting screwed by the system. In fact they're exactly unrelated which is why we're blasted with this stuff on the news 24/7. You're worried about a slippery slope argument when most of us are already being fleeced by current, real policies from government and corporations.

Congestion pricing is not the thing screwing over American families, it's the thing they're pointing at so you don't look at the actual thing.

The last time the world had multi-polarity, we had far more wars. Including the worst wars the world has ever seen.

Citations? Simply saying that World War 1 happened during a time of multi-polarity is just begging the question. Multi-polarity of varying degrees has always been the case throughout human history, and often times single-polarity is achieved only through extreme violence.

This is just "might makes right" but modern. I don't know that this is the consistent, wide-held belief that you seem to think it is. Plenty of people would rather our governments not engage in clandestine disruption and undermining of foreign governments.

Competition is inevitable, especially between geopolitical rivals, but we don't have to engage in Minitrue-style "the enemy has always been our enemy" rhetoric.

It's a chicken and egg issue. We don't build dense, mixed-use housing and commercial, we don't build transit. There's no way to live a Tokyo-like lifestyle in 95% of California. And the places where you can are often exorbitantly expensive.

They know that congestion pricing coming to their city would hurt them in very real ways.

But why should they even care to begin with? Just because the news and media made them aware of congestion pricing? This is the whole problem, that local issues are made mainstream news media specifically to cultivate fear and anger in people that literally have no skin in the game and a completely different lifestyle.

Two things can be true:

1) I wish we had better rail transit in the bay area and to the areas surrounding the bay

2) I have to own a car to get to places in Northern California

These don't seem like remotely contradictory positions.

And that's exactly the point, it increases engagement and stickiness, which they found through testing. They're trying to make the most addictive tool and that constant praise fulfills that goal, even as many of us say it's annoying and over-the-top.

My own experience is that it gets too annoying to keep adding "stop the engagement-driving behavior" to the prompt, so it creeps in and I just try to ignore it. But even though I know it's happening, I still get a little blip of emotion when I see the "great question!" come through as the first two words of the response.

I think there's a lot of utility to current AI tools, but it's also clear we're in a very unsettled phase of this technology. We likely won't see for years where the technology lands in terms of capability or the changes that will be made to society and industry to accommodate.

Somewhat unfortunately, the sheer amount of money being poured into AI means that it's being forced upon many of us, even if we didn't want it. Which results in a stark, vast gap like the author is describing, where things are moving so fast that it can feel like we may never have time to catch up.

And what's even worse, because of this industry and individuals are now trying to have the tool correct and moderate itself, which intuitively seems wrong from both a technical and societal standpoint.

I agree entirely, and I think it's worthwhile to note that it may not even be the LLM that has that behavior. It's the entire deterministic machinery between the user and the LLM that creates that behavior, with the system prompt, personality prompt, RLHF, temperature, and the interface as a whole.

LLMs have an entire wrapper around them tuned to be as engaging as possible. Most people's experience of LLMs is a strongly social media and engagement economy influenced design.

It's not just the pretraining, it's the entire scaffolding between the user and the LLM itself that contributes to the illusion. How many people would continue assuming that these chatbots were conscious or intelligent if they had to build their own context manager, memory manager, system prompt, personality prompt, and interface?

I agree 100%. Most people haven't actually interacted directly with an LLM before. Most people's experience with LLMs is ChatGPT, Claude, Grok, or any of the other tools that automatically handle context, memory, personality, temperature, and are deliberately engineered to have the tool communicate like a human. There is a ton of very deterministic programming that happens between you and the LLM itself to create this experience, and much of the time when people are talking about the ineffable intelligence of chatbots, it's because of the illusion created by this scaffolding.

There's no definition of thinking that isn't a purely internal phenomenon, which means that there's no way to point a diagnostic device at someone and determine whether they're thinking. The only way to determine whether something is conscious/thinking is through some sort of inference, which is why Turing landed on the Turing Test that he did. Problem is, technology over the past 5 years pretty easily passes variations of the Turing Test, and exposed a lot of its limits as well.

So the next definition of detecting "thinking" will have to be externally observable and inferrable like a Turing Test, but get into the other things that we consider part of consciousness/thinking.

Often this is some combination of introspection (understanding internal states), perception (understanding external objects), and synthesis of the two into testable hypotheses in some sort of feedback loop between the internal representation of the world and the external feedback from the world.

Right now, a chatbot can say all sorts of things about itself and about the world, but none of that is based on real-time, factual information. Whereas an animal can't speak, but they clearly process information and consider it when determining their future and current actions.

Innovator's dilemma. Leadership won't invest in the up-and-coming product because they've got a $1 billion revenue target they need to hit this year.

Funnily, Netflix is a common case study on how to transition past the dilemma.

I don't remember where I heard the original story, but this snippet from this article sums up why and how they deliberately cut the DVD team out of the company culture.

“In periods of radical change in any industry, the legacy players generally have a challenge, which is they’re trying to protect their legacy businesses. We entered into a business in transition when we started mailing DVDs 25 years ago. We knew that physical media was not going to be the future. When I met Reed Hastings in 1999, he described the world we live in right now, which is almost all entertainment is going to come into the home on the internet. And he told me that at a time when literally no entertainment was coming into the home on the internet. And it really helped us navigate this transition from physical to digital, because we just didn’t spend any time trying to protect our DVD business. As it started to wane, we started to invest more and more in streaming. And we did that because we knew that that’s where the puck was going. At one point, our DVD business was driving all the profit of the business and a lot of the revenue, and we made a conscious decision to stop inviting the DVD employees to the company meeting. We were that rigid about where this thing was heading.”

https://colemaninsights.com/coleman-insights-blog/netflixs-s...

I think there's some significant evidence that "users" of the private healthcare industry are actually quite unhappy about it. Due to some structural quirks of the US, it seems like it's less about how many people are happy or unhappy and more about how many dollars are spent in favor or opposition of a change.

The article takes this on a couple times:

The outcome is $4.9T - which would make it the 3rd largest economy in the world, a high 8% admin costs - compared to the UK’s 2% admin, with medical bankruptcy still possible. We’ve never agreed on what we value. So we built a system that embodies our disagreement: employer-based coverage (market choice) plus Medicare (social insurance) plus Medicaid (safety net) plus exchanges (regulated markets).

Decision #1: Workers pay at least twice

Here’s the first thing that jumps out: if you work a job in America (and you presumably do, to afford the internet where you’re reading this), you’re already paying for healthcare in multiple places on this chart:

    Taxes: federal, state, and local taxes finance Medicare, Medicaid, and various public health programs in so many places. Our attempt at embedding it in single payer.

    Payroll: if you’re employed, your employer pays taxes on Medicare (even though you presumably can’t use it until you retire at 65). This is a cost that doesn’t go to your salary.

    Insurance premiums: get deducted from your paycheck to fund the employer group plans ($688B from employees alone).

The operational resistance alone would be too much: I’ve watched hospital execs squeeze out thinning margins and payer executives navigate quarterly earnings calls. We’re talking about unwinding a $1T+ private insurance industry, reconfiguring every hospital’s revenue model, and convincing Americans to trust the federal government with something they currently (sort of) get through their jobs. That ship didn’t just sail - it sank decades ago.

In the current setup, having enough money protects you from the laws of the country and the judgment of others. Thiel is rich enough and therefore powerful enough that these culture wars will never personally affect him or anyone he cares about.