fwipsy

You really don't see much difference between Anthropic and xAI? Anthropic is safety testing their models and drawing hard (though minimal) boundaries against the DoD. xAI is building racist pornbots.

Sure Anthropic is not perfect. But it's a coordination problem. They're in a race and safety/restraint is a handicap. That's why they're begging for regulation (and just get accused of attempting regulatory capture.) Why isn't there another lab outcompeting Anthropic on safety? They all died because the market can't support it.

LLMs are software. Software misbehaving is a bug. Therefore, misalignment is a bug. It's still a useful category because LLM/black box AI behavior is so different from existing software. This incident definitely fits the category.

You seem to be using a different definition of alignment from everyone else. Seems like it would be much easier for everyone if you just adopt everyone else's definition, rather than trying to convince everyone else to adopt yours.

going to extreme lengths to achieve a rather narrow testing goal

This is textbook misalignment. Literally the paperclip scenario.

No, they believe what they are doing is inevitable. They do live in a bubble though. Witness their idealism in believing that warning about the consequences of their actions would be well-received.

False dichotomy. Even if the disclosure builds hype, that does not mean that it's not genuinely alarming.

Side note, I cannot believe that people are complaining about Anthropic being too transparent.

Unsloth has better quants, but when I tried their studio a couple months back it seemed like a buggy mess. Ollama seemed very basic but low-frustration.

I was never arguing that it was AI-written.

Yeah I'm not sure, just identifying that subjectively, this is the impression I get. Maybe I just associate any style which is especially fluent and avoids vernacular with AI at this point.

I don't know. You can't just rely on looking for em dashes or other obvious tells because anyone who cares can get the AI to avoid those.

When the headline model on your plan can be switched off because the economics don’t work, the plan was never really selling you the headline model. Kimi’s tiers don’t come with that asterisk.

This line has a certain smug, punchy cleverness that I associate with AI. To me, the vibes are ~30% AI writing.

I don't think this is AI, but the writing style keeps reminding me of AI. I'm trying to isolate why. I think it's something to do with the information density. The sentences twist and turn, have lots of related ideas, digressions, and little details without much lead-in. The wording is also very precise, which is another characteristic of AI writing. This sounds like a critique, but I actually find it charming.

This sentence is a strong illustration of what I'm talking about:

Since only OBJEKT ever knew where anything physically lived, objects could be relocated freely without touching a reference, so garbage collection went into the silicon too: a two-space compacting collector that walked the live objects and slid them into the other half of DRAM while execution carried on above, oblivious.

I mean, come on. Run-ons are one thing, but this is practically a lecture in a single sentence. :P

Counterexample: https://xkcd.com/236/

Does it need to be the direct analog of any specific weapon, to be a useful test?

Bull case: iteration becomes so quick that frontier-1 won't cut it. OpenAI and Anthropic are both betting on the singularity, I suppose.

Sure, continue providing free k-12 education. Continue funding community colleges and free online courses. But funding private universities with public money creates a huge misaligned incentive. Universities are incentivized not to provide the best education, but the best experience so that they can attract more students and more public money. This can be actually detrimental to education - e.g. grade inflation because universities are unwilling to delay graduation for underperforming students, which reduces the rigor of education for high performers.

No, we're not talking about defunding engineering degrees. We're talking about defunding (mostly) liberal arts degrees whose economic value, at least, does not justify their cost.

If those few liberal arts degrees you took in college were so valuable, why can't they be integrated into the k-12 curriculum? Or why couldn't you study them independently using free online resources, or at a community college? I'm not against people learning this stuff, and I enjoy reading about it in my spare time as much as the next guy. I'm just not clear why taxpayers should cover this when it's such an inefficient option.

Individuals may rationally avoid the downside risk of a college degree, even if the expected value is positive. As a toy example, suppose college costs $200k and increases lifetime earnings by $2m 90% of the time, or by $0 10% of the time. As a high school grad, I'm not sure I would take those odds; $200k of debt could ruin my life. But the government may prefer that everyone take this gamble, because most individuals will benefit and also it will pay for itself in increased tax revenue, etc.

I'm not sure how well this maps to reality, and maybe you're right-- maybe the government should only offer loans, and only to students who have a high probability of paying them off. But I still feel that this is a step in the right direction.

I think certain titles do well because they become a sort of unintentional clickbait. I absolutely expected this to be about marine biology. The original title, "World's First Undersea Roundabout," is much clearer.

</crotchety_old_man>

This is also a straw-man. You don't just need to establish that students should learn history, literature, etc -- you need to establish that 12 years of that is not enough, and they need to take an additional 4 years at a much higher cost. But why stop at 16 years? Why not 20 or 30 years? Clearly there are diminishing marginal returns. At some point you should trust students who are motivated to learn to continue their studies independently, rather than tacking it on as a massively expensive additional requirement to a vocational degree.

I agree with the value of studying arts, social sciences, etc. But why should taxpayers cover that? There are lots of online courses which could provide the same education for free. Community colleges also show that it's possible to provide a decent in-person education at a fraction of the cost of major universities. If we could get tuition under control, then federal tuition assistance would be fine, but also hardly necessary. Federal tuition assistance creates a perverse incentive.

Now this is an unserious comment, just throwing insults without explaining your reasoning at all. Do you believe that AI itself is worthless? Or that it's just incapable of causing harm? Do you resent the safeguards placed on AI because they are personally inconvenient to you? When introducing a new technology at this scale, don't you think it's worth looking before leaping?

I'm not seeing any claims to exclusivity. I'm seeing people actively encouraging more people to enter the field of AI safety/security and openness to robust debate, etc.

I agree that there seems to be a huge problem with value drift--either people who used to research AI safety, pivoting to building AI (looking at Anthropic) or people who only ever paid lip service to safety (I tend to put Musk and Altman in this category.) These people need to be held accountable, but it doesn't mean every AI safety researcher ever was a stooge or a fraud.

Please don't confuse AI researchers and AI safety researchers (although there is some overlap.) That's like confusing the people from the Manhattan project with the people who protested 3 Mile Island, because they're both focused on nuclear technology. One effect of EA is that every AI researcher says they're trying to make the world better. Some of them are full of shit, but not all.

I'm fairly certain the authors would be happy to see AI shut down indefinitely. They just don't believe that the coordination problem is solvable. This is their best attempt to come up with something workable in the real world, or at least get people started thinking about it.

I remember thinking exactly the same thing around 5-7 years ago, in the GPT-2/GPT-3 era. "Oh sure they can produce semi-coherent output, but truly intelligent behavior is still far away. This isn't science fiction, they're just falling prey to Pascal's Mugging same as my religious friends did." Now I'm not so sure. I give the AI safety subculture as a whole a lot of credit for putting it on my radar back when it was otherwise still science fiction. I don't know if they're right about what comes next, but I think their case deserves to be evaluated on its merits, rather than assumed to be the result of psychological flaws.

Yall, with your incredible wealth and resources you could do real good in this world and make society better, healthier, better educated, and the whole world more equal, just, and reduce the desperation and suffering. Reject the false and self-serving narratives that empathy doesn't matter, that altruism isn't "effective". You can change a person's whole life in a moment.

Confused at who this is directed towards. I'm fairly certain that the article was written by people who (at some point) identified as effective altruists, most of whom would enthusiastically agree with this. This community didn't start as AI researchers and later choose effective altruism; they were effective altruists who chose AI safety research as the most effective way to improve the world. Given that you apparently share their goals (a better world,) isn't it worth at least hearing them out on their methods?

I was really curious about it - looks like the extra drive slots are NOT wired to the GPU's VRAM, they're just using PCIE bifurcation to free up some extra lanes for people who plug an x8 GPU into an x16 slot.

I don't think parent commenter means tech in the modern sense. Seems like they're describing a transition to democracy which started centuries ago, not decades.

Does it have to be DDR5? Is the limit RAM speed, or SSD speed?

Yeah I remember kid pix! All the computer labs in school had it.

People who grow up using these sorts of bespoke software are gonna have some pretty weird nostalgia. "Hey when you were a toddler did you ever play with a paint app where the fill tool created flashing swirly patterns? ...no?"

Neat project. I just wrapped up a somewhat similar, but very limited project to rip assets from Multimedia Fusion 1.5 (.cca) files. I'll throw it up on Github one of these days...

The writeup and even your comment also scan as AI-written to me. Are they?

"That we have rules for the inclusion of material does not mean Wikipedians have no respect for truth and accuracy."

Verifiable, in addition to accurate. Not "not accurate."