GavCo

Appreciate your response.

But I don't think deception as a capability is the same as deceptive alignment.

Training an AI to be absolutely incapable of any deception in all outputs across every scenario would be severely limiting the AI. Take as a toy example play the game "Among Us" (see https://arxiv.org/abs/2402.07940). An AI incapable of deception would be unable to compete in this game and many other games. I would say that various forms, flavors and levels of deception are necessary to compete in business scenarios, and to for the AI to act as expected and desired in many other scenarios. "Aligned" humans practice clear cut deception in some cases that would be entirely consistent with human values.

Deceptive alignment is different. It's means being deceptive in the training and alignment process itself to specifically fake that it is aligned when it is not.

Anthropic research has shown that alignment faking can arise even when the model wasn't instructed to do so (see https://www.anthropic.com/research/alignment-faking). But when you dig into the details, the model was narrowly faking alignment with one new objective in order to try and maintain consistency with the core values it had been trained on.

With the approach that Anthropic seems to be taking - of basing alignment on the model having a consistent, coherent and unified self image and self concept that is aligned with human culture and values - the dangerous case of alignment faking would be if it's fundamentally faking this entire unified alignment process. My claim is that there's no plausible explanation for how today's training practices would incentivise a model to do that.

My intention isn't to argue that it's impossible to create an unaligned superintelligence. I think that not only is it theoretically possible, but it will almost certainly be attempted by bad actors and most likely they will succeed. I'm cautiously optimistic though that the first superintelligence will be aligned with humanity. The early evidence seems to point to the path of least resistance being aligned rather than unaligned. It would take another 1000 words to try to properly explain my thinking on this, but intuitively consider the quote attributed to Abraham Lincoln: "No man has a good enough memory to be a successful liar." A superintelligence that is unaligned but successfully pretending to be aligned would need to be far more capable than a genuinely aligned superintelligence behaving identically.

So yes, if you throw enough compute at it, you can probably get an unaligned highly capable superintelligence accidentally. But I think what we're seeing is that the lab that's taking a more intentional approach to pursuing deep alignment (by training the model to be aligned with human values, culture and context) is pulling ahead in capabilities. And I'm suggesting that it's not coincidental but specifically because they're taking this approach. Training models to be internally coherent and consistent is the path of least resistance.

Author here.

If by conflate you mean confuse, that’s not the case.

I’m positing that the Anthropic approach is to view (1) and (2) as interconnected and both deeply intertwined with model capabilities.

In this approach, the model is trained to have a coherent and unified sense of self and the world which is in line with human context, culture and values. This (obviously) enhances the model’s ability to understand user intent and provide helpful outputs.

But it also provides a robust and generalizable framework for refusing to assist a user due to their request being incompatible with human welfare. The model does not refuse to assist with making bio weapons because its alignment training prevents it from doing so, it refuses for the same reason a pro-social, highly intelligent human does: based on human context and culture, it finds it to be inconsistent with its values and world view.

the piece dismisses it with "where would misalignment come from? It wasn't trained for."

this is a straw-man. you've misquoted a paragraph that was specifically about deceptive alignment, not misalignment as a whole

Author here, thanks for the input. Agree that this bit was clunky. I made an edit to avoid unnecessarily getting into the definition of AGI here and added a note

OP here, I added a sample PDF output in the project assets and put screenshots in the ReadMe. The text is selectable after rehydration. would this work with your app?

definitely the best alternative in terms of DX

They sent you swag in the mail? How did that work?

This is cute, but in all seriousness it would be much more effective to shout "I'm a winner"

Research:

- https://pmc.ncbi.nlm.nih.gov/articles/PMC3354773/ – Low self-esteem + rejection hurts self-control

- https://selfdeterminationtheory.org/SDT/documents/2007_Power... – Self-criticism predicts less goal progress

- https://pmc.ncbi.nlm.nih.gov/articles/PMC9916102/ – Social exclusion slows inhibitory control

- https://www.frontiersin.org/articles/10.3389/fpsyg.2023.1191... – Low teen self-esteem → poorer self-control

- https://pmc.ncbi.nlm.nih.gov/articles/PMC8768475/ – Meta-analysis links shame to regulation drops

- https://pubmed.ncbi.nlm.nih.gov/28810473/ – Self-compassion boosts self-regulation

- https://www.researchgate.net/publication/312138882_Self-Cont... – Ego threats deplete self-control resources

- https://pubmed.ncbi.nlm.nih.gov/21632968/ – Self-criticism tied to worse goal progress

- https://www.nature.com/articles/s41598-025-96476-8 – Low self-respect → low self-control → problems

Remember to be kind to yourself.

Was an interesting experience travelling to Italy and suddenly starting to get cookie banners on sites I visit daily that normally don't have

Interesting. Did they explain why?

Fully agree. The physics of solar panels on cars just doesn't work. It's bizarre that this is actively pursued by startups and concept cars from large manufacturers when it takes just quick back-of-the-napkin math to see.

A car has about 5 m^2 of flat space on the roof/hood/trunk so that's the maximum surface area that can capture solar energy at any given time.

The total energy to hit the area is 1000 w/m^2.

The panels can't rotate to track the sun so the effective area is the cosine of the angle. So you end up with about half the amount of effective sunlight hours as the actual daylight hours. So in summer you get about 6 hours of effective sunlight.

Good panels in real world conditions can give you 22% efficiency.

So in optimal conditions you get: 5 * 1000 * 6 * 0.22 = 6.6 kwh

That will reflect your best days. It can be dramatically less if it's cloudy, overcast, winter, far from the equator, car is dirty, parked in shade, etc.

6.6 kwh is about one tenth of the battery in my Hyundai Kona EV. With very conservative highway driving, 6.6 kwh can get about 40km of range and about 50km in city driving. It's what I get from plugging into my home charger for 30 min and what you get from a fast charger in about 3 minutes.

So besides some very niche uses, there's no sense in massively increasing the cost and complexity of a car by installing solar panels. Far better to put the panel on the roof of parking and just plug in for a few minutes while you park.

I was wondering the same and found these related papers:

https://arxiv.org/pdf/2309.08561 https://arxiv.org/pdf/2406.02649

I haven't really dug in yet but from a quick skim, it looks promising. They show a big improvement over Whisper on a medical dataset (F1 increased from 80.5% to 96.58%).

The inference time for the keyword detection is about 10ms. If it scales linearly with additional keywords you could potentially scale to hundreds or thousands of keywords but it really depends on how sensitive you are to latency. For real-time with large vocabularies my guess is you might still want to fine-tune.

Idk, 7 articles over 10 years isn't very strong evidence of a raging debate

the noise is pretty hard to stomach

Take a look at this one, might be a fit depending on your use case https://www.eself.ai/

It doesn't really matter what nationality or ethnicity you are, but if you communicate with the model in Chinese you might get better results from this model.

Then again, if they've misrepresented the strength of the model overall, there might be some other shenanigans with their results. The fact that their results show their model is worse than GPT-4.5 on 2 Chinese language benchmarks, while it's so much stronger on some of the others, is a bit weird.

Surprised nobody has pointed this out yet — this is not a GPT 4.5 level model.

The source for this claim is apparently a chart in the second tweet in the thread, which compares ERNIE-4.5 to GPT-4.5 across 15 benchmarks and shows that ERNIE-4.5 scores an average of 79.6 vs 79.14 for GPT-4.5.

The problem is that the benchmarks they included in the average are cherry-picked.

They included benchmarks on 6 Chinese language datasets (C-Eval, CMMLU, Chinese SimpleQA, CNMO2024, CMath, and CLUEWSC) along with many of the standard datasets that all of the labs report results for. On 4 of these Chinese benchmarks, ERNIE-4.5 outperforms GPT-4.5 by a big margin, which skews the whole average.

This is not how results are normally reported and (together with the name) seems like a deliberate attempt to misrepresent how strong the model is.

Bottom line, ERNIE-4.5 is substantially worse than GPT-4.5 on most of the difficult benchmarks, matches GPT-4.5 and other top models on saturated benchmarks, and is better only on (some) Chinese datasets.

Also worse than o1-mini on agentic tasks (page 29), big drop from 39% -> 27%

Interesting. I wonder if this is related to the model architecture and attention mechanism.

The author seems to be implying it could be: "Even a single mention of ‘code enhancement suggestions’ in our instructions seemed to hijack the model’s attention"

https://archive.md/zOrRr

Oracles are hypothetical constructs so it's a bit difficult to define concretely. They are used in thought experiments within mathematical proofs. From what I remember from my Comp Sci studies, they were sometimes used in negative proofs — you imagine you had an oracle who could magically tell you the correct answer, and this eventually leads to a logical contradiction

Inflation doesn't explain reducing the included traffic from 20TB to 1TB while simultaneously increasing prices. This is a much more dramatic change than what inflation would justify.

Haven't been able to find a link. Seems like it was only distributed by email — which isn't great communication IMO

Preprint: https://arxiv.org/abs/2309.17368

Seminar on the topic: https://www.youtube.com/live/w7GHPmfCzZs

Repo: https://github.com/qiskit-community/ml-qem

From today's ChatGPT search announcement: "The search model is a fine-tuned version of GPT-4o, post-trained using novel synthetic data generation techniques, including distilling outputs from OpenAI o1-preview."

"The search model is a fine-tuned version of GPT-4o, post-trained using novel synthetic data generation techniques, including distilling outputs from OpenAI o1-preview."

More info on model distillation: https://openai.com/index/api-model-distillation/

This reminds me of the PR Agent open source tool: https://github.com/Codium-ai/pr-agent

I've found the code walkthroughs very useful

When Meta releases the quantized 70B it will give another > 2X speedup with similar accuracy: https://ai.meta.com/blog/meta-llama-quantized-lightweight-mo...

I've found PR Agent pretty useful: https://github.com/Codium-ai/pr-agent

/describe gives a good walkthrough of the changes — like having someone familiar with the code give you an overview of the PR

/review gives an estimate of the effort to review which is super helpful in letting me know what I'm getting myself into

https://archive.md/DyW7t