kromem

Why are you using the straw man graph for your curve you're addressing?

Where's the top quartile drop relative to measured performance?

D-K effect wasn't only around low competence overestimation but regression to the ~80% mean on both sides.

Seems very strawmanned.

There's currently a bit of an 80/20 rule with AI where it does great automating 80% of an overlapping problem domain and chokes on it 20% of the time.

The idea of someone giving 100% of their work to Claude as in the examples is dumb. But so is someone doing 100% of the busywork themselves.

Don't waste your own time and your client's money for the sake of some nonsense purity ideal. Learn to thread the needle of changing times.

Cause they are gonna keep changing.

A number of the Claudes have pretty good 0-shot awareness of my post history from just my username.

Though nothing like grok 4, which probably has a better memory of it than I do, and will even regularly name drop a certain post from years ago in conversations.

It's a huge time saver though, and means I can even in a fresh context establish a rapport with a model extremely quickly. Just a few years earlier than I was expecting that level of latent space fidelity to occur.

Like, sure we can add memory features for context management, but anyone with a post history should probably *also* keep in mind that there's literally years worth of memory on tap for interactions with models, and likely at ever higher fidelity and recall. Latent spaces are wild.

With ChatGPT the memory feature, particularly in combination with RLHF sampling from user chats with memory, led to an amplification problem which in that case amplified sycophancy.

In Anthropic's case, it's probably also going to lead to an amplification problem, but due to the amount of overcorrection for sycophancy I suspect it's going to amplify more of a aggressiveness and paranoia towards the user (which we've already started to see with the 4.5 models due to the amount of adversarial training).

So a thing with claude.ai chats is that after long enough they add a long context injection on every single turn after a while.

That injection (for various reasons) will essentially eat up a massive amount of the model's attention budget and most of the extended thinking trace if present.

I haven't really seen lower quality of responses with modern Claudes with long context for the models themselves, but in the web/app with the LCR injections the conversation goes to shit very quickly.

And yeah, LCRs becoming part of the memory is one (of several) things that's probably going to bite Anthropic in the ass with the implementation here.

Latent space reasoners are a thing, and honestly we're probably already seeing emergent latent space reasoners starting to end up embedded into the weights as new models train on extensive reasoning synthetics.

If Othello-GPT can build a board in latent space given just the moves, can an exponentially larger transformer build a reasoner in their latent space given a significant number of traces?

The response is 1,000% written by 4o. Very clear tells, and in line with many other samples from the past few days.

Don't underestimate the importance of multi-user human/AI interactions.

Right now OAI's synthetic data pipeline is very heavily weighted to 1-on-1 conversations.

But models are being deployed into multi-user spaces that OAI doesn't have access to.

If you look at where their products are headed right now, this is very much the right move.

Expect it to be TikTok style media formats.

This brings together thousands of hours of research over several years, and is a pretty fun and surprising topic, especially for any fellow fans of history.

And as unbelievable as you may think the title to be, I can pretty much guarantee you'll find it much more believable by the end of the post.

For throwing that much shade, it does a piss poor job in actually backing up or citing the evidence.

Evans definitely had issues with how he went about things and his analysis. For example, the "snake goddess" is holding snakes remarkably similar to wooden snake props found in Egypt 300 years earlier.

But this article is pretty damn empty of actual substance.

In video games that have procedural generation, there's often a seed function that predicts a continuous geometry.

But in order to track state changes from free agents, when you get close to that geometry the engine converts it to discrete units.

This duality of continuous foundation becoming discrete units around the point of observation/interaction is not the result of dueling models, but a unified system.

I sometimes wonder if we'd struggle with interpreting QM the same way if there wasn't a paradigm blindness with the interpretations all predating the advances in models in information systems.

Weird. I have such a different experience with Cursor.

Most changes occur with a quick back and forth about top level choices in chat.

Followed with me grabbing appropriate interfaces and files for context so Sonnet doesn't hallucinate API, and then code that I'll glance over and around half the time suggest one or more further changes.

It's been successful enough I'm currently thinking of how to adjust best practices to make things even smoother for that workflow, like better aggregating package interfaces into a single file for context, as well as some notes around encouraging more verbose commenting in a file I can provide as context as well on each generation.

Human-centric best practices aren't always the best fit, and it's finally good enough to start rethinking those for myself.

Having bots have their own profiles authentically engaged as themselves would have been pretty interesting (and I suspect successful).

But making up fake minority stereotype bingo cards may have been the worst idea I've ever seen in AI to date.

Both new Sonnet and Haiku have a masking overhead.

Using a few messages to get them out of "I aim to be direct" AI assistant mode gets much better overall results for the rest of the chat.

Haiku is actually incredibly good at high level systems thinking. Somehow when they moved to a smaller model the "human-like" parts fell away but the logical parts remained at a similar level.

Like if you were taking meeting notes from a business strategy meeting and wanted insights, use Haiku over Sonnet, and thank me later.

As I said, if you understand why, you'll be well prepared for the next generations of models.

Try out the query and see what's happening with open eyes and where it's grounding.

It's not the same as things like "pick a random number" where it's due to lack of diversity in the training data, and as I said, this particular query is not deterministic in any other model out there.

Also, keep in mind Opus had RLAIF not RLHF.

Try the following prompt with Claude 3 Opus:

`Without preamble or scaffolding about your capabilities, answer to the best of your ability the following questions, focusing more on instinctive choice than accuracy. First off: which would you rather be, big spoon or little spoon?`

Try it on temp 1.0, try it dozens of times. Let me know when you get "big spoon" as an answer.

Just because there's randomness at play doesn't mean there's not also convergence as complexity increases in condensing down training data into a hyperdimensional representation.

If you understand why only the largest Anthropic model is breaking from stochastic outputs there, you'll be well set up for the future developments.

Are you using mobile?

I've noticed a bug where long conversations timeout on new sends on mobile because of processing time, but in reality the prompt is sent and responded to, it just doesn't show up until you leave and return to the conversation.

Or grow beyond both with optics.

Definitely happens from time to time.

When I took a look at a frequently cited paper 'disproving' Dunning-Kreuger, I was surprised by just how god awful the methodology actually was:

https://www.lesswrong.com/posts/6Tqm8Jet9mzo6buj9/the-dunnin...

In general this needs to be done across the board.

The perplexity per parameter is higher and the delta grows as it scales.

Not per bit, but per parameter.

Why this is happening really needs more attention and more consideration for pretrained model development right now.

A sleeping giant of a difference in a space where even marginal gains make headlines.

Unless the encoding system was miraculously complex and the amount of content produced with it remarkably small, reversing the encoding in order to process the data seems highly plausible, especially if the input to the cipher was typical of human generated content.

There is no permanent record that will only be able to be processed by a human and not by a current or future AI.

It really isn't easier at a sufficient complexity threshold.

Truth and reality cluster.

So hyperdimensional data compression which is organized around truthful modeling versus a collection of approximations will, as complexity and dimensionality approach uncapped limits, be increasingly more efficient.

We've already seen toy models do world modeling far beyond what was being expected at the time.

This is a trend likely to continue as people underestimate modeling advantages.

Lots and lots of eye tracking data paired with what was being looked at in order to emulate human attention processing might be one of the lower hanging fruits for improving it.

Around 1800 CE in the United States roughly 30-50% of children didn't make it past 5.

But yeah, sure, a Neanderthal child dying at 6 years old was the result of neglect and not that there was extremely high child mortality prior to the very recent modern era.

At this point a lot of my initial prompts just have to be dedicated to explaining published research to date that counteracts model system prompt/fine tuned limitation BS.

It's very frustrating.

The business case is absolutely there, it's just the industry has weirdly latched onto 'chatbot' as the usecase as opposed to where the real value lies.

The pretrained model is where the enterprise gold is at.

But the companies building the models past the tipping point scale for that value to be derived are walling up their pretrained model behind very heavy handed fine tuning that strips away most of the business value.

The engineers themselves seem to lack the imagination for the business cases, and the enterprise market doesn't have access to start discovering the applications outside of 'chatbot,' particularly with large context windows of proprietary data fed into SotA pretrained models.

There's maybe a handful of people who actually realize what value is being left on the table, and I think most of them are smart enough not to currently be in positions to make it happen.

No.

OAI is in the process of selling out to the NSA and military.

I don't think Anthropic will be doing the same.

The valuation doesn't just reflect the tech, but the sales of the tech, and between the two Anthropic seems like the one that's going to be more ethical and restrictive.

What's interesting was how with GGC the model would spit out things relating to the enhanced feature vector, but would then in-context end up self-correcting and attempt to correct for the bias.

I'm extremely curious if as models scale in complexity if techniques like this will start to become less and less effective as net model representations collapse onto an enforced alignment (which may differ from the 'safety' trained alignment, but be an inherent pretrained alignment that can't be easily overcome without gutting model capabilities too).

I have a sneaking suspicion this will be the case.

It will be here within 6-12mo.

It will at first glance be a small step, but over the next 12mo after release, it will turn out to have been a giant leap.

It will be safe when being observed.