mnk47

In my experience, the model's performance in silly tasks like these is usually (not always) correlated with its performance in other areas except tool use/agent stuff.

You can just use the official Claude Code, OpenAI Codex, and Gemini extensions on VS Code. You get diffs just like in Cursor now. The performance of these models can vary wildly depending on the agent harness they're on.

The official tools won't necessarily give you the best performance, but they're a safer bet for now. This is merely anecdotal as I haven't bothered to check rigorously, but I and others online have found that GPT-5-Codex is worse in Cursor than in the official CLI/extension/web UI.

Go is still going strong after 15 years. Dart, the language of Flutter, is 13 years old.

In Obsidian, open the daily file amd copy the contents from yesterday

What's the point of this? Isn't it easier to just keep reusing the same note?

Did you read the article? All it basically says is that OpenAI faced struggles this past year -- specifically with GPT-5 aka Orion. And now they have o3, and other labs have made huge strides. So, yes, show me AI progress is slowing down!

So LLMs finally hit the wall

Not really. Throwing a bunch of unfiltered garbage at the pretraining dataset, throwing in RLHF of questionable quality during post-training, and other current hacks - none of that was expected to last forever. There is so much low-hanging fruit that OpenAI left untouched and I'm sure they're still experimenting with the best pre-training and post-training setups.

One thing researchers are seeing is resistance to post-training alignment in larger models, but that's almost the opposite of a wall, they're figuring it out as well.

Now someone has to have a new idea

OpenAI already has a few, namely the o* series in which they discovered a way to bake Chain of Thought into the model via RL. Now we have reasoning models that destroy benchmarks that they previously couldn't touch.

Anthropic has a post-training technique, RLAIF, which supplants RLHF,and it works amazingly well. Combined with countless other tricks we don't know about in their training pipeline, they've managed to squeeze so much performance out of Sonnet 3.5 for general tasks.

Gemini is showing a lot of promise with their new Flash 2.0 and Flash 2.0-Thinking models. They're the first models to beat Sonnet at many benchmarks since April. The new Gemini Pro (or Ultra? whatever they call it now) is probably coming out in January.

The current level of LLM would be far more useful if someone could get a conservative confidence metric out of the internals of the model. This technology desperately needs to output "Don't know" or "Not sure about this, but ..." when appropriate.

You would probably enjoy this talk [0], it's by an independent researcher who IIRC is a former employee of Deepmind or some other lab. They're exploring this exact idea. It's actually not hard to tell when a model is "confused" (just look at the probability distribution of likely tokens), the challenge is in steering the model to either get back to the right track or give up and say "you know what, idk"

[0] https://www.youtube.com/watch?v=4toIHSsZs1c

Then there is the matter of actually defining general intelligence. It may also be the definition of consciousness, or at least require it. But currently, there is no mutually agreed upon definition of "general intelligence".

Here lies the problem. We should have a rule that any time we discuss AGI, we preface with the arbitrary definition that we choose to operate on. Otherwise, these discussions will inevitably devolve into people talking past each other, because everyone has a different default definition of AGI, even within the SF AI scene.

If you ask Yann LeCun, he'll say that no LLM system is even close to being generally intelligent, and that the best LLMs are still dumber than a cat.

If you ask Sam Altman, he'll say that AGI = an AI system that can perform any task as well as the average human or better.

If you ask Dario Amodei, he'll say that he doesn't like that term, mostly because by his original definition AGI is already here, since AGI = AI that is meant to do any general task, as opposed to specialized AI (e.g. AlphaGo).

Announcement on X: https://x.com/FishAudio/status/1853655232779313408

Demo: https://huggingface.co/spaces/fishaudio/fish-agent

Repo: https://github.com/fishaudio/fish-speech

Weights: https://huggingface.co/fishaudio/fish-agent-v0.1-3b

Repo: https://github.com/Standard-Intelligence/hertz-dev

Sam Altman just replied: https://x.com/sama/status/1849661093083480123

fake news out of control

it is absurd to expect something to keep increasing forever just because it did increase for a short duration previously

The problem isn't that it stopped increasing. It's that it's steadily decreasing now. I made a comment about this recently with sources: https://news.ycombinator.com/item?id=41844739

Not convinced by the blog author's theories, but it's a real problem worth discussing.

Am I misremembering or is this an exact plot point of Pluto (the manga/anime)?

yep, we want the same thing here. we want to be minorities in our own cities

where in Western Europe is this currently happening? Which cities?

I wish trends like the Flynn effect (rise in IQ in most of the world throughout the 20th century, including China [0], Japan and Korea [1]) had more concrete answers by now. AFAIK we still don't have any answers beyond conjectures. To me, having a better understanding of this is vital, because IQ is currently in decline in many developed countries, even when you account for immigration and demographic/race changes. Something in our environment is causing a reverse Flynn effect [2] [3], we don't know what it is, and not enough people seem to care.

[0] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3834612/

[1] https://www.sciencedirect.com/science/article/abs/pii/S01918...

[2] https://www.nbcnews.com/think/opinion/iq-rates-are-dropping-...

[3] https://www.pnas.org/doi/pdf/10.1073/pnas.1718793115

By Meta FAIR, UC Berkeley and NYU

X thread by one of the authors: https://x.com/jaseweston/status/1846011492245672043

LLMs do not generate new content, they just shuffle old content together in new ways

You can say the same thing about most technical books. They're quite often little more than a more digestible summary of what you get in docs and reference manuals, with some toy examples.

edit: They've added a cookbook article at https://cookbook.openai.com/examples/orchestrating_agents

It's MIT licensed.

The latest of this fad is o1-preview

Not for programming it's not. It's confusing, but o1-preview is currently pretty broken for many tasks, or in the words of Sam Altman [0], "deeply flawed". o1-mini is the recommended model [1] for programming exercises and is superior to o1-preview in OpenAI's programming benchmarks [2].

[0] https://analyticsindiamag.com/ai-news-updates/sam-altman-say...

[1] https://help.openai.com/en/articles/9824965-using-openai-o1-...

[2] https://openai.com/index/openai-o1-mini-advancing-cost-effic...

Personally, I'm sticking with Claude Sonnet 3.5 until more people figure out how to use these new models effectively. OpenAI employees said on launch day that traditional prompting techniques might not work with this one so we'll just have to experiment.

Any tips on prompts for o1? I'm struggling to figure out how much scope/detail/context I should include in my prompts.

Sincerity? I don't know. He seems to really believe in AGI and the singularity and all that, but everything else seems to be lie after lie. This article [0] on the New York Magazine paints him as an incredibly insencere manipulator, which lines up with accusations made by Helen Toner and others. Things he repeatedly says (not caring about money, caring a lot about safety) are often at odds with his actions.

https://nymag.com/intelligencer/article/sam-altman-artificia...

Starting to wonder why this is so common in LLM discussions at HN.

Someone says "X is the model that really impressive. Y is good too."

Then someone responds "What?! I just used Z and it was terrible!"

I see this at least once in practically every AI thread

To you anyone reading this who can relate and is happy with their career: what do you do? I don't want to be a magpie developer [0] for the rest of my life. I feel that actual, focused specializations are more valuable now, especially in the ChatGPT era. You have a jack of all trades in your pocket, so why hire me?

I'm like you but a bit worse because I almost never finish projects unless there's money on the line. I'm finding that medications helps a lot with this, but if I take them late I can't sleep (even IR) so I can't rely on them every day (I struggle wih unrelated chronic pain that affects my sleep schedule, along with being a night owl).

I've taken so many career assessments and nearly all of them return fields that are completely unrealistic for me to pursue: game designer, musician, filmmaker, writer, journalist... Might as well tell me to become a youtuber, podcaster or NBA superstar.

[0] https://blog.codinghorror.com/the-magpie-developer/

The founder was featured in the o1 release promotional videos. Looks like he had an early access deal with OpenAI and now he's working on upgrading Devin

But it's just that

It's not just that if you use a lora, like this: https://www.reddit.com/r/StableDiffusion/comments/1fe7d7o/am...

or this:

https://www.reddit.com/r/StableDiffusion/comments/1eywnv8/re...

Often slang has originated from marginalised groups such as the queer community or POC (People of Colour).

And that hasn't changed, that is still exactly the case. When it's not 4chan lingo (which mostly stays in text/image form), it's AAVE. Not sure why it's always "Gen Z" or teenage girls getting credit for this.

Just look at their list of phrases that are "in":

Phrases that are "in"

Bet — 'I'm down'. For example, if someone asks you to go out for dinner, you say 'bet' You devoured/ate — said in response to someone succeeding with style Full send — not caring, just going for it Slaps — If something slaps, it's really good (normally in a food context) Ceebs — a variation of CBF [can't be f*ked] Ick — a sudden feeling of grossness, often about a person's behaviour or mannerisms

out of these, only "Ceebs" (sounds like the a twist on "can't be arsed") and "Ick" aren't AAVE.

And the list of phrases that are "out":

No cap — the truth Lit — something is great, but now considered very "2017-2018" Fam — used in reference to friends Slay — used when someone performs well, but considered dated and you'd only use the term ironically

These are all AAVE.

So the Ballmer Peak is real!

https://xkcd.com/323/

And that is with gpt-4o, the latest flagship thingy... I am sure Claude users will now tell me their pet is more perfect.

It's definitely much better than GPT-4o for most programming tasks, yes. And that's just the free version.

Music producers have been doing it with sample-based music spanning hundreds of genres since the 80s. So yes, you can.

There's very little tech that somebody is going to hire you for knowing. It's a tool like many others.

I guess this must be true on places like SF since I see this so often on HN, but almost every single job listing I've seen strictly requires knowledge of a specific tech stack, with the exception of a few internship programs.

I like Rust, but to be honest, your comment just supports my belief that it really isn't worth the investment for most web devs and non-C++ devs (and even in C++, it's an awful investment for gamedev, for example).

There is a long list of languages that are more structured than Python/PHP/JavaScript and provide enough safety guarantees for most use cases, with the benefit of a GC. Languages that are easy to learn, cheap to hire for, productive, and great for teams of all sizes.

I'm glad it's working out for you, though.