crackalamoo

I love this, easy intuitive explanation

I find it works as focus when the tasks are all related/in the same repo. Not the same as single-task flow state of old style coding, but also still a kind of flow

Interesting, how common is this vs just unit testing? How do you avoid formally verifying something against a spec that could subtly fail in production?

This is super fun! Very elegant idea

Has anyone done this for larger neural nets? Is there a way to extract some kind of pattern or is the image just noise no matter how you construct it? I'd be curious to see something like that

Isn't this kind of the same as an AI copilot, just with higher autonomy?

I think the limiting factor is that the AI still isn't good enough to be fully autonomous, so it needs your input. That's why it's still in copilot form

This seems like a solvable engineering problem. For example, you could have a lightweight subagent with its own context for reading the skills and determining which to use

I also use pure HTML and CSS (and a touch of hand-written JavaScript)

I'm a little skeptical of AEO. What's the point if AI users just ask the LLM to retrieve the information and never visit your blog? I almost never click the links ChatGPT gives me

Maybe it makes sense if you're selling a product or service, but I don't see the appeal of AEO as the new SEO. Maybe I'm missing something?

My two cents: if you're not doing anything too political or controversial, it's fine or even beneficial to mix in the occasional personal essay with the professional.

After all, many of your readers are also human beings with lives, maybe even lives similar to yours based on your professional content. (The rest of your readers are LLMs.) Your readers might appreciate your perspectives on random life things or just getting to see what their favorite blogger is up to.

I make heavy use of the "temporary chat" feature on ChatGPT. It's great whenever I need a fresh context or need to iteratively refine a prompt, and I can use the regular chat when I want it to have memory.

Granted, this isn't the best UX because I can't create a fresh context chat without making it temporary. But I'd say it allows enough choice that overall having the memory feature is a big plus.

Grade inflation is common at many schools. And many difficult technical classes grade on a curve, sometimes to the point where you can get an A with an 85%.

But yeah, I still don't see how an 85% average would be a 4.0.

Why is dependence on LangChain an issue?

Not that I disagree necessarily, just wondering if there's a consensus that LangChain is too opinionated/bloated/whatever for real industry applications, or if there's some other reason.

The recent Apple paper seemed pretty flawed. Anthropic/Open Philanthropy did a rebuttal paper, The Illusion of the Illusion of Thinking.

Do they reason? No. (Before you complain, please first define reason).

Defining reasoning is the problem. No, they don't reason in the same way as humans. But they seem to be able to go through reasoning steps in some important way. It's like asking if submarines swim.

Not sure how tools like Cursor work under the hood, but this seems like an easy model context engineering problem to fix.

I agree. And if human civilization survives, your concerns about energy and resources will be only short term on the scale of civilization, especially as we make models more efficient.

The human brain uses just 20 watts of power, so it seems to me like it is possible to reach human-level intelligence in principle by using much greater power and less of the evolutionary refinement over billions of years that the brain has.

I agree humans only rarely step outside the circle, but I do have this intuition that some people sometimes do, whereas LLMs never do. This distinction seems important over long time horizons when thinking about LLM vs human work.

But I can't quite articulate why I believe LLMs never step outside the circle, because they are seeded with some random noise via temperature. I could just be wrong.

Yeah, unfortunately I feel like despite all the advances in Unicode tech, my modern terminal (MacOS) still bugs out badly with emojis and certain special characters.

I'm not sure how/when codepoints matter for wcwidth: my terminal handles many characters with more than one codepoint in UTF-8, like é and even Arabic characters, just fine.

Yes, 100% this. And even more so for reasoning models, which have a different kind of RL workflow based on reasoning tokens. I expect to see research labs come out with more ways to use RL with LLMs in the future, especially for coding.

I feel it is quite important to dispel this idea given how widespread it is, even though it does gesture at the truth of how LLMs work in a way that's convenient for laypeople.

https://www.harysdalvi.com/blog/llms-dont-predict-next-word/

I use /foo/bar/ with the trailing slash because it works better with relative URLs for resources like images. I could also use /foo/bar/index.html but I find the former to be cleaner

I don't think this article is paywalled? You can just click "I'll do it later" on the banner

To some extent, AI needs less extensive teaching than the problem-solving skills taught in school because it works in natural language anyways and builds off whatever else you know. You can easily talk to an AI about CS if you know CS, or even use it for coding, but not so much if you only know "AI".

It's not that learning AI isn't important, it's just that school isn't necessarily the place for it: AI is always changing, and can be learned relatively easily online, while properly learning something like linear algebra or operating systems online is pretty unlikely except for a very motivated student.

I made a custom GPT of myself using my blog. It understood who I am, but wasn't able to replicate me very well, and mostly sounded like generic ChatGPT with some added interest in my interests.

I would imagine fine tuning with enough data would be different though

I think standardizing scientific terms and the like is a very minor concern when it comes to preserving endangered languages. Most science is done in English anyways, and that's unlikely to change.

I think the commenter you're replying to might have been referring to pictograms like many Chinese characters, not icons that you have to figure out. Or maybe not

Yeah, that's the ambition.

More generally, ML is good at approximating many NP-hard problems efficiently, so I wonder if it will be a more practical alternative to quantum computing for things like molecular simulation.

Can machine learning accomplish the same? See e.g. Microsoft MatterSim

https://www.microsoft.com/en-us/research/blog/mattersim-a-de...

You can do both. Preserving the corpus and building the LLM probably gives the best chance for future generations.

But of those 126,301 people who have edited in the last 30 days, some of them have edited more than one article. In fact, some have made up to millions of edits (lifetime), which disproportionately increases the total. At least 5000 people have edited more than 24,000 times.

https://en.wikipedia.org/wiki/Wikipedia:List_of_Wikipedians_...

(And also: each editor has (approximately) 2 eyes :) )

As far as I can tell, OpenAI really did invent RLHF in its current form. https://arxiv.org/pdf/1909.08593

It probably doesn't matter much either way, but I use vanilla HTML/JS