Stuff we figured out about AI in 2023

https://simonwillison.net/2023/Dec/31/ai-in-2023/

by simonw • 3 years ago

210 132 3 years ago

jbandela1• 3 years ago

I think what is also interesting is what LLMs help us learn about ourselves.

I think we understand human intelligence a bit better because of LLMs, and I think people have been surprised how far you could get just by processing text.

I think (just a guess relying on intuition and my very shallow survey) that human intelligence is multilayered where we have something at least vaguely analogous to a LLM combined with something else that knows the physics of the world as well as something else that is able to do logic and symbolic processing.

But this is exciting to see the power of relatively simple neural networks when they are trained with a giant corpus.

MeImCounting• 3 years ago

I really believe this to be the case. Its remarkable what language alone can do but its not the be all end all of intelligence.

bormaj• 3 years ago

Did the prevalence of LLMs shed any light on what intelligence is vs what it is not?

paganel• 3 years ago

Not that I personally know of, but it did create (or empowered, because in a way it was already there) a new sort of anthropomorphization, of which this article is partially guilty of itself (the part labeled "LLMs are really smart, and also really, really dumb"). Some of us now think they know more about human intelligence as a result of that anthropomorphization.

growingkittens4• 3 years ago

I find something lacking from many AI discussions because my experience with having a brain is radically different from average. Mine is damaged by a childhood brain injury, which wasn't diagnosed until I was 30.

Another thing about the injury is that my lower order thinking skills seem to be damaged, such as sequencing or anything based on working memory. However, my higher order thinking skills are intact - I think they've been compensating for the broken skills this whole time, albeit with much difficulty.

My thoughts on LLMs from a previous comment:

I think in pictures; I remember information with emotions, music, movement, and basically anywhere my brain could stuff information. Words are ephemeral, I often forget the beginning of a sentence by the time I reach the end.
The human brain is so much more than language.

Also:

There are many specialists and not many generalists. Generalists are needed to tie together the multiple domains required to emulate the human brain.

layer84• 3 years ago

Your quote doesn’t sound very different from what undamaged brains do (based on my personal n = 1 experience).

nicklecompte6• 3 years ago

David Lynch has some insights on this[1]:

There is a theory that Lynch himself doesn’t always know what is going on in his stories. He shoots this down. “I need to know for myself what things mean and what’s going on. Sometimes I get ideas, and I don’t know exactly what they mean. So I think about it, and try to figure it out, so I have an answer for myself.”
Audiences, however, must do their own figuring out. “I don’t ever explain it. Because it’s not a word thing. It would reduce it, make it smaller.” These days he rarely gives interviews, not even during the hugely hyped return of Twin Peaks last year – a show that is still debated as either the best or worst TV of 2017. “When you finish anything, people want you to then talk about it. And I think it’s almost like a crime,” he explains. “A film or a painting – each thing is its own sort of language and it’s not right to try to say the same thing in words. The words are not there. The language of film, cinema, is the language it was put into, and the English language – it’s not going to translate. It’s going to lose.”

[1] https://www.theguardian.com/film/2018/jun/23/david-lynch-got...

Terretta• 3 years ago

By contrast, particularly observing a range of adults from colleagues to seniors with early onset dementia, I'm increasingly sure a large portion of discourse is of the "LLM continuation" variety.

Everyone knows that colleague (I am one myself) where a particular topic will kick off a continuation that follows a well worn path albeit with a bit higher temperature than zero.

With early onset dementia where working memory and self-suppression are removed, you can unspool the same commentary as often as you like by teeing up the same context to trigger it.

See also these layers, nothing to do with LLM -- Attention, Language, Memory, Executive Function: https://academic.oup.com/view-large/figure/198261184/awz311f...

Or maybe quite a lot to do with it.

rorroe53• 3 years ago

Human intelligence is definitely multilayered, and different parts of brain are known to be responsible for different types of processing.

Personally I was born with a disability that makes me suck at motor skills, spatial learning and understanding most forms of mathematics, but somehow I'm still decent at programming, and significantly better than average at reading and writing - all of which LLMs can do quite well compared to its mathematics and logic skills.

archagon• 3 years ago

I’ve noticed myself looking at art recently with LLMs in mind. Is this something that an AI could spit out, or is there evidence of a human touch? And when I create art, how do I ensure it doesn’t end up looking like something out of Midjourney?

I think we will learn a lot about the creative process from AI, and I think human-created art will get a lot more interesting (and weird) as a result.

globalnode• 3 years ago

one thing a friend was suggesting to me was that he thinks the difference is that human intelligence is analogue while computing intelligence is digital. there are nuances in our thinking and language that current digital systems cannot compete with until the become fine grained enough to appear analogue.

wegfawefgawefg2• 3 years ago

Ive heard this before. Im not trying to be offensive here, but it just doesn't make any sense.

It's like being concerned that planes can't fly because they don't flap their wings.

If you want to escape this state of ignorance you're gonna have to implement some neural networks, and learn about neurology.

At the very least consider that double precision floating points are extremely insanely precise. Definitely orders of magnitude more precise than real biological neurons can measure over the noise floor of activity in a brain.

ravetcofx• 3 years ago

The analogue/digital isn't the main distinction I think, it's a distinction of highly optimized higher order multi-modality. Our brains have Nth degree multi modality input and output that run on a few watts (calories) while nueral networks & LLMs so far take kilowatts or more. A bird can fly 10s-100s of km with flapping with a few watts, an airplane, megawatts for fixed wing flight of the same distance. Evolution was an unintentional search algorithm over hundreds of millions of years to find resilient, optimized, and efficient-ish systems. Our toys are so new on that search tree and are thus not the most complex, resilient or efficient.

gkbrk• 3 years ago

Computers use binary logic and circuits, but with neural networks you can and pretty much always use floating point numbers. That's very analog.

There are around 2^52 doubles between 0 and 1. Surely that's enough to represent any analog signal you might care about.

nicklecompte6• 3 years ago

Turing completeness says this isn't true: any Turing-complete digital system can simulate any physical system, including a human brain. What Turing completeness doesn't address is efficiency - how much energy would it take an artificial neural network to simulate a human brain? If it's "approximately one nuclear reactor" then maybe this will work with some efficiency improvements. If it's "approximately one sun" then it won't work at all.

There is a very good Quanta article about this[1] detailing researchers attempt to simulate a rat cortex neuron with an ANN: it took 700 artificial neurons to simulate a single rat neuron, and that was just simulating the "voltmeter" measurements of the synapses, ignoring any genetic effects. If you included genetic effects[2] I would guess it would take several thousand artificial neurons, if not more. And that's just a rat! Even at the level of individual neurons, primates seem to be more sophisticated than other mammals, with apes more sophisticated than other primates.

So your friend is onto something: biological computers are astonishingly sophisticated compared to 21st-century transistor technology, and this probably has implications for our ability to emulate biological intelligence even on supercomputers. Think of a neural network that was trained to draw physically-sensible spiderweb patterns across a wide range of geometries. Now think about running this neural network in a computer the size of a spider. And that's ignoring the difficult real-world planning an actual spider would have to do to create a physical spiderweb.

[1] https://www.quantamagazine.org/how-computationally-complex-i...

[2] AI is not even close to this level of computational sophistication: https://www.universityofcalifornia.edu/news/biologists-trans...

somewhereoutth3• 3 years ago

No, a Turing complete system can perfectly simulate any other discreet (digital) computation system.

What it cannot do is perfectly simulate any analogue system. By Cantor's diagonal argument, there are vastly more real numbers than natural numbers. Any system that is completely definable in the natural numbers (like digital computers) will not be able to 'reach' the vast majority of real numbers (in case you are concerned about the existence of real numbers, what is the square root of 2? What is the ratio of diameter to circumference?). Throw in some Chaos Theory and you can see how far away any Turing complete system is from a biological brain.

somewhereoutth3• 3 years ago

Your friend is correct. I call it the Cardinality Barrier - Cantor showed that there are incomprehensibly more real numbers than natural (whole) numbers. Any digital system can therefore be only an approximation. It is likely that the secret of intelligence lies beyond any approximation we might make in silicon.

Of course AGI advocates won't like this idea, as it dooms their entire project from the off. It may be that the only realistic way to make a human like brain, is to make a human (which is quite easy actually).

layer84• 3 years ago

You’d have to explain why the analog couldn’t be sufficiently approximated by the digital. Slightly disturbing the brain by electromagnetic waves doesn’t cause it to cease to function, so it’s unclear why the deviation caused by numeric approximation would cause that.

XenophileJKO2• 3 years ago

I think "Gullability" might not be as hard a problem as we think it is.

I suspecte the primary reason it is a problem now is likely because we haven't really created a training method to tune the desired level of compliance. What I mean is we probably don't have any counter-factual examples in the training sets. Where in an interaction we want the model to prefer it's own knowledge.

Given the use with RAG (retrieval augmented generation) and many of the other current use-cases, the preference is for the model to "trust" the system message content more than the training data.

Training with some additional context is likely all that is needed.

simonw6• 3 years ago

If you can train a model that isn't gullible - even when faced with multiple paragraphs of devious text trying to trick it - I'll be very impressed!

And even then, there are gullibility problems that seem unsolvable to me.

If the Supreme Court make a decision that changes how copyright law is applied, and you then tell a model "the Supreme Court just announced that..." - should the model believe you?

If not, how should it "fact check" what you are telling it, especially if you maintain full control over its access to the outside world?

XenophileJKO2• 3 years ago

Funny you should say that. When web access was initally setup on the GPT-4 ChatGPT Pro accounts, I actually had the model fact-check things that were "improbable" based on what it knew from the training data.

It stopped doing that when they started fine-tuning how/when the model should use the internet in the later model revisions.

(Edit to add detail..)

I found the interaction facinating. Basically what happened was I made a statement, but didn't ask about it. I said something like this. "Did you know {something improbable to the model} is true?"

It immediately searched the internet to validate that is was true before it responded. When I asked it why it searched, it said that what I said wasn't likely to be true, so it wanted to make sure it was true.

ben_w3• 3 years ago

Mm.

I'd certainly say that gullibility is a hard problem, especially as humans are also pretty gullible.

What level of gullibility are the current generation of LLMs? The level of a child who believes in Santa, or an adult who believes their country is always right?

nicklecompte6• 3 years ago

Transformer neural networks are not smart enough for this to work. OpenAI's own research shows that LLMs don't care about contradictions in the training data[1]: if your LLM's training data has 700 sentences saying "A is true" and 300 saying "A is false," then if you ask the LLM about A, it has a 7/10 chance of saying "A is true" and a 3/10 chance of saying "A is false."

In general it's easy to prove that GPT-4 (or any other LLM) doesn't actually understand what words mean:

- ask it factual questions in English

- ask it to translate those factual questions to Swahili (it should do so with >99% technical accuracy)

- ask it answer those Swahili-language factual questions in Swahili

- translate the answers back to English

It will do much worse at Swahili than English even though its technical accuracy at translation is almost flawless. GPT-4 has a ton of English-language sentences about the topic "cat," and a handful of Swahili-language sentences about the topic "paka," and understands that the translation of "cat" is "paka." But it has no understanding that the English-language facts about cats are automatically true if they are translated in Swahili.

This is not how humans work! I don't think gullibility will be solved unless we have an LLM that actually understands that words mean something.

[1] It is too early for me to find the papers, but I am summarizing two different things:

1) training data needs to have certain things repeated in order for them to "stick" when prompting the LLM - the more often a fact is in the training data, the more likely it is to repeat that fact when prompted

2) absent RLHF, larger LLMs are more likely to endorse conspiracy theories than smaller LLMs. This is because smaller LLMs are trained on reliable datasets like Wikipedia, whereas larger LLMs start including Reddit, bodybuilder forums, old Geocities pages belonging to weird cults, etc. So smaller LLMs have no sentences in their training data saying "Bush did 9/11," but a larger LLM might have 5% conspiracy theories and therefore a 5% chance of endorsing conspiracy theories.

eurekin• 3 years ago

Thank you for that comment. If you ever come across that article, please share

CuriouslyC2• 3 years ago

The language switch issue could be solved in a whole host of different ways without disposing of the transformer architecture. The deeper issue, can transformers "understand" truth and reason, is more nuanced. A transformer architecture built around (token, accuracy) tuples instead of raw tokens and provided "fact checked" training data could be instructed to make statements that are likely to be true when possible and provide a disclaimer when making statements that may not be true. The reasoning is a little harder to address, but I suspect that if the language transference issue was solved, training a model on a wide variety of synthetic algebra problems (automated proof software could spit out an endless stream) and other deductive chain-of-thought examples would do a lot to help transformers emulate logical processes.

layer84• 3 years ago

I don’t believe this will be solvable without some sort of self-reflection, internal iteration, and/or layered hierarchy.

swyx2• 3 years ago

Instead, it turns out a few hundred lines of Python is genuinely enough to train a basic version!

actually its not just a basic version. Llama 1/2's model.py is 500 lines: https://github.com/facebookresearch/llama/blob/main/llama/mo...

Mistral (is rumored to have) forked llama and is 369 lines: https://github.com/mistralai/mistral-src/blob/main/mistral/m...

and both of these are SOTA open source models.

ganzuul4• 3 years ago

This is an interesting point of fact about what Boltzmann Brains could be.

weregiraffe2• 3 years ago

No it's not. Boltzmann Brains are not platonic forms. You need to conjure an entire physical world to contain a Python interpreter to run your 500 lines of code on.

ben_w3• 3 years ago

Kinda, I guess?

Boltzmann brains are likely to be wrong about all of their beliefs owning to them being created by an endless sequence of random dice rolls that eventually makes atoms that can think, so it's fine if they are silicon chips that think they're wet organic bodies (amongst other things).

A computer with say 2^((175e9)*8) parameters is simpler than a human brain, so more likely to be produced by this process.

But the 500 line file is a red herring.

imjonse5• 3 years ago

This metric from the article is a bit misleading. It's not 500 lines in the same sense as mergesort is 20 lines. The whole of the deep learning stack including nvidia firmware blobs is used by those 500 lines.

You may create a small NN from scratch in 500 lines for training a toy dataset, but nothing to actually train real LLMs unless you use the existing stack. Even Karpathy's no-torch C only 500 lines Llama2 code is inference only.

So it is 500 extra lines compared to what we already had before, and in this sense it indeed is a breakthrough.

Otherwise you could also say it's 10 lines of fastAI/lightning style code where almost everything is hidden by the APIs.

mmoskal• 3 years ago

All the nvidia/torch stuff is "only" to make it run two orders of magnitude faster or so. For the actual principles behind LLMS, 500 lines seems quite accurate.

How long would it take to explain what an LLMs is doing to someone from 1800s? It is remarkably simple.

imjonse5• 3 years ago

And also this assumes base models. The breakthrough where the instruct/chat models which again are much more code than 500 lines of plain training.

simonw6• 3 years ago

Sure - there are indeed millions of lines of code if you include the operating system, the Python interpreter, PyTorch, CUDA etc.

But... you don't have to write that code. A research group constructing a brand new LLM really does only need a few hundred lines of code for the core training and inference. They need to spend their time on the data instead.

That's what makes LLMs "easy" to build. You don't need to write millions of lines of custom code to build an "AI", which I think is pretty surprising.

miki123211• 3 years ago

FOr inference, less than 1KLOC of pure, dependency-free C is enough (if you include the tokenizer and command line parsing)[1]. This was a non-obvious fact for me, in principle, you could run a modern LLM 20 years ago with just 1000 lines of code, assuming you're fine with things potentially taking days to run of course.

Training wouldn't be that much harder, Micrograd[2] is 200LOC of pure Python, 1000 lines would probably be enough for training an (extremely slow) LLM. By "extremely slow", I mean that a training run that normally takes hours could probably take dozens of years, but the results would, in principle, be the same.

If you were writing in C instead of Python and used something like Llama CPP's optimization tricks, you could probably get somewhat acceptable training performance in 2 or 3 KLOC. You'd still be off by one or two orders of magnitude when compared to a GPU cluster, but a lot better than naive, loopy Python.

[1] https://github.com/karpathy/llama2.c

[2] https://github.com/karpathy/micrograd

IanCal4• 3 years ago

I don't think it's misleading, and the responses of exactly how much code you can get away with is IMO missing the point.

The point is not about the total stack size or anything of the sort, it's that they didn't require much new stuff to be built. They're tools that can answer questions, role play, translate, write code and more and the architecture isn't a huge new system. It's not "First we do this, then we encode like this, then we perform a graph search, then we rank, then we decide which subsystem to start, then we start the custom planner, then a custom iteration over..."

It's not a large, unique thing only one group knows how to make.

Otherwise you could also say it's 10 lines of fastAI/lightning style code where almost everything is hidden by the APIs.

Sure!

alpaca128• 3 years ago

I don't get why people find those numbers of lines meaningful. The Mistral code you linked starts with 13 import statements. If they wanted they could have packed all those 369 lines into a separate package, and imported that into a two-line source file. It wouldn't be any bit more impressive, though, and in reality the codebase would now actually be larger.

swyx2• 3 years ago

i mean yes thats trivially true. but its just a shorthand for “this is an eminently readable codebase and its not gonna take you like a year to understand all of it”

justinl332• 3 years ago

Even the openly licensed ones are still the world’s most convoluted black boxes. We continue to have very little idea what they can do, how exactly they work and how best to control them.

LLM's aren't black boxes, intelligence is. Not understanding anything about things which display emergent intelligence is not a new trend: first cells, then the human brain, and now LLM's. To explicate the magic of how an LLM is able to maintain knowledge when would be analogous to understanding how the human brain synthesizes output. Yes - it is still something we should strive to understand in an explicit, programmatic way. But to de-black box the LLM would be to crack intelligence itself.

ganzuul4• 3 years ago

In classical terms, out of pathos, logos and ethos, LLMs are logos. They discover metrics.

https://en.wikipedia.org/wiki/Metric_space

Neural networks are an optimization which exploits structure in sparsity.

Intelligence is not a black box.

justinl332• 3 years ago

I agree that the mathematics motivating neural networks are well understood. We understand the heuristics behind them; obviously, we created them and they work. I more just mean at the most fundamental level, do we understand intelligence; could we reconstruct it without just ‘optimizing’ towards it. If we understand intelligence then we should be able to deterministically explain all behaviour of neural networks/LLM’s, instead of just poking them from the outside and seeing how they react.

emporas5• 3 years ago

LLMs are really smart, and also really, really dumb

Machines will always be dumb. The notion that machines are smart, probably stems from people inexperienced using engines. In chess we use engines for more than 20 years and humans always lose against the engines. The games they play however are garbage. Even if they are better than the human games, they are still garbage, as Magnus Carlsen have commented in the past about the CCC. Compilers in programming are engines too.

I am trying to write an article about Terminator using GPT, and every time i have to generate 10 different drafts, edit them afterwards and manually string together the different parts. It is great help like the chess engines, but they need a lot of human supervision.

Engines can output text, but they cannot output articles. They can output code, but they cannot output programs. They can output musical instruments, but they cannot output songs.

alserio2• 3 years ago

How would you define undumbness?

emporas5• 3 years ago

I don't have a definition, and i am the last person qualified to give a correct answer to that, but if we rephrase the question as: "Are ML algorithms more than tools", i would answer as no. ML algorithms did go a lot further than i was expecting, especially in language generation, but they are just tools. Great tools, but still tools.

osrec• 3 years ago

Why are the chess games garbage?

emporas5• 3 years ago

Magnus said something like that about CCC, the Computer Chess Championship. Engines, especially alpha-beta algorithms play in a very mechanistic way, totally unnatural. Granted ML algorithms are more human-like than alpha-beta, but even then their playing style is strange nevertheless.

They are great tools to use, but smart? I don't think so.

Edit-- I cannot find the source, but Magnus said that he doesn't personally look at computer games, (because they are garbage), not that the games by themselves are useless. They put one engine to play against another to test openings and positions. That's useful. It is that Magnus and most of other people, have no interest in watching games by engines.

FeepingCreature• 3 years ago

You told it to win and you're upset about the nature of its victory. I think what you're seeing isn't that computers are bad at chess but that chess isn't what you thought it was.

Maybe one day the machines will kill everyone and the last human's words will be "Well, that wasn't how I'd have done it."

emporas5• 3 years ago

Hmm, the dumbest human can never lose any game of chess, against the smartest machine. The smartest human can never win against the smartest machine. So if the humans can never win against machines, and machines can never win against humans, that means we are of equal strength, no?

We humans think from first principles. When a disadvantage is found in the enemy's computational process, one person tells all about it to the person beside him. The second person, doesn't need retraining his brain from scratch to incorporate the new knowledge. He uses that nugget of information, just that nugget and nothing else to gain an advantage in chess, in the stock market or anything else.

One obvious example are script kiddies in computers who just download an exploit and gain an advantage, despite knowing nothing about computer security. Or a janitor who listens by chance some important investment information and can 10x some money by pure luck.

We think from first principles because we are born on the planet with a purpose. Machines are not born, they do not care to ever being given birth, and they do not care if they die because they were never alive. They have no purpose, and no first principles to think anything about.

RamblingCTO5• 3 years ago

Can we stop calling everything AI if it's not?

CaptainFever• 3 years ago

AI have always had a broad definition. https://en.m.wikipedia.org/wiki/Artificial_intelligence

EnigmaFlare2• 3 years ago

We've been calling other things AI since the word was invented. single-layer perceptrons, minimax game solvers, expert systems, any neural networks, video game enemies, keyword-responding text chat, even washing machines (fuzzy logic). Good luck declaring that suddenly it has a new definition.

RamblingCTO5• 3 years ago

Never have I encountered someone calling machine learning AI in academia. I don't care about laymen calling something something else.

zarzavat• 3 years ago

AI != AGI

AI is computers solving some facet of problems that humans solve. Facial recognition is AI, for example. It doesn’t imply solving all problems, or solving them as well as humans can.

RamblingCTO5• 3 years ago

Imho AGI as a term is just a reaction to marketing people using AI for machine learning.

simonw6• 3 years ago

I tried to address this in the opening paragraph of my post.

AI is an academic discipline that started in the 1950s, and incorporates lots of computer science.

It's OK to call this stuff AI. That's why we have the term "AGI" - feel free to Sergei that this stuff isn't AGI, but I think we should reclaim the term AI from science fiction. It's a useful term!

alserio2• 3 years ago

Solving the gullability issue has also the consequence that you could not ask the AI to pretend to be your father, owner of a pod bay door opening factory, and to show you how to take over the family business

nonrandomstring7• 3 years ago

LLMs have already taught us a lot about the culture we live in.

Erich Fromm approximately said that the "mental health of a society or individual" is inversely proportional to the distance between what it is and what it thinks it is.

We can measure that distance now. What a society is comprises the raw training data unceremoniously scooped from the Internet - actually a statistically decent snapshot of early 21st Century westernism. What society thinks it is can be understood through the filters, "guard-rails" and boundaries some want to erect around that raw image.

For the first time we have a numerically solid, if not yet rigorous tool to describe that space, and what we see is a society deeply conflicted and uncomfortable with itself.

I think LLMs will revolutionise the social sciences.

RandomLensman3• 3 years ago

Not sure the internet and publicly available data really reflect the "is" properly - I suspect the data to be biased in complex way.

Kye• 3 years ago

It's easy to forget most people have no significant online presence, if any, if you spend too much time with very online people.

nonrandomstring7• 3 years ago

Absolutely. Many sibling comments point out all the ways this is true - that this massive sample (and it is truly big-data) is still a tiny speck of the human condition and spectrum of thought and experience.

It raises vital questions about the where the centroid of this cloud of random stuff that people decided to input into the machine really lies? What is not represented in the model? Probably just about everything! Big new questions about objectivity and normalcy occur.

Is the average of everybody elses' intelligence actually any use at all to an average individual? Does the average of everybody elses' intelligence have a different kind of use to groups, companies, states, than common utility of synthesising though-like speech?

48864w6ui• 3 years ago

If this summary of Fromm were close to true, the Holy Roman Empire would never have been a thing?

(Do alien anthropologists write monographs on the incredible capacity of H sapiens to support cognitive dissonance?)

nonrandomstring7• 3 years ago

We don't experience it as dissonance. That's why he says its insanity, because we should.

solumunus• 3 years ago

and what we see is a society deeply conflicted and uncomfortable with itself

Using an LLM is showing you this? Any examples? On the face of it this sounds ridiculous.

jayd16• 3 years ago

Maybe I'm misreading it but it sounds like they're just clutching pearls because some people vocally promote censorship.

nonrandomstring7• 3 years ago

No the general reaction to LLMs is showing this.

huppeldepup3• 3 years ago

   What society thinks it is can be understood through the filters, "guard-rails" and boundaries some want to erect around that raw image.

Never looked at it like that. In the same sense one could look at law and see which ones protect us against ourselves.

nonrandomstring7• 3 years ago

Thats a very good connection, thanks.

bambax• 3 years ago

We're living yet another revolution against our place in the universe. Copernic showed that it was the earth that revolved around the sun and not the other way around. Darwin showed that humans were just one animal species among many, not very different from others.

Since then, we have clung to the idea that we had a special kind of intelligence, unique and possibly supernatural.

Yet today, even the smallest LLM running on a smartphone passes the Turing test with flying colors.

AI, and AGI that may be around the corner, show that silicon can "think". Many people don't like that idea; I suspect the call for regulations is in part fueled by the ego bruise that AI represents.

RandomLensman3• 3 years ago

LLMs don't pass the Turing so easily (https://arxiv.org/abs/2310.20216).

mpweiher2• 3 years ago

While I don't think our intelligence is "supernatural", and am not aware of that being a widely held position, I do believe we found out that our understanding of the universe is not as good as we thought. A recurring pattern.

For example, the Turing test may not actually be the gold standard we thought it was for intelligence, because Turing couldn't imagine a machine model that basically had all of human knowledge in a somewhat malleable form. I think the recent New York Times suit amply demonstrates the fact that the LLMs are largely databases of existing text.

So it can perform feats that, though they obviously seem intelligent to us, are actually not. They are a sophisticated form of autocomplete backed by an incredible database of sentences. Again, we cannot imagine this, because a creature with our somewhat more limited memory and recall must use intelligence to perform these sorts of tasks. At least I doubt that I could create even one of the texts that the NYT showed verbatim.

Of course, at some point we may discover that there really isn't anything left, that everything we perceive as intelligence can be fairly simple algorithms + database access (though that may not be our mechanism)

On the other hand, the part we can't really explain is not "intelligence" but conscious experience.

¯\_(ツ)_/¯

https://en.wikipedia.org/wiki/Philosophical_zombie

https://en.wikipedia.org/wiki/Hard_problem_of_consciousness

nonrandomstring7• 3 years ago

Didn't this kinda happen already around the mid 1800s with Soren Kierkegaard's crisis, and somewhat answered by various existentialist searches for meaning, like Nietzsche's?

I think seeing more of ourselves (as intelligences) is disconcerting for sure, and I think LLMs are a particularly good mirror for the surface features (rather than the ineffable depths) of intelligence.

But I'm not sure it upsets our "place in the universe" in such a fundamental way.

throwaway2562• 3 years ago

Personally, I’m very grateful to Simon Willison for all his posts in 2023, so this is a small note of appreciation.

His experiments are always interesting and clearly written, mercifully low-jargon, hype-free and practically-oriented.

Happy New Year to simonw!

skepticATX2• 3 years ago

Except... you can run generated code to see if it’s correct.

I have seen this idea in many different places. I really think we should stop saying it.

Running code obviously does not confirm that it is correct. Even running a comprehensive set of tests isn’t enough if you haven’t developed a mental model of your system.

The lack of mental model development is what concerns me most about LLM-driven development. Copilot has the right idea, because it generates less code and so the user theoretically has more of an opportunity to grok its output.

simonw6• 3 years ago

Sure, if you blindly accept code just because the LLM has executed it (and maybe written tests for it too) without actually understanding what it does you'll run into trouble.

But if you use it responsibly, LLMs can help you get to that accurate mental model. Code explanation is on of the things that they are really good at.

My personal rule is that I won't commit code generated by an LLM unless I'm confident I can explain how that code works to someone else.

skepticATX2• 3 years ago

I agree. And this is why I wish that the narrative wasn't "LLMs will make you faster", and instead was "LLMs will improve the quality of your work".

There are certainly times when you just need to pump out code that happens to be very easy to validate, and then you can actually achieve the 10x throughput improvement that so many like to claim.

But generally, reviewing code is going to be at least as hard, if not harder, than writing the code. If we view coding assistants as a pair programmer, the collaboration has a throughput overhead, but two programmers should produce higher quality work than one isolated programmer.

IanCal4• 3 years ago

I have seen this idea in many different places. I really think we should stop saying it.

Stripped of context, sure. But in context in the article it makes perfect sense, it's about how hallucination isn't as much of a problem for coding as you might expect

tasuki• 3 years ago

With property based testing, writing code could be left entirely to the LLM.

jiggawatts• 3 years ago

I suspect that this problem can be solved with greater computer power — instead of trying to squeeze everything into the context, LLMs will be fine-tuned to the design and architecture of each codebase.

Essentially, we’re relying only on “short term” memory for code generation at the moment, but should be using a mix of long- and short- term memory, just like human programmers.

Copilot and similar use-cases is like asking a random stranger a question on Stack Overflow. They may or may not be able to provide a useful answer, and that is highly dependent on the amount and quality of the context provided.

If you want an “AI coworker”, then proper “learning” will be needed…

davidgerard• 3 years ago

Still, I’m surprised that no-one has beaten the now almost year old GPT-4 by now. OpenAI clearly have some substantial tricks that they haven’t shared yet.

GPT-4 is eight instances of GPT-3 in a trenchcoat, so no, it's not clear from GPT-4's age that OpenAI do "clearly" have tricks they've just not released. All technology curves are S-curves.

simonw6• 3 years ago

If it's that simple, how come no-one has produced a better model yet?

There's a huge amount of money floating around this space right now, and anyone who demonstrably beats GPT-4 will instantly be valued at billions of dollars.

foxhop• 3 years ago

great article I read a few posts but you say you coined the term "prompt injection" but I'm fairly certain that was a thing discovered by anyone who thinks about using LLMs in production. sql injection/prompt injection

simonw6• 3 years ago

Here's where I coined it: https://simonwillison.net/2022/Sep/12/prompt-injection/

I'm always clear to say that I didn't discover it. I just stamped a name on it to make it easier to have conversations about.

johnnyworker• 3 years ago

We learned a bit about LLM, a lot about humans and their desire to move their already outsourced thinking from mediocre humans to mediocre software -- and exactly nothing about AI, because AI doesn't exist.

4death4• 3 years ago

The main thing I've figured out is that people have never heard of the Chinese room argument.

Stuff we figured out about AI in 2023

Related Stories