HN user
Send me a note at turingt+fromhackernews (at) gmail.com
I hope the spam filter will respect it!
Whichis is fair and good, bc nobody would expect you to rebuild the actual rules of chess from the board alone.
However it would still be useful if archeologists used the board to figure out some games similar to checkers, or go; or if they also have the pieces they could guess it was a combat game like Shogi. Any of those would give you insight about the kinds of leisure that people may get from that board.
I believe the sweet spot that makes it practical and reliable will be combining LLMs with formal verification, although I doubt current hardware is up to the task (yet).
LLMs basically solve the classic Frame problem that prevented general problem solvers to be able to reason logically about the real world; however on their own they are utterly unpredictable and unreliable.
However if the database of weights is merely used as a heuristic to guide the logical reasoning engine to promising regions of the problem space, and the program itself is written to specification directly by an inference engine, the result would be classic software not affected by hallucinations.
The LLM could even help debugging the specifications by pointing out unclear or contradicting requirements, improving the process without compromising the integrity of the result.
Which an unreliable answer is not.
Hallucinations are not a matter of some "details" being off. They are a matter of plausible, confident-sounding claims that are just plain wrong.
This is no worse than Wikipedia, or the original encyclopedia for that matter. Those contain dubious claims that you'll need to verify on your own too.
LLMs help because they have a gigantic amount of compressed knowledge, and they are able to find relevant information and present it incredibly fast. You wouldn't trust the ten first results of a Google search either, but you wouldn't say that having a search engine is totally useless and in no way an improvement over your local library, would you?
the poor person who's asking can't tell is wrong, because it sounds plausible and is stated with such confidence.
True, but having to learn how to use a tool properly doesn't make the tool useless, even if it can hurt those who use it carelessly.
But the answers it gives are not reliable. They sound plausible if you don't know anything about the subject, but they're not reliable.
Do not underestimate the utility of having a starting point overview on a topic you know absolutely nothing about. It may be immensely valuable even if some details are off. That's what made the XVIII's Encyclopedia such a valuable tool for civil society.
By the time you get to the point where those wrong details become relevant, you have gotten a basic understanding of what the overall topic is about, so you're prepared to get a second opinion from a different source - and this time you may know enough to start asking relevant questions, rather than starting from full ignorance.
Societal changes are slow beasts, they may very well take several centuries to develop. Nation-states were a direct consequence of the printing press, yet they didn't arrive until XIX century.
They had this project called Cairo that was supposed to throw out that scruffy old file-based filesystem and bring in a shiny new Object Based File System instead. It never happened, so we'll never know exactly how it might have turned out.
Nowadays we call those APIs. They are REST based rather than file-based to make them distributed, the main difference is that you don't get a common user interface that all providers adjust to; you need to choose your own client to read them and write into them.
And because they're created by programmers for programmers, they're not what you'd call user-friendly. Usually the only efficient way to use them is programmatically, so that you need to create a specific user interface for each API. Somehow, I doubt that Cairo would have come to be anything much different from that in the end.
I would just reverse the architecture of the whole system. Build a classic deterministic program, and use LLMs as heuristics adapting the system to the environment - the functions that you call on the 'if's and 'switch' statements to decide where the system should go.
I see this as the most robust way to build a predictable system that runs in a controlled way while taking advantage of probabilistic AIs while reducing the impact of their alucinations.
LLMs simply can't be trusted to follow instructions in the general case, no matter how much you constraint them. The power of very large probabilistic models is that they basically solved the _frame problem_ of classic AI: logical reasoning didn't work for general tasks because you can't encode all common sense knowledge as axioms, and inference engines lost their way trying to solve large problems.
LLMs fix those handicaps, as they contain huge amounts of real world knowledge and they're capable of finding facts relevant to the problem at hand in an efficient way. Any autonomous system using them should exploit this benefit.
Yes, my point is that those three arguments may be compelling but they assume that reality is correlated to the shape of their thoughts. What they have in common is that they all miss the insight that you need to actually test your assumptions to improve your certainties, and that's not feasible for theoretical all powerful entities that can bend reality.
I find Pascal's wager is of the same nature as Aquinas' Five Ways to prove God, or accelerationists about the inevitability of a Singularity: believing that your own rational argument can be the basis to prove a fact about reality merely because it feels internally consistent.
Needless to say, I don’t find them at all convincing. This 'nothing' is much better than catching unconvincing unneeded supernatural entities.
No, I'm saying that they are not cause and effect but coevolution. Their agitprop could have such huge impact because of the conditions of workers in Zarist Russia and the Republic of China respectively. They wouldn't have worked in a different society; so no, they didn't single-handedly create the conditions for their own power, there was a previous substrate they could work on.
You think Lenin and Mao didn't have behind them an ideology in their societies that supported them? Why did people follow their orders then, mind control?
I would argue plenty of significant societal changes were caused by the behavior of relatively small number of people
Specific breaking points in history yeah, maybe. But that's possible because they're well connected people near the center of the network.
Those breakpoints are possible because either those few people share a viewpoint held by a large number of their peers, or benefit from knowledge accumulated throughout their civilization. Think how every dictator needs support from a huge following to get their power (and how easy it is to find another dictator to replace them if they die), or how often some breakthrough discoveries are made by multiple people at the same time. There's always a last straw that breaks the camel's back, but the lone wolf hardly ever gets a significant impact on society at large; they need a receptive audience to get any impact. Humans are herd animals.
Following the metaphor, the butterfly effect is only possible because a storm was brewing in the first place; the butterfly wings only decide where it will appear. Butterfly wings just don't have that much energy.
History is told from the perspective of kings, but kings can reign only within a society that believes in their divine right to rule.
Who gets to say that the demos is fundamentally flawed? Each in-group have their own opinions on what's a flaw.
Society evolves through epiphenomena caused by the behaviour of the majority; the fact that some minorities view that evolution as 'flawed' cannot change that evolution, unless they're able to influence the majority to also see it that way.
Now, democracy is essentially a way for everybody to broadcast their views on society's flaws on non-violent ways. The alternative is that some groups broadcast their opinions in violent ways, and we have learned to see that situation as undesirable.
The problem with that thought is that Goole isn't creating a good solution, it's creating this specific one.
We're finally getting there. The model of web notebooks look a lot like Hypercard stacks in terms of usability; there's only missing someone packing them in and easy-to-use distribution and sharing environment that does not depend on users installing their own web server.
And if that package includes some reasonable local LLM model, creating simple programs by end users could be even easier than it ever was with Hypercard.
Isn't that the same as compressing the whole book, in a special differential format that compares how the text looks from any given point before and after?
I can only speak for myself but for me, it's all about the syntax. I am terrible at recalling the exact name of all the functions in a library or parameters in an API, which really slows me down when writing code. I've also explored all kinds of programming languages in different paradigms, which makes it hard to recall the exact syntax of operators (is comparison '=' or '==' in this language? Comments are // or /*? How many parameters does this function take, and in what order...) or control structures. But I'm good at high level programming concepts, so it's easy to say what I want in technical language and let the LLM find the exact syntax and command names for me.
I guess if you specialise in maintaining a code base with a single language and a fixed set of libraries then it becomes easier to remember all the details, but for me it will always be less effort to just search the names for whatever tools I want to include in a program at any point.
It is possible to try it, and some people do (high speed trading is just that, plus taking advantage of privileged information that speed provides to react before anyone else).
However there are two fundamental problems to computational predictions. The first one obviously is accuracy. A model is a compressed memorization of everything observed so far; a prediction with it is just projecting into the future the observed patterns. In a chaotic system, that goes only so far; the most regular, predictable patterns are obvious to everybody and give less return, and the chaotic system states where prediction would be more valuable are the less reliable. You cannot build a perfect oracle that would fix that.
The second problem is more insidious. Even if you were able to build a perfect oracle, acting on its predictions would become part of the system itself. That would change the outcomes, making the system behave in a different way as it was trained, and thus less reliable. If several people do it at the same time, there's no way to retrain the model to take into account the new behaviour.
There's the possibility (but not a guarantee) to reach a fixed point, that a Nash equilibrium would appear where such system becomes into a stable cycle, but that's not likely in a changing environment where everybody tries to outdo everyone else.
The word he's looking for is "formal system".
For some reason he doesn't like doing mathematical demonstrations so he shuns the practice of doing them, and invented a new word to describe that way of using formal systems.
You never lived the 90's
But both are doing the winning thing, which is more valuable than just the battle thing. Unless you do it just for fun and don't mind the result.
Sometimes that's because they're making it worthwhile, by connecting the thing with those who will benefit from it and explaining how to use it, which is as valuable as doing the thing.
I.e. by making sure that they're doing the right thing.
A human rubber-stamping code being validated by a super intelligent machine is the equivalent of a human sitting silently in the driver's seat of a self-driving car, "supervising".
So, absolutely necessary and essential?
In order to get the machine out of trouble when the unavoidable strange situation happens that didn't appear during training, and requires some judgement based on ethics or logical reasoning. For that case, you need a human in charge.
Wikipedia is and has always been a wiki; reverting bad or controversial edits has always been expected from day one.
Also Wikipedia has developed an editorial line of its own, so it's normal that edits that go against the line will be put in question; if that happens to you, you're expected to collaborate in the talk pages to express your intent for the changes, and possibly get recommendations on how to tweak it so that it sticks.
It also happens that most of contributions by first timers are indistinguishable from vandalism or spam; those are so obvious that an automated bot is able to recognize them and revert them without human supervision, with a very high success rate.
However if those first contributions are genuinely useful to the encyclopedia, such as adding high quality references for an unverified claim, correcting typos, or removing obvious vandalism that slipped through the cracks, it's much more likely that the edits will stay; go ahead and try that experiment and tell us how it went.
You're right, I mistyped it.
The main issue with neutral people is that we do not know in which camp they are.
And that's a good thing, 'cause it means they're living to their standards.
Oh dear, you need to learn about the GamerGate incident which started August 2012. All the extreme division and online manipulation through the collaborative creation of false narratives started right there, with that issue, before contaminating the entire political landscape.
It's the Eternal September of our generation, and it's not recognised enough as such. Before that, the internet was a different place.
Why? What you're asking for is a machine that never breaks.
No, I'm saying than 'alignment' is a concept that doesn't help to solve the problems that will appear when the machine ultimately breaks; and in fact makes them worse because it doesn't account for when it'll happen, as there's no way to predict that moment.
Following your metaphor of criminals: you can control humans to behave following the law through social pressure, having others watching your behaviour and influencing it. And if someone nevertheless breaks the law, you have the police to stop them from doing it again.
None of this applies to an "aligned" AI. It has no social pressure, its behaviours depend only on its own trained weights. So you would need to create a police for robots, that monitors the AI and stops it from doing harm. And it had better be a humane police force, or it will suffer the same alignment problems. Thus, alignment alone is not enough, and it's a problem if people depend only on it to trust the AI to work ethically.
If your training data is sufficiently moral, the outputs will be as well.
Correction: if your training data and the input prompts are sufficiently moral. Under malicious queries, or given the randomness introduced by sufficiently long chains of input/output, it's relatively easy to extract content from the model that the designers didn't want their users to get.
In any case, the elephant in the room is that the models have not been trained with "sufficiently moral" content, whatever that means. Large Language Models need to be trained on humongous amounts of text, which means that the builders need to use a lot of different, very large corpuses of content. It's impossible to filter all that diverse content to ensure that only 'moral content' is used; yet if it was possible, the model would be extremely less useful for the general case, as it would have large gaps of knowledge.
If you tell this robot to take a knife and cut onions, alignment means it isn't going to take the knife and chop of your wife
Yeah, I agree that alignment is a desirable property. The problem is that it can't really be achieved by changing the trained weights; alleviated yes, eliminated no.
we can greatly reduce the probabilities they will show it
You can change the a priori probabilities, which means that the undesired problem will not be commonly found.
The thing is, then the concept provides a false sense of security. Even if the immoral behaviours are not common, they will eventually appear if you run chains of though long enough, or if many people use the model approaching it from different angles or situations.
It's the same as with hallucinations. The problem is not that they are more or less frequent; the most severe problem is that their appearance is unpredictable, so the model needs to be supervised constantly; you have to vet every single one of its content generations, as none of them can be trusted by default. Under these conditions, the concept of alignment is severely less helpful than expected.