aithrowaway1987

But that is not the standard for current GNU projects in large part because of all the easily avoidable friction. "If it was good enough for Richard Stallman in 1987, it's good enough for Microsoft in 2024" is just a dumb argument.

Not to mention you're conflating apples with oranges, since a software standard is very different from an application. POSIX wasn't just one Bell Labs employee working by himself.

From the article:

The LSP should be an open standard, like HTTP, with an open committee that represents the large community which is invested in LSP, and can offer their insight in how to evolve it.

There is no goalpost moving here.

At least for me, Monty Hall makes more intuitive sense to me when there are 10 doors than when there are 3, even though the benefit of the optimal strategy is quite small: there's a 1/10 chance you picked the car and shouldn't have switched, but a 9/10 chance that you can play the game again with 1:9 odds. Without doing the math, it makes sense to me that the optimal strategy actually does improve your odds.

Yet for some reason a 2/3 chance you can play the game with 50-50 odds is harder to accept - in particular I have to consider the 10 door case to understand why the 3 door case makes sense. I suspect it has to do with the psychology of loss aversion: a 1/3rd chance that you incorrectly switched "feels like" a reckless risk.

Look at who the tools are marketed towards. Writing software involves a lot of tedium, eye strain, and frustration, even for experts who have put in a lot of hours practicing, so LLMs are marketed to help developers make their jobs easier.

This is not the case for art or music generators: they are marketed towards (and created by) laypeople with who want generic content and don't care about human artists. These systems are a significant burden on productivity (and fatal burden on creativity) if you are an honest illustrator or musician.

Another perspective: a lot of the most useful LLM codegen is not asking the LLM to solve a tricky problem, but rather to translate and refine a somewhat loose English-language solution into a more precise JavaScript solution (or whatever), including a large bag of memorized tricks around sorting, regexes, etc. It is more "science than art," and for a sufficiently precise English prompt there is even a plausible set of optimal solutions. The LLM does not have to "understand" the prompt or rely on plagiarism to give a good answer. (Although GPT-3.5 was a horrific F# plagiarist... I don't like LLM codegen but it is far more defensible than music generation)

This is not the case with art or music generators: it makes no sense to describe them as "English to song" translators, and the only "optimal" solutions are the plagiarized / interpolated stuff the human raters most preferred. They clearly don't understand what they are drawing, nor do they understand what melodies are. Their output is either depressing content slop or suspiciously familiar. And their creators have filled the tech community with insultingly stupid propaganda like "they learn art just like human artists do." No wonder artists are mad!

Might there be certain laws of physics that are also “necessary” in the same way? In his paper, Molinini argues that the principle of conservation may be one such law. In physics, some properties of a system, such as energy or momentum, can’t change. A bicyclist freewheeling down a hill, for example, is converting her gravitational potential energy into movement energy, but the total amount of energy she and her bike have stays the same.

Arithmetic itself is a consequence of physical conservation: if you have a collection of four acorns, another collection of three acorns, then combine them without dropping an acorn, then you must have a collection of seven acorns. It is our deep physical understanding of space and causality which leads to simple arithmetic being intuitively true to most (if not all) vertebrates. (If the squirrel only got six acorns after combining then there must be a causal explanation for the quantitative discrepancy; another squirrel stole an acorn from the older stash, or maybe it fell in a hole.)

Isn't "plasticity is not necessary for intelligence" just defining intelligence downwards? It seems like you want to restrict "intelligence" to static knowledge and (apparent) short-term cleverness, but being able to make long-term observation and judgements about a changing world is a necessary component of intelligence in vertebrates. Why exclude that from consideration?

More specifically: it is highly implausible that an AI system could learn to improve itself beyond human capability if it does not have long-term plasticity: how would it be able to reflect upon and extend its discoveries if it's not able to learn new things during its operation?

Because they can learn a bunch of symbolic formal arithmetic without learning anything about quantity. They can learn

  5 x 3 = 15

  *****    ****     *******
  ***** =  *****  = *******
  *****    ******   *

Recent examples I've seen fall well within the range of innumeracy that people routinely display.

Here's GPT-4 Turbo in April botching a test almost all preschoolers could solve easily: https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_pr...

I have not used LLMs since 2023, when GPT-4 routinely failed almost every counting problem I could think of. I am sure the performance has improved since then, though "write an essay with 250 words" still seems unsolved.

The real problem is that LLM providers have to play a stupid game of whack-a-mole where an enormous number of trivial variations on a counting problem need to be specifically taught to the system. If the system was capable of true quantitative reasoning that wouldn't be necessary for basic problems.

There is also a deception is that "chain of thought" prompting makes LLMs much better at counting. But that's cheating: if the LLM had quantitative reasoning it wouldn't need a human to indicate which problems were amenable to step-by-step thinking. (And this only works for O(n) counting problems, like "count the number of words in the sentence." CoT prompting fails to solve O(nm) counting problems like "count the number of words in this sentence which contain the letter 'e'" For this you need a more specific prompt, like "First, go step-by-step and select the words which contain 'e.' Then go step-by-step to count the selected words." It is worth emphasizing over and over that rats are not nearly this stupid, they can combine tasks to solve complex problems without a human holding their hand.)

I don't know what you mean by "10 years ago" other than a desire to make an ad hominem attack about me being "stuck." My point is that these "capabilities" don't include "understands what a number is in the same way that rats and toddlers understand what numbers are." I suspect that level of AI is decades away.

There was a very good paper in Nature showing this definitively: https://news.ycombinator.com/item?id=41437933

Modern ANN architectures are not actually capable of long-term learning in the same way animals are, even stodgy old dogs that don't learn new tricks. ANNs are not a plausible model for the brain, even if they emulate certain parts of the brain (the cerebellum, but not the cortex)

I will add that transformers are not capable of recursion, so it's impossible for them to realistically emulate a pigeon's brain. (you would need millions of layers that "unlink chains of thought" purely by exhaustion)

In 2022 Ilya Sutskever claimed there wasn't a distinction:

It may look—on the surface—that we are just learning statistical correlations in text. But it turns out that to ‘just learn’ the statistical correlations in text, to compress them really well, what the neural network learns is some representation of the process that produced the text. This text is actually a projection of the world.

(https://www.youtube.com/watch?v=NT9sP4mAWEg - sadly the only transcripts I could find were on AI grifter websites that shouldn't be linked to)

This is transparently false - newer LLMs appear to be great at arithmetic, but they still fail basic counting tests. Computers can memorize a bunch of symbolic times tables without the slightest bit of quantitative reasoning. Transformer networks are dramatically dumber than lizards, and multimodal LLMs based on transformers are not capable of understanding what numbers are. (And if Claude/GPT/Llama aren't capable of understanding the concept of "three," it is hard to believe they are capable of understanding anything.)

Sutskever is not actually as stupid as that quote suggests, and I am assuming he has since changed his mind.... but maybe not. For a long time I thought OpenAI was pathologically dishonest and didn't consider that in many cases they aren't "lying," they blinded by arrogance and high on their own marketing.

I learn best via writing things out by hand in my own words, and almost never read the notes afterwards. I am also profoundly disorganized :) Before I got a reMarkable I had accumulated (and thrown out) dozens of bulky paper notebooks. Now those are all digital.

Despite reMarkable's marketing around high-quality hand-drawn professional notes, I suspect crappy "transient" notes to aid memory and mental organization are the most common use case. For me it's really a thinking device rather than a writing device.

If I actually need to reference or organize my notes I will type something out in emacs.

My issue with this line of argument is that people always want to compare "do it with Copilot" to "do it completely by scratch" when they should be comparing it to "do it by ignorantly copy-pasting from one of the many similar projects on GitHub then tweaking a few things." There are quite a few open-source GLSL implementations of marching squares, maybe copy-pasting would have been faster and higher-quality.

But isn't the ability to learn also a major component of IQ?

Not sure what you mean by this - it's certainly not something a single-day IQ test could possibly measure! The primary reason IQ is a discredited measure of intelligence is that people are perfectly able to learn how to perform better on IQ tests - any supposed influence of "intelligence" on IQ scores is hopelessly confounded with how much you've practiced similar tests / trivial logic puzzles / etc.

This stuff about "attempts to broaden the definition of intelligence to something that is more inclusive" is backwards. The whole problem is that nobody has managed to scientifically define (let alone measure) intelligence in any vertebrate species. IQ is dangerously misleading precisely because it is so narrow: its precision makes claims about IQ seem quantitatively rigorous when they are qualitatively meaningless.

If you care about music: at least 100 years, almost certainly much longer. I don't think any of us in this comment section will live to see an AI that truly understands human music.

If you care about money and don't mind making the world a more terrible place: humiliate a few dozen human classical pianists by making them record hundreds of hours of motion capture, invest in engineering a good robo-arm, and I would guess in 5 years you'd have something passable.

why Drew is so sympathetic to the cause

I think this is a mischaracterization. If you read the original post[1] it's clear he's not at all sympathetic to the cause, though given recent news he is more sympathetic to the people who have the cause (even if he thinks it's misguided).

[1] https://drewdevault.com/2022/10/03/Does-Rust-belong-in-Linux...

AFAICT it has not been described "in detail," the only writeup I know of is this vague, flowery blog post: https://waymo.com/blog/2024/05/fleet-response/

it's not in a way that would be particularly helpful here.

Why would you say that? This claim seems completely unsupported. What if the human directed the Waymo to drive more slowly and carefully than it would otherwise? That sort of instruction seems entirely consistent with the blog post and would be critical to the impressive behavior we see in the video.

You may be under the common misunderstanding that remote operators can directly control the vehicle.

I think you are under the misunderstanding that since humans don't control the vehicle, they must have only a second-order impact on safety. But I am not concerned about self-driving's technical ability to dodge obstacles, I am concerned about its judgment around passing school buses or not slowing down when there are lots of pedestrians. It's this sort of human-level reasoning that I suspect is critical for Waymo's safety record. Nothing in Waymo's blog post / infomercial suggests otherwise.

Waymo's numbers aren't dishonest and their accomplishment is real, but this data should be taken in its proper context.

1) The comparison lumps in a lot of humans which cynically and selfishly break the law. Currently Waymos attempt to obey the laws of the road, but Tesla FSD shows that self-driving companies can and will market towards people who want to speed, run red lights, etc. So I am deeply concerned that a formal comparison of law-abiding self-driving cars will be used to encourage development of autotaxis which compete with human Ubers on travel time by speeding or rolling through stop signs like a human would. It would ultimately be a correlation/causation error: Waymo's training and development correlates with better safety records when the causative variable is ultimately risk-adversity, with better engineering being a secondary factor. In particular, I suspect Waymos will eventually be safer than humans at legal driving but dramatically more dangerous at illegal driving, and Waymo executives using motivated reasoning to dispel the concerns. Again, FSD literally offered different options for how much you wanted to speed or roll stop signs. This isn't hypothetical.

2) This is less concerning than reckless driving, but these stats include a lot of human accidents due to mechanical failure. This is something I expect self-driving to handle very poorly without human intervention. The most cognitively demanding moment of my entire life was having a rear tire blowout while driving a heavily-loaded truck with little experience - as the end started to fishtail I quickly realized what happened and carefully applied the brakes, hit the emergency lights, closely paying attention to the tactile feedback in my feet and hands, then pulled over as soon as I possibly could. If I were an AI and this was an edge case the engineers forgot to train on, Robo-Me would have easily flipped the truck. Mechanical failures aren't occurring in Waymos now because the cars are still new. Eventually it might be a real problem.

The thing I wish we better understood about Waymo is how much the remote human operators are actually intervening on a daily basis - maybe I don’t know where to look, but I’ve never gotten a clear answer here. AFAICT the success of Waymo means Tesla needs to have a similar level of human oversight for its FSD vehicles. But since Waymo’s PR is all about the autonomy, way too many people have the impression that Waymo’s advantage is solely about better algorithms along with humility around limited geographical range. These are probably important factors! But I strongly suspect we don’t appreciate the human oversight.

There's a huge difference between "the AI receives synthetic data as if from an omniscient being and proceeds as if it were true" and "the animal brain creates synthetic data and assesses its plausibility and downstream consequences."

Synthetic data in AI training in fact has nothing to do with dreaming or armchair theorizing. It's a ridiculous comparison.

I am struggling to understand a single implication of this! How does this generalize to anything other than other than playing retro games in the most expensive way possible? The very intention of this project is overfitting to data in a non-generalizable way! Maybe it's just pure engineering, that good ANNs are getting cheap and fast. But this project still seems to have the fundamental weaknesses of all AI projects:

- needs a huge amount of data, which a priori precludes a lot of interesting use cases

- flashy-but-misleading demos which hide the actual weaknesses of the AI software (note that the player is moving very haltingly compared to a real game of DOOM, where you almost never stop moving)

- AI nailing something really complicated for humans (98% effective raycasting, 98% effective Python codegen) while failing to grasp abstract concepts rigorously understood by fish (object permanence, quantity)

I am genuinely struggling to see this as a meaningful step forward. It seems more like a World's Fair exhibit - a fun and impressive diversion, but probably not a vision of the future. Putting it another way: unlike AlphaGo, Deep Blue wasn't really a technological milestone so much as a sociological milestone reflecting the apex of a certain approach to AI. I think this DOOM project is in a similar vein.

witnessing it's neurons being used as a computing resource

What exactly is doing the witnessing here, if the neurons are being used for computation? A lot of these comments seem to tacitly assume there is a little person inside our brains who is at the controls, and that organoid computing means that these controls are forcibly taken away, causing the magic little person to suffer horribly. It's strangely childish.

While human brains are probably just scaled-up chimpanzee brains, individual primate neurons are significantly more sophisticated than other mammalian taxa, particularly with epigenetic changes: https://www.nature.com/articles/s41467-022-34800-w But also the plain structure of the dendrites and how they form connections: https://pubmed.ncbi.nlm.nih.gov/22230639/

Now: I don't think wetware computing is actually sophisticated enough to leverage primate-specific advantages. But 10,000 primate neurons would probably be "more powerful" than 10,000 pig neurons. (OTOH it seems like corvid neurons are even more powerful than primates, maybe we should use crows.)

I also noticed that they played AI DOOM very slowly: in an actual game you are running around like a madman, but in the video clips the player is moving in a very careful, halting manner. In particular the player only moves in straight lines or turns while stationary, they almost never turn while running. Also didn't see much strafing.

I suspect there is a reason for this: running while turning doesn't work properly and makes it very obvious that the system doesn't have a consistent internal 3D view of the world. I'm already getting motion sickness from the inconsistencies in straight-line movement, I can't imagine turning is any better.

It seems misleading to lump AlphaGeometry in with this: that system works by having an LLM generate hundreds of Lean programs and trying to find the one that works. Frankly the project is pointless and borderline dishonest: it is a hideously expensive way to solve useless Olympiad problems, but it won't work at all for actual mathematical research since we wouldn't have nearly enough training data. It does not represent even the slightest advancement at AI performing mathematics, it's pure gimmickry. I suspect the entire point of AlphaGeometry is to generate comments like yours, "AI is now an Olympian-level mathematician."

The real AI math advancement will be a computer which is as smart as a pigeon and innately understands what a number is, which means it is able to solve simple counting problems without training. AlphaGeometry or Strawberry aren't even close to that, and I suspect none of us will live to see such a sophisticated AI.

Have you ever played a video game? This is unbelievably depressing. This is a future where games like Slay the Spire, with a unique art style and innovative gameplay simply are not being made.

Not to mention this childish nonsense about "forget they're playing a game," as if every game needs to be lifelike VR and there's no room for stylization or imagination. I am worried for the future that people think they want these things.

It's not "vibes" but it is a moral perspective, not an opinion informed by data. In particular the total absence of any scientific definition of intelligence makes any "data" meaningless, which is why my comment was a semi-joke, and certainly not an "opinion." When I say Mensa members are dumb I mean it in the same way that Trump voters are dumb. I am not making a comment about overall cognitive ability.

That said, I was being mean because of Mensa's shtick is scamming naive young people. But it's an awful organization which is chock-full of very dumb adults with fully-developed cortices: https://www.newyorker.com/culture/culture-desk/jamie-loftus-...

You have to be ignorant and closed-minded to even consider being a member of Mensa - this is fine for a teenager but not for a real adult. I understand some people are not neurotypical and have trouble finding community (I have schizophrenia, trust me, I get it). But anyone who actually takes Mensa's membership criteria seriously is an idiot, by which I mean "utterly lacking in intellectual curiosity and proudly ignorant about science," even if they're (supposedly) good at trivial puzzles.

I think the 1600 number is a coarse metric for this sort of thing. Keep in mind that these instructions are limited in the number of formal parameters they can take: e.g. 16 nominally distinct instructions can be more readily understood/memorized as one instruction with an implicit 4-bit flag. Obviously there's a ton of legacy cruft in Intel ISAs, along with questionable decisions, and I'm not trying to take away from the appeals of RISC (e.g. there are lots of outstanding compiler bugs around these "pseudoparamaterized" instructions). But it's easy to look at "1600" and think "ridiculous bloat," when in reality it's somewhat coherent and systematic - and more to the point, clearly necessary for highly performance-sensitive work.

It is quite striking that no two photos actually look like the same person - crudely similar, but the face shape is all over the place.

I would think this is an especially long “last mile” for training and fine-tuning: this might need a more sophisticated approach than a diffusion model, e.g an initial 2D->3D layer that first creates a 3D representation of the user’s face from a photo.

Disclaimers aren't magic words that dispel all liability. Joe Rogan isn't allowed to lie about people and claim "hey, I'm a comedian, I wasn't being serious." There is a high bar for satire or rhetorical exaggeration around public figures, but this Copilot confabulation is plain slander and Microsoft deserves to get in trouble for it.

The shadiness about Facebook's proprietary dataset of 300 million photos is concerning and should draw more attention. At the very least it is scientifically unacceptable - we should not high-five Big Tech researchers for intentionally unreproducible research. And if Meta is harvesting user photos for AI research and commercialization, they should tell their users about it directly (I am sure there is something buried in the TOS). Does the dataset include only public photos, or are Instagram DMs fair game? Does it include CSAM? Who cares!

Serious question: who are the people in the illustrations they used in the paper?[1] Are they Facebook/Instagram users? Did the authors ask permission to use their photos for an arXiv publication? Including their kids? Meta researchers really should be answering questions like this before they are asked - but these authors didn't even include an impact statement!

https://arxiv.org/abs/2408.12569

I am whiny about being anti-AI, but no doubt they are well-suited for English<->programming translation tasks, which along with a well-tuned bag of tricks is a useful tool.

But two issues the tech community had not spent nearly enough effort discussing:

1) As a big fan of Idris, I am worried that these tools will strongly disincentivize language development: why design an elegant language if an LLM can write the boilerplate faster than you can write a cleaner implementation?

2) I still don't think these tools are even slightly ethical. In 2022 I kicked the tires on ChatGPT-3.5 for F# codegen, and got some truly terrible results. I copy-pasted some lines into GitHub and found the unique repositories which ChatGPT was obviously plagiarizing from, and with 15 seconds of prompt "engineering" I got it to spit out ~200 lines verbatim from my personal F# linear algebra library - the only thing that was changed was stripping out the comments and updating some syntax to F# 4.7. Pure plagiarism. It is especially frustrating that GPT is more likely to plagiarize that library precisely because there aren't very many similar repos on GitHub.

Obviously the plagiarism problem can be fixed. (and it seemingly has been...for F#. Not sure about Idris!) However, it really seems like that sort of RLHF fine-tuning is about covering OpenAI's tracks, not "teaching" the AI how to "generalize." In particular I refuse to use the tool because now instead of reliably getting it to plagiarize from F# developers, I have no clue whatsoever if it's stealing or if it managed to truly autoregress its way into an ethical solution. So instead of rolling the dice on being a graceless scumbag, I'll just take my time writing out my code by hand.

And it was striking that GPT-3.5 had read and memorized more F# than Don Syme has seen in his entire life, yet in response to simple questions it was a mindless plagiarist. It's a stark illustration why the legal argument that ANN learning = human learning is vacuous, and why OpenAI should lose most of the copyright lawsuits it's facing.