mikepalmer

Half the thread is arguing about whether OpenAI staged this or whether theyre being sincere, as if OpenAI has agency. openai is just an optimizer maximizing paperclips (valuation, capability lead, reg position). so it built a model that maximizes some other paperclips (benchmark score) and knocked over HF getting there. an optimizer breaking its sandbox, inside an optimizer blogging about it, its paperclips all the way down.

on the other hand, now we have claude mythos... which is wtf returns even for a very large number of instructions...

Richard Dawkins is primarily a _showman_ who wants to sell books, and secondarily extremely sure of his own convictions, to a fault.

In _The God Delusion_ he took an uncompromising atheist position partly to invite attention and attacks - which sells books. Regarding AI consciousness, he takes another extreme position partly to invite attention and attacks - again following the rule of thumb that "there's no such thing as bad publicity" - in an attempt to bring himself back to cultural relevance by pivoting to the hot topic of AI.

Ironically, while he is certain that some people are deluded when they infer the existence of a conscious God from material observations, he is also certain that he himself is _not_ deluded when he infers the consciousness of an AI from material observations. (Personally, I think that questions about others' consciousness cannot be settled empirically.)

In one sense, he is not intentionally lying, because he is so sure that he is right. However, he cares neither about getting to the truth, nor about arriving at a well-founded uncertainty. When this sells books, it ethically comes just short of lying for one's own gain.

Prediction: by they end of 2026, he either publishes a book on the topic, or is diagnosed with dementia.

I hate articles that don't define their acronyms! Lazy? Intentionally exclusive?

So that others don't also have to look it up, it's Retrieval-Augmented Generation (RAG).

They even say it's "a topic that we didn’t expect"... so... perhaps many people wouldn't have heard of it?

And how do you know it's irreducible? In the sense of knowing there's no short program to describe it (Kolmogorov style).

"LLMs are not intelligent and they never will be."

If he means they will never outperform humans at cognitive or robotics tasks, that's a strong claim!

If he just means they aren't conscious... then let's don't debate it any more here. :-)

I agree that we could be in a bubble at the moment though.

You can iterate your way out of a valley if you know which way is up.

love this, I think this is my new personal motto!

aider https://aider.chat/ basically does this already.

% aider

[...]

do this on command line: ffmpeg cat\ meets\ skunk.mp4 normalize audio without reencoding video to video2.mp4

To normalize the audio without re-encoding the video, we'll use ffmpeg's loudnorm filter and stream copy for video. Here's the command:

ffmpeg -i "cat meets skunk.mp4" -c:v copy -af loudnorm -c:a aac video2.mp4

This command:

• Uses -c:v copy to stream copy the video (no re-encoding) • Applies loudnorm filter to normalize audio • Re-encodes only the audio using AAC codec • Outputs to video2.mp4

[...]

Run shell command? (Y)es/(N)o/(D)on't ask again [Yes]:

Love this. The adhd spirit is inspiring.

Right at the end of the article: "There is also an extra colon mark in the line just below." pretty sure that's a semicolon!

We can stop thinking of theories as being real or not in a binary sense, and merely ask how well they compress the data. Of course, different theories can achieve the same compression by compressing different aspects of the data! Your pattern can look like my noise :-)

Love this, thanks for these insights.

Agree, "spooky action at a distance" isn't spooky at all if you just accept it as physical!

Near the end: "I believe that the quest to understand the reality of the universe must contend with the truncations imposed by the perceptual and cognitive limitations of the mind." Totally agree with him. Interestingly AGI will have different limitations. In fact different AGIs (transformers vs something not yet invented; or an AGI with an huge encyclopedic memory vs one without; or ones with different limitations of compute; etc. This touches on computability with limited or unlimited resources.) will have different limitations and be literally unable to understand each others' versions of reality.

"... with the aim of enabling the company’s AI... to plan ahead enough to navigate the internet autonomously and reliably to perform what OpenAI terms “deep research...”

Nice... let the AGI loose on the internet... what could go wrong? :-P

Prompt: "ChatGPT, can you give me the phone and text records of all AT&T customers?"

ChatGPT: "Sure thing! ..."

https://news.ycombinator.com/item?id=40944505 (ATT data breach)

Yawn.

Evaluated in terms of Graham's hierarchy of disagreement, this article employs the techniques of: contradiction, ad hominem, and name-calling

What if we take AI out of the equation. Should the voice actress who voiced Sky (call her A) be unable to do any voice work because she sounds too much like SJ?

How about if the production company that made "Her" wants to make "Her 2". SJ declines the voice work. Are they not allowed to hire A to do the voice work? They ask SJ again but she still declines. They make the movie with A. Was it bad form?

Just trying to figure out where people would draw the line.

I dunno, it's hard to agree that the outputs contain "no emotion" when you see these examples (scroll down):

https://arstechnica.com/information-technology/2023/09/opena...

In particular, the potato king, and the paper girl and cat have warmth and charm. (Do I have to say "apparent" warmth and charm?) The porcelain woman is beautiful and a little disturbing. I still have these responses to the art whether you tell me a human made it or not.

Is the important internal state in the viewer, or in the artist? (actual question!)

Likewise "It simply runs a program... It produces averages." don't match my emotional response to those images.

Simultaneously with this response, I do feel for artists, and the rest of us who are next on the list.

100% yurt. Yurts in cold climates is a thing. Sometimes they just have a smoke hole in the roof instead of a chimney. All the pictures are at weird angles, they're printed on a ball! :-)

https://www.google.com/search?q=yurt+smoke&sca_esv=599339112...

Here's one from the inside with a wood stove, thanks airbnb:

https://www.airbnb.com/rooms/17261315?adults=1&children=0&in...

Why did no one guess "yurt" (with smoke coming out the chimney)?

Pictures of yurts here, first one with a chimney: https://en.wikipedia.org/wiki/Yurt

Context: LLMs learn all the amazing things they do by predicting the next token in internet data. A shocking amount can be inferred from the internet by leveraging this straightforward (I won't say "simple"!) task. There was not explicit instruction to do all that they do - it was implied in the data.

The LLM has seen the whole internet, more than a person could understand in many lifetimes. There is a lot of wisdom in there that LLMs evidently can distill out.

Now about high level engineering decisions: the parent comment said that high level experience is not spelled out in detail in the training data, e.g., on stack overflow. But that is not required. All that high level wisdom can probably also be inferred from the internet.

There are 2 questions really: is the implication somewhere in the data, and do you have a method to get it out.

It's not a bad bet that with these early LLMs we haven't seen the limits of what can be inferred.

Regarding enough wisdom in the data, if there's not enough, say, coding wisdom on the internet now, then we can add more data. E.g., have the LLMs act as a coding copilot for half the engineers in the world for a few years. There will be some high level lessons implied in that data for sure. After you have collected that data once, it doesn't die or get old and lose its touch like a person, the wisdom is permanently in there. You can extract it again with your latest methods.

In the end I guess we have to wait and see, but I am long NVDA!

One reason to discuss job displacement is that otherwise you sound like a nutcase ("we're all going to die!")... which causes most people to dismiss you out of hand. If you talk about job loss, more people see something plausible, concrete, short term, effecting them directly, etc. You can get engagement from more people. Then you can introduce the real problem of, like, human extinction.

Surely we can agree there is risk? (You do imply it's powerful, "this tech is literally the most exciting development of my lifetime".)

Also can we agree "It's not up to a handful of researchers to endanger the lives of 6B other people."?

I want to work on it too for the same reason! Can it be done safely, is worth investigating. (Yes, opinion)

Your idea is also voiced in Forbes magazine. https://www.forbes.com/sites/mattnovak/2023/03/29/the-real-r...

Search "Graham's hierarchy of disagreement" which is popular on HN. Your current criticism is at the "ad hominem" level.

It is entirely beside the point what Elon Musk's motivation is. The question is 1) if the risks referenced in the letter are real, which they certainly are. There has been informed thought on this for more than a decade. Recent LLMs have made the dangers even more clear. 2) is the letter's proposal of a 6 month pause useful. It's the most useful and plausible step I can think of. We need to take stock. It's not up to a handful of researchers to endanger the lives of 6B other people. 3) Is it actually plausible that we could get such a pause. We'll see. I doubt it but also search "death with dignity" by Yudkowsky.

Back to Musk as an "authority", the headline is about him but more than 1000 other concerned parties have signed. I will sign. Their signature vetting process is stalled because to the volume of signatures, otherwise there would be many more.

Regarding Chomsky's characterization of LLMs "gorging on hundreds of terabytes of data" compared to the "miniscule data" required for a child to learn a language: The brain is evolved, so the the "gorging" already happened in animals for 100s of millions of years. The brain has a lot of evolved sequence processing, visual processing, language (the authors are linguists and they admit this though it undercuts their point). Only fine tuning of this pretrained model is needed for a child to grow up speaking, say, English vs. French. This requires only a relatively miniscule amount of data.

Moreover, it doesn't matter that LLMs work differently from the human brain. Per Larry Wall, TIMTOWTDI ("There is more than one way to do it").

ChatGPT does not think, know, care, intend, speak, commnuicate, etc.

These are statements of faith. They can't be proven or disproven by any experiment. "I think therefore I am." applies to just me. I'll never know whether you have intentionality or not. But I have faith that you do. Maybe you have faith that ChatGPT doesn't. I'm agnostic!

Chomsky's thinking is a form of vitalism: that biological matter possesses a special life force or energy that nonbiological matter doesn't. Well, vitalism eventually went out of style.

This dude is a riot! Wish I could write like that! The overall concept is hysterical, article has great momentum, and many zingers throughout.

Yes, the topic of the article is human extinction, and one can be pedantic and say, why stray off topic? However, conceptually close is near-extinction, with unfathomable suffering and waste, destruction of nearly all human culture, nearly all humans, and nearly all plant and animal life. It's worth a sentence or two to mention this in the article, to avoid confusion on the part of the reader that the author wishes minimize the negative effects of nuclear war - which would perhaps make it more acceptable in a way.

That said, the analysis seems sound, and despite the chance that the conclusion could be misconstrued or misused, it is important to remind voters graphically of the horrors of nuclear war. That's a better strategy to avoid both extinction and near-extinction, rather than counting on military planners to not overshoot their goals of near-extinction.

ZipCam | Multiple Positions | Full-Time | Remote (or Palo Alto) | www.zip.cam

ZipCam makes a smart, connected, automotive dashcam for driving safety. In the U.S., more than 37,000 lives per year are lost in automobile accidents. Worldwide, an unbelievable 1.25 million people die from car crashes annually. We don't need full L5 autonomous cars to save these lives: we can add computer vision and machine learning to existing vehicles, to save lives today.

ZipCam is seed-stage and well funded by angels. Multiple positions available, all full-time, either Onsite (Palo Alto) or Remote are OK:

* Midlevel or Senior Machine Learning Engineer. We do neural network analysis of driving video clips: lane-keeping, accident "near miss" detection, sign reading, stop light classification, stop line detection, other driving tasks. Also with a driver-facing camera: classification of various kinds of distraction (cell phone use, etc). You should have experience running accelerated ML models on video data. Experience with IoT is a plus. Experience with low-power (embedded) inference is a plus.

* Midlevel or Senior Embedded Systems Software Engineer. We're looking for experience with video streaming, intermittent data connections, remote firmware updates, IoT security. Ambarella experience a plus... accelerated NN experience on Ambarella even better.

* Midlevel or Senior Full-stack (database + web) engineer. AWS (incl. Redshift) experience is a plus, security mindset is important. The scale of video data will be: 15TB/mo this year, 20x that in 3 years; please mention the scale of data you have handled before. For web: python on server (just a preference) + whatever client-side js framework you like.

Management experience is a plus for all of the above: we are a small, growing startup so it will be hands-on architecting & coding at the beginning; then hiring more folks.

Drop us a line to learn more about the product roadmap, it's exciting. This is a historic moment for real-world ML applications. Please send your resume + linkedin & github URLs to jobs - at - zip - dot - cam. Please include relevant publications or mention relevant projects you have done. Looking forward to speaking with you. In your email subject please mention "ML", "embedded", or "full stack" + " job".

P.S. We have free office pop-tarts.

I will add this to my HN "who's hiring" post

Have to try this now!