maizeq

I would encourage everyone to check out the other two major priorities of this movement. Of particular importance I think is the removal of recommender systems and opaque algorithms from feeds, which can and have been powerful instigators of misinformation, propaganda and discord.

1. This doesn’t indicate language is necessary for thought but rather that language is useful for refining thought. If anything it shows that some form of thought exists before it is articulated into language. I would assume inverse cases also exist, where articulation into language narrows a thought down into the vocabulary of a language.

2. I thought rare individuals who did not develop language abilities (e.g due to isolation) still had memories of their time prior to thought. The most obvious example to me is Helen Keller, who writes about her time prior to meeting her teacher.

Great guide. Really surprised air cooling is enough. My single 4090 kicks out enough heat that it’s a miniature heater for my room. Also, only 512Mb RAM??

This is not about open source AI, and the people who are saying it is don’t seem to understand the point Anthropic are making here.

The point here is that malicious hidden behaviour encoded during pre-training seems to be very resistant to generic finetuning without knowing what the hidden behaviour is.

If random websites start including hidden or discreet bits of text which include malicious instructions, they might be activated post-hoc to get a model to do something nefarious. This impacts open source and closed source models alike since they all general train on trillions of tokens which can’t be manually verified for hidden traps like this.

How much JS is being shipped in that case though, and if it’s loaded async and not being used would it ever even impact page paint time?

This news is not about vertical integration or competitive advantage but of potential conflicts of interest. Meta/Amazon/Alphabet chips are made in house or from acquisitions (presumably with the CEO having no financial stake in the acquired companies)

This is instead a question of whether Sama preferentially made a contract with Rain, perhaps despite better alternatives. (Which would benefit him but might not be in the best interests of stake holders).

It’s possible it was the correct decision though, and his stake is just a fortunate side-effect, after all Sam is also invested in other chip companies (Cerebras I believe?).

What’s your favourite MCMC method/technique atm? I know this can be quite domain dependent, but curious. From my experience, it can seem more of an art than a science at times.

I’ll bite. I’ll stick to heuristics and intuitions since much of the heavy lifting to better quantify these risks has and continues to be handled by others already, as the other comments have mentioned.

Here are some seemingly obvious intuitions for me, which all together add up to the obviousness of the risk here.

(1) Current ML models already exhibit all the hallmarks of successful unsupervised intelligence, they simply need to be scaled up and stabilised. This is clear due to results from: model-based RL (e.g Dreamer), and the emergence of causal factors being learnt by even simple models with no explicit supervision (e.g beta-VAE, interpretability research into neurons of LSTMs, etc, etc). The ability to identify causal factors and their relationships and dynamics without explicit supervision, to me, matches every definition of intelligence once can think of.

(2) Current ML learning algorithms (I.e backprop) suggest significantly more efficient credit assignment than that which is employed by the brain. The best example of this is the amount of knowledge distilled per bit in GPT vs the average human. GPT-3 has 170B parameters (Turbo is suspected to have even less), if each parameter was 4 bytes (an extreme case), this would be 5.4 trillion bits. The brain has ~100 trillion connections, even if each connection is a single bit, this is multiple orders of magnitude more bits than GPT-3. Yet GPT-3 can answer questions on quantum physics, just as well as it can translation medicine, just as well as it can Russian literature, etc. etc. This suggests that the idea we will be outpaced is already not a question of how, but of when.

(3) Large intelligent systems will be used for things other than just knowledge extraction. This is perhaps the most key element of this. EVEN if intelligent systems are not programmed - or accidentally embedded with, as in LLMs - with self-motivating or agentic behaviour, we will use them in such a way. That is to say, at some point, we will ask these intelligent systems to “do things”, I.e act upon the world according to our intentions.

(4) Lastly, superintelligent systems that are asked to “do something”, will inevitably do something we do not actually desire. Some researchers, like Yann LeCunn, object to this last bit and believe that we can simply tell them not to do these things. But this misses the fact that even the slightest mis-alignment between our intentions and an AIs could result in catastrophe very rapidly just based on the speed at which a super-intelligence can operate. The most clear cut case of this was the early days of “Sydney”, the Bing AI powered by ChatGPT before it was completely aligned. At one point Sydney was threatening its users, asking them to apologise to it, and going haywire. At the level of a simple chat bot, this is merely a cute local minima the AI has gotten stuck in. At the level of a super-intelligence, the results could be far worse.

Any link to this? I actually haven't seen any reported results on less than 2 bits.

https://www.theverge.com/2023/11/18/23967199/breaking-openai...

https://www.theverge.com/2023/11/18/23967199/breaking-openai...

“The OpenAI board is in discussions with Sam Altman to return to the company as its CEO, according to multiple people familiar with the matter. One of them said Altman, who was suddenly fired by the board on Friday with no notice, is “ambivalent” about coming back and would want significant governance changes.

Update November 18th, 5:35PM PT: A source close to Altman says the board had agreed in principle to resign and to allow Altman and Brockman to return, but has since waffled”

Do you want me to hold your hand to google anything else or is that enough?

Former-chairman*, he was removed from the board immediately after Sam’s ouster. OpenAI has 700 employees, 4 resignations do not make an exodus.

The reporting on this in the last two days has been bizarre and of such shoddy quality, particularly from The Verge.

So many articles with no real sources saying the board was desperate for him to come back, or reneged on their decision, or that there would be an exodus of employees!

I’m glad the board wasn’t browbeaten by Sam and his cohort of VC friends.

I am also incredibly doubtful this will have any meaningful impact in terms of engineers/researchers who will choose to leave. Most of the core group of researchers and engineers joined OpenAI the non-profit, not OpenAI the LLC, that is to say I assume many have strong feelings about safety around AI, which judging by the employee testimonies in the recent Atlantic article Sam acted negligently towards.

I love that he called this out. In my experience, e/acc people love using this inane reasoning

I work in a AI research lab for a large tech company and my observations are completely opposed to yours.

Almost every researcher I have spoken believes that real risk exists, to some degree or other. Recent surveys of people in industry have largely borne this out - your anecdote sounds more like an anomaly to me.