HN user

imranhou

238 karma
Posts3
Comments98
View on HN

This is a very interesting point, in some ways the implicit belief is that we just need to get beyond the 700g limitation in terms of scaling LLM models and we would get human intelligence/superintelligence. I admit I didn't really get the body/brain analogy, I would have been better satisfied with a simpler graph of brain weight to intelligence with a scaling barrier of 700g.

Coming from a layman's perspective, a genuine question regarding: "Implements SAE training with auxiliary loss to prevent and revive dead latents, and gradient projection to stabilize training dynamics".

I struggle to understand this phrase "to prevent and revive ", perhaps this is simple speak to those that understand the subject of SAEs, but it feels a bit self contradictory to me, could anyone elaborate?

What do you think about having codebuff write a parser for javascript? Something that is specifically built to enhance itself that goes beyond the regular parsers and creates a more useful structure of the codebase to be then used for RAG for code writing? This would be double useful as a great demo for your product as well as enhancing your product intrinsically. For example the new parser can not only build the syntax tree but also provide relevant commentary for each method to describe what it does to better pick code context.

From the page: "We also show that our model outperforms AlphaZero's policy and value networks (without MCTS) and GPT-3.5-turbo-instruct."

Why compare this to GPT-3.5-turbo-instruct? Is that near SOTA in this space?

Based on what I've read it is allowed for a non profit to own a for profit asset.

So I'm assuming the game plan here is to adjust the charter of the non profit to basically say we are going to still keep doing "Open AI" (we all know what that means), but through the proceeds it gets by selling chunks of this for-profit entity, so the essence could be the non-profit parent isn't fulfilling its mission by controlling what openai does but how it puts the money to use it gets from openai.

And in this process, Sam gets a chunk (as a payment for growing the assets of the non-profit, like a salary/bonus) and the rest as well....?

There is a difference between Peak Oil "Output" vs Peak Oil "Demand". The article is about demand and not production capacity which won't matter once the demand starts to go downhill... will never be zero but will not be something that can prop up regimes that have sucked the life out of their people for more than half a century.

There are multiple seats per row and only the first person of that row may benefit from the concurrency, as each person gets into the isle for that row they will add those seconds and that is sequential not concurrent. Also you likely haven't been in a trampling crowd situation, bag on a chest doesn't occupy zero space.

From the summary, it is unclear if this study takes into account human behavior vs a dummy when the hands are on top of the front seat back. While a dummy may not hold the front seat with a tighter grasp with its hands, a human during impact may or may not change the outcome through either pushing the front seat out or pulling it backwards which can directly alter the outcome of the study.

This is more evidence that the man wants to gain more than his fair share of power and ofcourse the money. Working with such people is no fun as they do not respect boundaries of power and responsibility.

It is baffling to see Ilya and Murati did a U turn, perhaps they didn’t want him fired and hoped they could change him, such people are unlikely to change their behavaior, its an innate trait that cant be “trained out”.

The rest of the employees probably don’t work with him close enough to know the kind of person he is, plus the money factor!

I think what he is trying to say is they are compromising their underlying goal of being a non-profit for the benefit of all, to ensure the survival of "OpenAI". It is a catch-22, but those of pure intentions would rather not care about the survival of the entity, if it meant compromising their values.