crypto420

Carlini

CNNs excel in vision tasks where you have limited compute, limited memory, limited data, and want something that works super well and quick. People usually don't hook CNNs up to a transformer to get language understanding either, you have to train bespoke CNNs for specific tasks

ViTs excel where you're unbounded in compute + data and also want text understanding or have a conversation about an image

I spent part of my life growing up in a major Asian city where communal ties are very strong, and spent a very significant fraction of my childhood outside of my house, just because there was so much to do - whether it was playing soccer, playing hide and seek, or taking the local bus/train with friends to shows, coffee shops or restaurants. All of this was before cell phones. If I got lost or needed help I'd ask a stranger for directions or navigation.

Nothing bad ever happened to me, and most people were actually quite kind and helpful to a 14 year old kid asking for help and directions. That sense of confidence in myself - that I could figure out how to pretty much do anything - has stuck with me in a way that a lot of my friends who grew up in sanitized suburban neighborhoods just don't have.

Kids really don't know what they're missing.

My PhD advisor found out that my English writing skills are quite good and the rest of their lab were Chinese internationals, so they started making me write all of their research grants. 30 pages every 2 months, pre-ChatGPT days.

Unfortunately, consciousness is deceptively hard to define, and so any benchmark to measure or quantify it can be endlessly debated.

You can argue that it's a property that all living beings have in common - and even among *unconscious* beings there's a form of consciousness and self-awareness that's ever present, but definitions are elusive and vague and tough to pin down.

The mechanistic argument against LLMs - that they're just matrix multiplications - breaks down because they can clearly pass the Turing Test which was the gold standard for what intelligent behavior really meant, thus breaking the old notion that intelligence has to have some form of biological basis. Yet its clear that there are forms of intelligence that rats have which the frontier LLMs don't possess (is that consciousness? or a different kind of intelligence), and its hard to pinpoint what exactly that is, so we probably need the philosophy departments of major universities to come up with newer definitions of intelligence and consciousness.

I personally believe that intelligence and consciousness are 2 separate forms of emergence from simple automata that may occur together (such as in humans) or not (such as consciousness in plants and intelligence in LLMs)

There hasn't been real product innovation at Apple in over a decade.

Everyone says this but forgets that AirPods were released in 2016, Pros in 2020, and they're now the most popular headphones in the world

Well, both, obviously.

So, 95 with 1k?

Peter Thiel and Mamdani are more alike than you may think.

https://x.com/aphysicist/status/1937879912221667792

https://finance.yahoo.com/news/billionaire-peter-thiel-warns...

Nobody wants to hear this because it departs from the 'billionaire bad' trope. But Thiel has been remarkably consistent in his criticism of housing being the center of all of the Millenial economic woes.

Not Palantir. Their CEO has explicitly said that they don't collect data on US citizens.

https://www.youtube.com/watch?v=Y-IH7EVrBbQ

The whole point of Apple intelligence is that it shouldn’t feel like you’re talking with an LLM, it’s that it fades into the OS but delivers new features that make the OS more intelligent. I’m actually very bullish on it

All of this, as well as the crazy weird behaviors by o3 around it's hallucinations and Claude on deceiving users - is pointing to an interesting quote I saw about scaling RL in LLMs: https://x.com/jxmnop/status/1922078186864566491

"the AI labs spent a few years quietly scaling up supervised learning, where the best-case outcome was obvious: an excellent simulator of human text

now they are scaling up reinforcement learning, which is something fundamentally different. and no one knows what happens next"

I tend to believe this. AlphaGo and AlphaZero, which were both trained with RL at scale, led to strategies that have never been seen before. They were also highly specialized neural networks for a very specific task, which is quite different from LLMs, which are quite general in their capabilities. Scaling RL on LLMs could lead to models that have very unpredictable behaviors and properties on a variety of tasks.

This is all going to sound rather hyperbolic - but I think we're living in quite unprecedented times, and I am starting to believe Kurzweil's vision of the Singularity. The next 10-20 years are going to be very unpredictable. I don't quite know what the answer will be, but I believe scaling mechanistic interpretability will probably yield some breakthroughs into how these models approach problems.

I'm not sure if people here even read the entirety of the article. From the article:

We applied the AI co-scientist to assist with the prediction of drug repurposing opportunities and, with our partners, validated predictions through computational biology, expert clinician feedback, and in vitro experiments.

Notably, the AI co-scientist proposed novel repurposing candidates for acute myeloid leukemia (AML). Subsequent experiments validated these proposals, confirming that the suggested drugs inhibit tumor viability at clinically relevant concentrations in multiple AML cell lines.

and,

For this test, expert researchers instructed the AI co-scientist to explore a topic that had already been subject to novel discovery in their group, but had not yet been revealed in the public domain, namely, to explain how capsid-forming phage-inducible chromosomal islands (cf-PICIs) exist across multiple bacterial species. The AI co-scientist system independently proposed that cf-PICIs interact with diverse phage tails to expand their host range. This in silico discovery, which had been experimentally validated in the original novel laboratory experiments performed prior to use of the AI co-scientist system, are described in co-timed manuscripts (1, 2) with our collaborators at the Fleming Initiative and Imperial College London. This illustrates the value of the AI co-scientist system as an assistive technology, as it was able to leverage decades of research comprising all prior open access literature on this topic.

The model was able to come up with new scientific hypotheses that were tested to be correct in the lab, which is quite significant.

So, I'm not a believer in the classic winner takes all approach here where one company turns into this trillion dollar behemoth and the rest of the industry pays the tax to that one company in perpetuity.

I agree with this sentiment. There are a lot of frontier model players that are very competent (OpenAI, Anthropic, Google, Amazon, DeepSeek, xAI) and I'm sure more will come onboard as we find ways to make models smaller and smaller.

The mental framework I try to use is that AI is this weird technology that is an enabler of a lot of downstream technology, with the best economic analogy being electricity. It'll change our society in very radical ways, but it's unclear who's going to make money off of it. In the electricity era Westinghouse and GE emerged as the behemoths because of their ability to manufacture massive turbines (which are the equivalent of today's NVIDIA and perhaps Google).

I wonder if its just about having competent adversaries? The West has always consisted of near-powers competing for resources and technology. A notable exception is modern history, where America hasn't had a serious technological adversary since the Space Race in the 1960s, and we've sort of slumped into this malaise. China appears to be a serious adversary now, but I don't think many Americans seriously possess the will or the skills to take on China.

It most definitely does.

If money is a medium for people to convert goods and services from one form to another, then in a technologically advancing world where things become cheaper in terms of SI units, we should expect the value of money (relative to other items) to become higher, or for goods and services to become so cheap that we don't even bother about their prices.

In a technologically stagnating economy, we can expect the opposite: we can expect that if technology becomes worse off, then the value of money will get worse and the same physical item will become less valuable i.e will cost more in money. We see this happening with oil prices which are at the heart of energy intensive economies - gasoline is MORE expensive than in the 1970s even when adjusted for inflation. Same goes for other goods central to the economy, such as housing, healthcare and food. The more likely outcome is that to artificially stimulate the economy, central banks will keep printing more money and devalue goods in a never ending cycle. The only things that can really save us is technology that can make things cheaper.

https://www.usinflationcalculator.com/gasoline-prices-adjust...

It seems that Peter Thiel's ideas of a technologically stagnating society are making more and more sense.

Not sure what you're talking about. Neural networks are powering a LOT of the internet.

Siri/Cortana/Google Home/Alexa, powered by DeepSpeech+language models

Google Search, powered by BERT

Tesla, powered by variants of YOLO

Facial recognition powered by MTCNN+FaceNet

AirBnB product search+recommendations

Amazon product recommendations

GANs are a bit artsy, but JFC they're not even a decade old - we've gone from shitty MNIST clones to fully synthetic faces in the span of 5 years!

Predictions: 1. I suspect we're going to see DeepFakes in Hollywood - famous people might license their faces to movies that they might not have the time to star in

2. People are going to start building even more powerful versions of search, like combinations of CLIP

3. Neural networks still aren't optimized for edge devices - we're going to see a deluge of cheap drones with cutting edge computer vision by default