Things I don't know about AI

https://blog.eladgil.com/p/things-i-dont-know-about-ai

by todsacerdoti • 2 years ago

248 92 2 years ago

blog.eladgil.com

eladgil3• 2 years ago

Would love to get any views / counterpoints on these topics.

Eg which AI infra approach thrives relative to the big clouds?

What new architectures are needed for agents vs transformers?

Etc...

Lots of stuff to figure out....

texas2toss• 2 years ago

Do you think there is the possibility of consumer or end-user apps collecting enough specialized data to move downwards on your graph to infra and foundational models?

eladgil3• 2 years ago

I think this is already true for things based on diffusion models - e.g. Midjourney, Pika etc

For LLMs, Character and ChatGPT are arguably two vertically integrated consumer apps (with some B2B applications for ChatGPT as well)

danielmarkbruce2• 2 years ago

There is a decent (<50%, >20%) chance that frontier foundation models are less oligopoly like than it seems. The reason is that there are so many levers to pull, so much low hanging fruit.

For example: * Read the Bloomberg GPT paper - they create their own tokenizer. For specialized domains (finance, law, medicine, etc) the vocabulary is very different and there is likely a lot to do here, where individual tokens really need to map to specific concepts and having a concept capture in several tokens makes it too hard to learn on limited domain data. * Data - so many ways to do different data - more/less, cleaner, "better" on some dimension. * Read the recent papers on different decoding strategies - there seems to be a lot to do here. * Model architecture (SSM etc). If you speak to people who aren't even researchers, they have 10 ideas around architecture and some of them are decent sounding ideas - lots of low hanging fruit. * System architecture - ie likely to see more and more "models" served via API which are actually systems of several model calls, and there is a lot to do here. * Hardware, lower precision etc likely to make training much cheaper

It's reasonably likely (again, guessing < 50% > 20%) that this large set of levers to pull become ways to see constant leap-frogging for years and years. Or, at least they become choices/trade-offs rather than strictly "better".

eladgil3• 2 years ago

I agree this is a potential outcome. One big question is generalizability versus niche models. For example, is the best legal model a frontier model + a giant context window + RAG? Or is it a niche model trained or fine tuned for law?

Right now at least people seem to decouple some measures of how smart the model is from knowledge base, and at least for now the really big models seem smartest. So part of the question is well is how insightful / synthesis centric the model needs to be versus effectively doing regressions....

jchonphoenix• 2 years ago

Arguably one of the earliest consumer use cases that found footing was AI girlfriend/boyfriend. Large amounts of revenue spread across many small players are generated here but it's glossed over due to the category.

notpachet• 2 years ago

I think one related area we'll start seeing more of in the future is "resurrected" companions. You have a terminally ill family member, so you train a model on a bunch of video recordings of them, then you can talk to "them" after they've shuffled off this mortal coil.

rockostrich• 2 years ago

Weird how close your user name is to Joaquin Phoenix, star of the film "Her" centered around an AI girlfriend.

PeterisP• 2 years ago

Given that how widespread romance scam schemes already are (the "market" is at least $0.5 billion/year), I would expect any reasonably functioning AI girlfriend/boyfriend model to be massively (ab)used also against unwilling/unwitting "partners".

doctorpangloss6• 2 years ago

I haven't listen to your great podcasts so hard to say what is not covered.

Architectures matter a lot less than data. "Knowledge" and "reasoning" in LLMs is a manifestation of instruct-style data. It won't matter how much cheaper training gets if there is limited instruct data for use cases.

How do you make 100k context window data for example? Still need thousands of people. Same with so-called niches.

Maybe it turns out to be a complex coordination problem to share the data. That's bad for equity investors and giant companies. Anyway, all of this would cost less than the moon landing so it's practicable, you don't need cheaper, you just need risk-taking.

The obviousness of the path from here to there means it's not about innovation. It's all about strategy.

If Google could marshal $10b for Stadia it could spend $10b on generating 100k context window instruct style data and have the best model. It could also synthesize videos from Unity/Unreal for Sora-style generation. It would just be very hard in an org with 100,000+ people to spend $10b on 10 developers and 10,000 writers compared to 400 developers and 3,600 product managers and other egos. At the end of the day you are revisiting the weaknesses that brought Google and other big companies to this mess in the first place.

Anyway I personally think the biggest weakness with ChatGPT and the chat-style UX is that it feels like work. Netflix, TikTok, etc. don't feel like work. Nobody at Google (or OpenAI for that matter) knows how to make stuff that doesn't feel like work. And you can't measure "fun." So the biggest thing to figure out is how much technical stuff matters in a world where people can be poached here and there and walk out with the whole architecture in their heads, versus the non-technical stuff that takes decades of hard-worn personal experiences and strongly held opinions like answers to questions "How do you make AI fun?"

mywittyname• 2 years ago

"How do you make AI fun?"

Bring back text-based adventure games.

3abiton• 2 years ago

Agents and transformers are kinda different things, built on different layers.

xianshou• 2 years ago

From the article:

"It is important to note that the scale of investments being made by these cloud providers is dwarfed by actual cloud revenue. For example, Azure from Microsoft generates $25B in revenue a quarter. The ~$10B OpenAI investment by Microsoft is roughly 6 weeks of Azure revenue. This suggests the cloud business (at least for now) is more important than any one model set for Azure."

The most interesting implication in the short term is that the impact of model availability on cloud provider choice is more important than the models themselves. Will organizations choose Azure because it offers GPT-4, or AWS because it offers Claude?

This also explains reciprocal VC structures whereby e.g. Amazon invests billions in Anthropic while Anthropic in turn promises to spend billions on AWS cloud resources. Anthropic gets a higher valuation and broader distribution, while AWS gets more customers. It would be a fascinating outcome if the predicted oligopoly in fact ends up as several exclusive partnerships.

infecto• 2 years ago

Doubtful, at the end of the day the models are just like any other API. No reason not to signup for all the big players (GCP/Azure/AWS) and have access to the various models. Should have no weight as to where you should build out your infra.

simpletone• 2 years ago

The ~$10B OpenAI investment by Microsoft is roughly 6 weeks of Azure revenue.

Revenue isn't what is important. Profit is since profit is the money you have after costs to invest. But since cloud is highly profitable as it is, the point still stands I suppose.

It would be a fascinating outcome if the predicted oligopoly in fact ends up as several exclusive partnerships.

I thought people were predicting independent silos? Where highly profitable and cash rich companies like apple, microsoft, google, facebook, etc own their entire stack or exist as their own ecosystem. Where the likes of apple does everything in-house - AI, cloud, OS, chips, etc. The whole she-bang.

bbor2• 2 years ago

Opened the comments for exactly this. This person seems very nice but also convinced that they’re a business maven, which I think leads them to making “clear” observations like that. Microsoft hasn’t invested a full year of revenue into OpenAI, so it’s not that invested? Meh

jjjjj55555• 2 years ago

Stupid question, but where would their competitive advantages be in these silos? Sounds like at that point cloud+AI is just a commodity.

jffry• 2 years ago

The relative scale still stands - Microsoft's net income in October, November, and December 2023 was over $21 billion, according to https://www.microsoft.com/en-us/Investor/earnings/FY-2024-Q2

bane3• 2 years ago

The confusion that everybody feels right now about "what do I do in this rapidly evolving space" is likely more an artifact of one of the most vertical lead-edges to a hype curve in a generation. There is so much piling on, at absolutely tremendous investment amounts, that it's not really even possible to pick a launch point to a business plan that would survive even months at this point.

The second problem is more obvious, but not talked about in concrete terms yet. As these technologies mature, the legal, policy, and social constructs around them will also mature, and will shape the future of what is coming as much as any technical or business decision. Some examples:

- Imagine if courts really find that these models are in gross copyright violation and that model builders will need to work through a scheme similar to sampling in music in order to do their work into the future. Not only will it dramatically increase the cost of model development, but will drive some of the work to alternative sources of data if it's cheaper. What effect will this have on these models?

- Suppose these models are found to be reliable enough for certain mission critical work (healthcare, defense, etc.) and they lead a user down a path that results in harm or death? Who's at fault? Suppose it becomes a liability of the company offering the model? How does the insurance market respond?

- What if these models become good enough, and the automation around them as well, that we start turning them into fully automated agents that make up a significant part of our economy? Do people still have jobs? Or will these things just become the equivalent of hiring more people into an economy and we work in a blended AI/Human Intelligence world? What happens when these things decide to tank capital markets or choke supply chains?

- What about the long-term use of these models by individuals with severe mental impairments or psychological issues? What happens when those models go down, or produce a response that's not liked, or there's a loss of historical conversational data? What about otherwise normal people who end up down a path of social isolation?

- There's an entire legal market about to be created for remote-work fraud where somebody just replaces themself with one of these models for 90% of their work. It's sci-fi utopia, but fraudulent if you misrepresent who's doing the work in many cases.

- Some company, somewhere, will crack the nut on figuring out how to improve the speed and performance of these models, and build custom hardware that offloads the entire effort onto rapidly commoditizing systems. When multi-modal LLMs and diffusion models come preloaded on disposable $.10 SOCs that are put on cereal boxes and in Birthday cards, we'll be well beyond every one of these questions. But we'll be firmly in a different future.

We have yet to encounter most of these things to the point that as a society we have to build responses, and that itself points to the immaturity of both these technologies and the use cases for which they will be adopted.

Which cloud to pick and why aren't consumers spending thousands of dollars getting the complex systems in place to use this stuff aren't even the problems that are going to become interesting in the next 10 years.

xamuel• 2 years ago

Suppose these models are found to be reliable enough for certain mission critical work (healthcare, defense, etc.) and they lead a user down a path that results in harm or death? Who's at fault? Suppose it becomes a liability of the company offering the model? How does the insurance market respond?

So much this. I think a lot of people overlook this aspect so much. I wonder if we'll see a corporate push to make people think of individual instantiations of LLMs as being their own entities, for scapegoating purposes. It's completely ludicrous, but you know they'd just love to point a finger at Machine #73583 and make that take all the blame.

chasd00• 2 years ago

corporate push to make people think of individual instantiations of LLMs as being their own entities, for scapegoating purposes

an airline already tried this.

"Air Canada argues it cannot be held liable for information provided by one of its agents, servants, or representatives – including a chatbot. It does not explain why it believes that is the case. In effect, Air Canada suggests the chatbot is a separate legal entity that is responsible for its own actions."

https://www.forbes.com/sites/marisagarcia/2024/02/19/what-ai...

shermantanktop3• 2 years ago

What happens when AI-generated content organically becomes a large percentage of the user-generated corpus that is used for training, with no way to differentiate it from human-generated content?

We may look back on 2022/3 as the last time we had a training set that was clean. The fact that it was already polluted with SEO garbage will be a quaint problem.

bane3• 2 years ago

The current thinking is that it becomes a negative feedback loop, a kind of lossy "compression" of our cultural output. Combined with normal bitrot we'll eventually start losing components of our shared culture until civilization ends.

ericvsmith• 2 years ago

I like to think of it like mad cow: that’s what you get when you feed animals to animals. Same with AI.

citizenpaul2• 2 years ago

I've been thinking about this a lot lately. I guess the 4D chess move is to think about what will be the next market that will benefit from whatever this effect causes.

I have a completely un-worked thought that the music industry might have some clues. As music becomes more and more entrenched in codified process ie pop culture music. It all starts to become very generic.

I'm starting to notice that music is returning to the old way of discovery. Where you hear music from a 3rd party that is playing it and ask them who that is. There is no longer a good bubble to the top source like radio because the corps have cannibalized themselves with production line pop music that has to play on their vertical stack of companies or be licensed. Seems the stuff I find now is some artist with less than 100k followers like (I'm guessing) back in the pre internet days.

somewhereoutth• 2 years ago

- What happens if everyone realises that the areas where LLMs can be usefully applied are few and not particularly lucrative?

bane3• 2 years ago

That's the far right of the hype curve.

rglover• 2 years ago

This is one of those moments where Peter Thiel's "last mover advantage" makes a whole lot of sense.

cs7026• 2 years ago

I would add to the list a few questions about the evolution of cost dynamics going forward, given the advent of new sequence modeling architectures.

As we all know, Transformers are very expensive to train and run because their compute cost is quadratic in context length: O(N²).

If newer model architectures like RWKV, Mamba, and various others, which incur cost that is linear in context length, O(N), prove as successful as Transformers, the demand for compute for a query of N tokens would decline from O(N²) to O(N).

For a sequence with N = 100,000 tokens, it would mean cost dropping by a factor of 100,000×. That's not peanuts!

The implications for all market participants would be significant.

nyrikki6• 2 years ago

In the general case that would be the same as hierarchy collapse of PH, which would prove P=NP

cs7026• 2 years ago

Well, it would depend on how many tokens from the vocabulary are required, in the worst case, to solve NP problems.

Think of the tokens as symbols, and of the vocabulary as a finite alphabet of symbols, in a formal system.

In other words, no, it would NOT necessarily prove P = NP.

dadadad1002• 2 years ago

As in the case of constraint solvers there are tons of heuristics that have been applied over the years to make performance look like it scales somewhat linearly in many or most problems. This doesn’t mean p==np

hansvm2• 2 years ago

One the one-hand, "no". On the other, that's an interesting observation. There are known Θ(N^2) problems, which a linear transformer substitute couldn't possibly solve (in a single pass), and which current transformers might be able to. It's an open question how many of the current LLM capabilities rely on quadratic compute, but linear transformers might be a pipe dream for some current use cases.

Generally though, I don't think that's quite the same. Collapsing all O(N^2) problems down to O(N) would collapse the hierarchy and prove P=NP, but it's blatantly impossible and would prove a lot of falsehoods too.

doctorpangloss6• 2 years ago

The implications [of a bunch of different architectures] for all market participants would be significant.

I don't know, who cares. How much good training data is there that correctly exercises 100k context windows?

All the costs and complexity are tied up in authoring good training data, not compute. 1 person can invent an alternative easier to train architecture. 1 person cannot author 100k context window instruct examples with the same quality as the 4k ones. It will still take thousands of people for 100k just like it took 1000s of people for 4k windows, image labeling, etc.

famouswaffles7• 2 years ago

All the costs and complexity are tied up in authoring good training data, not compute.

No it's compute. Post-Training by Human reinforcement is not necessary. Anthropic employs RLAIF and it works just fine. Some don't bother with reinforcement learning at all and just leave it at fine-tuning on Instruction-response pairs.

The work being done to pre or post-training data is insignificant in comparison.

You don't need 100k instruct-tuning examples. The vast majority of instruct-tuning data is nowhere maxing even a 4k context.

100k pre-training runs would probably be very helpful but the thing stopping that from happening even in domains that regularly match or exceed that context (fiction, law, code, etc) is the ridiculous compute it would require to train with that much context.

araes• 2 years ago

With the Big O notation, are there also possibilities for either the whole process to go to O(nlogn) or possibly subportions to the O(logn) range? There's also a lot of math operations over the years that have gone to O(Nª) where 1 < a < 2.

jxy• 2 years ago

architectures like RWKV, Mamba, and various others, which incur cost that is linear in context length, O(N)

These are actually O(N*M) where M is independent of inference time context length, but it controls the actual amount of information that gets retained during inference. Ignoring that factor of M is misleading.

hansvm2• 2 years ago

All models are wrong. Some are useful.

The "linear" models I'm aware of have a cost of O(N * M), where M is the amount of information retained _per token_ (with some exceptions that store less state but still have intermediate computations for some M about the same size as the others, and with less state you necessarily have forgetting issues and can't scale to unbounded sequences anyway).

Contrast that with a transformer, which has a cost of O(N^2 * M), where M is still the amount of information retained _per token_. Dividing the two you still find a multiplicative speedup of O(N).

For accurate cost estimates or any number of other use cases, yes, the M is critical. When comparing architectures (or when talking about a single architecture with the current implicit conversation happening that whatever you say is meant to be taken with respect to other architectures) it's conventional to note that the M term always appears as a multiplicative factor beyond the rest of the costs. You drop it because it's obviously there but it's just visual noise that doesn't help you understand where the models sit with respect to each other. You only include it if it's relevant, like if you found a way to get similar performance with a smaller M.

It's kind of like how we always drop the log term on a hash table (equivalently, we work in the "word RAM" model of computation), but bignum algorithm papers carefully consider every log(log(log(N))) that crops up. It's relevant in some contexts and not in others. You can't ever include enough context to fully represent your thoughts to another person, so you truncate somewhere, hopefully capturing the essence of what matters.

chillee4• 2 years ago

This is extremely wrong.

The attention component that's quadratic is a relatively small portion of compute.

cochne• 2 years ago

I think they are correct, do you have a source? From my knowledge the only other components are the fully connected networks which are not big contributors.

dTal• 2 years ago

To anyone doubting this, note that llama.cpp does not slow down by a factor of 16 when you pass -c 2048 instead of -c 512.

cs7026• 2 years ago

I'm the author of the parent comment, and just saw your response.

First of all, by "a factor of 100,000×" I mean a model-specific factor, call it c, of 100,000x, but looking at what I wrote I can see how it could be misinterpreted. Second, as sequences become longer, the quadractic component will become more important.

casercaramel1442• 2 years ago

Huh? I thought the issue before ringattention is the memory requirement of the softmax layer, since you have to load the whole matrix in at once? It's O(s^2) no?

Also hi horace.

bbor2• 2 years ago

The Google CEO said that this (unexpected!) invention is more important than fire or electricity. In that light, I think it’s arrogant to try to soothsay like this. Which is tough advice cause this is otherwise great analysis, and acknowledging the great unknown feels scary and of dubious value. But I think it’s the only honest way forward

galaxyLogic• 2 years ago

Would it be possible to start an Open-Source AI project that works like SETI that runs on people's PCs on their spare time and collaborate to train an AI model. People could also help by providing feedback to the models?

haberdasher• 2 years ago

Basically: https://rendernetwork.com/

m0llusk• 2 years ago

Or this one: How is AI not outsourcing all over again? People pay almost as much for work that needs to be checked before it can be relied on. Sure, there is value, but there is also a lot of risk and questionable margins depending on the context.

mr_toad2• 2 years ago

One difference is that we can expect models to improve, become more reliable, cheaper and need less hand-holding.

RyanShook• 2 years ago

Surely we are in an AI bubble not completely unlike the dotcom bubble, right? The promise and potential for AI to be a long-term growth driver exists but the current market euphoria can’t be sustained in the short-term… at least that’s what I’ve told myself every time NVDA jumps and I feel like a loser.

causal• 2 years ago

If it's a bubble, it's early days and not likely to pop soon. If it's a dotcom bubble, that means a road bump for the tech as it continues to take over the world.

Things I don't know about AI

Related Stories