Deegy

So what's the business strategy here?

Google is the only USA based frontier lab releasing open models. I know they aren't doing it out of the goodness of their hearts.

Is it though? Do we still have the expectation that LLMs will eventually be able to solve problems they haven't seen before? Or do we just want the most accurate auto complete at the cheapest price at this point?

And now there are 1% of the number of farmers that there used to be

Exactly. It's a lot easier to prompt an LLM when you spent a year understanding the problem.

I appreciate the response but I'm not sure I can agree with 'nothing will happen...'

When I have 275lbs on my back I'm very anxious that any lapse in focus could cause major injury to my knees, back, etc.

They're increasing reps and therefore total load. That's still a form of progression ('pushing yourself'). This style will slightly favor hypertrophy gains over strength gains.

At 40 I recently made this switch in style as well. The weight was getting so high that my anxiety was causing a mental aversion to working out altogether. Consistency is really 95% of exercise so I think this is a reasonable trade-off.

That said, I understand where you are coming from. There's something to be said about facing the fear of the weight head on. I've already done that in my younger years though. I'd much rather avoid injury and get 80% of the benefits.

Extensive tailwind training data in the models. Sure there's something more efficient but it's just safer to let the model leverage what it was trained on.

Professional legal services seem to be picking up steam. Which sort of makes sense as a natural follow on to programming, given that 'the law' is basically codified natural language.

It is unclear. Everyday I seem to read contradictory headlines about whether or not inference is profitable.

If inference has significant profitability and you're the only game in town, you could do really well.

But without regulation, as a commodity, the margin on inference approaches zero.

None of this even speaks to recouping the R&D costs it takes to stay competitive. If they're not able to pull up the ladder, these frontier model companies could have a really bad time.

We currently have human-in-the-loop AGI.

While it doesn't seem we can agree on a meaning for AGI, I think a lot of people think of it as an intelligent entity that has 100% agency.

Currently we need to direct LLM's from task to task. They don't yet posses the capability of full real world context.

This is why I get confused when people talk about AI replacing jobs. It can replace work, but you still need skilled workers to guide them. To me, this could result in humans being even more valuable to businesses, and result in an even greater demand for labor.

If this is true, individuals need to race to learn how to use AI and use it well.

They know that LLMs as a product are racing towards commoditization. Bye bye profit margins. The only way to win is regulation allowing a few approved providers.

That would be true in a monopolistic market. But these frontier models are all competing against each other. The incentive to 'just work and get shit done fast' is there as they each try to gain market share.

Not required of course. This move to finally move some of his cash horde into equities could be a result of the US governments recent announcement that they're ending their quantitative tightening policy. This likely signals a move back to a period of quantitative easing which could cause assets to continue to inflate while the value of the dollar continues to erode.

Basically he could just be trying to protect the value of his dollars by putting them into a very stable company that also has exposure to the upside of AI.

You don't even know what a write-off is do you?

I'd call text the most versatile interface, but not sold on it being the ultimate. As the old saying goes 'a picture is worth a thousand words' and well crafted guis can allow a user to grok the functionality of an app very quickly.

Happened with the electrical grid too.

I think you make a very interesting observation about these bubbles potentially being an inherent part of new technology expansion.

It makes sense too from a human behavior perspective. Whenever there are massive wins to be had, speculation will run rampant. Everyone wants to be the winner, but only a small fraction will actually win.

Good point. But can the models even behave that way? They depend on probability. If they put a greater weight on novel/unexpected outputs don't they just become undependable hallucination machines? Despite what some people think, these models can't reason about a concept to determine it's validity. They depend on recurring data in training to determine what might be true.

That said, it would be interesting to see a model tuned that way. It could be marketed as a 'creativity model' where the user understands there will be a lot of junk hallucination and that it's up to them to reason whether a concept has validity or not.

I believe I addressed that in my third paragraph?

It does suck that there are only a few companies with enough resources to offer these models. But it's hard to escape the power laws.

I'm hoping that costs come down to the point where these things are basically a commodity with thousands of providers.

I guess I'll take the other side of what most are arguing in this thread.

Isn't it a great thing for to us to collectively allow LLM's to train on past conversations? LLM's probably won't get significantly better without this data.

That said I do recognize the risk of only a handful of companies being responsible for something as important as the collective knowledge of civilization.

Is the long term solution self custody? Organizations or individuals may use and train models locally in order to protect and distribute their learnings internally. Of course costs have to come down a ridiculous amount for this to be feasible.

If your work was truly novel, wouldn't the odds of it showing up in later models be extremely low given that these are probabilistic?

In a sense these machines are outputting the aggregate of the collective thoughts of the commons. In order for concepts to be output they have to be quite common in the training data. Which works out kind of nice for privacy and innovation because by the time concepts are common enough to show up through inference they probably deserve to be part of the public knowledge (IP aside).

This looks coveted af

If google had to face the reality that distilling their search engine into multiple case-specific engines would have resulted in vastly superior search results, they surely would done (or considered) it.

Fortunately for them a monolith search engine was perfectly fine (and likely optimal due to accrued network effects).

OpenAI is basically signaling that they need to distill their monolith in order to serve specific segments of the marketplace. They've explicitly said that they're targeting STEM with this one. I think that's a smart choice, the most passionate early adopters of this tech are clearly STEM users.

If the tech was such that one monolith model was actually the optimal solution for all use cases, they would just do that. Actually, this is their stated mission: AGI. One monolith that's best at everything is basically what AGI is.

I've also noticed that with cGPT.

That said I often run into a sort of opposite issue with Claude. It's very good at making me feel like a genius. Sometimes I'll suggest trying a specific strategy or trying to define a concept on my own, and Claude enthusiastically agrees and takes us down a 2-3 hour rabbit hole that ends up being quite a waste of time for me to back track out of.

I'll then run a post-mortem through chatGPT and very often it points out the issue in my thinking very quickly.

That said I keep coming back to sonnet-3.5 for reasons I can't perfectly articulate. Perhaps because I like how it fluffs my ego lol. ChatGPT on the other hand feels a bit more brash. I do wonder if I should be using o1 as my daily driver.

I also don't have enough experience with o1 to determine if it would also take me down dead ends as well.

Bubble's don't always imply fraudulent underlying tech.

The dot com bubble was a real thing, and yet the internet has gone on to be one of humanities most valuable innovations.

Usually, the words 'astroturfing' and 'propaganda' aren't reserved for describing the marketing strategies of valuable products/ideologies. Maybe reconsider your terminology.

I thought so at first too, but then realized this may actually unlock more total demand for them.

I think this is the correct take. There might be a small bubble burst initially after a bunch of US stocks retrace due to uncertainty. But in the long run this should speed up the proliferation of productivity gains unlocked by AI.

It's mostly a decision to manage the total amount I spend on LLM tools. Given unlimited money I suppose I'd still be subscribed to Perplexity because the UI is slightly better than Claude and chatGPT for web results. But Claude and chatGPT are plenty enough for my web use cases while allowing me full access to all of their models for non web search use cases.

Once chatGPT and Claude (through MCP) added web search functionality I completely dropped Perplexity. I assume I'm not unique in this regard. Feels like the writing is on the wall for Perplexity.

I mean, they might make back the $4b on the value it brings to programming alone.