airspresso

OpenAI's upcoming mega IPO

Time for me to go research the early history of electrification.

The Stepchange podcast has an amazing episode on The Grid [1], walking us through the arc of history of how it became the utility it is today.

[1] https://www.stepchange.show/grid

posting this sentence was part of the deal to get the compute

This 100%

but given that you can actually run the models yourself on AWS Bedrock

That's not exactly how it works. Anthropic are hosting their models in AWS Bedrock as a managed service. Customers call those LLMs just like calling any other API. There's no visibility into what kind of AWS infrastructure is serving that API request.

Fun that it has oscillated from instant boot then to minutes-long boot a decade later back to instant boot (or resume to be fair) today.

I had the same confusion and closed the tab, only to discover here on HN that there's more. Open by default sounds reasonable.

There is a surprising amount of code needed in each of the inference frameworks (LM Studio, llama.cpp, etc) to support each new model release. For example to format the input in the right way using a chat template, to parse the output properly with the model-specific tokens the model provider decided to standardize on for their model, and more.

This particular instance was a fix to the output parsing [1] in LM Studio, described like this:

"Adds value type parsers that use <|\"|> as string delimiters instead of JSON's double quotes, and disables json-to-schema conversion for these types."

[1] https://github.com/ggml-org/llama.cpp/pull/21326/commits/a50...

edit: formatting

I think we lost that terminology war. Open source models mean open weight. There are only a couple examples of fully open source models with open data and code, and the labs are not incentivized to go that far.

That's a good point. Charging stations benefit from being a service station too though, with amenities and a cafe etc, since people want something to do while they charge. So a gas station is a better candidate than a parking lot when decisions are made for where to place the new charging infrastructure. Lots of other factors too of course.

Yes, retooling gas stations is the way to go. Already happening in Norway where stations now show the price of kWh in addition to gas and diesel prominently on signs by the road. Charging is just a different kind of pump.

America certainly did not invent electric cars. Depending on which electric car you consider the first real one, the inventor was either French, British or German [1].

[1] https://en.wikipedia.org/wiki/History_of_the_electric_vehicl...

Unsloth is providing the best and most reliable libraries for finetuning LLMs. We've used it for production use-cases where I work, definitely solid.

Also wrestling with this challenge at the moment and curious to hear experiences from others. Even though it requires human input, the capture and the way it's updated has to get automated.

Spend time building a test harness and evaluations of whether the solution meets the requirements. Then you don't need to look at the code because those other pieces will bring the necessary guarantees and trust.

Because we all prefer it over Gemini and Codex. Anthropic knows that and needs to get as much out of it as possible while they can. Not saying the others will catch up soon. But at some point other models will be as capable as Opus and Sonnet are now, and then it's easier to let price guide the choice of provider.

Still do. Great for workloads where it's okay to bundle a bunch of requests and wait some hours (up to 24h, usually done faster) for all of them to complete.

Thank you, was confused there for a second XD

You’re ignoring Jevons paradox. Everyone, both people and companies, will be making exponentially more software with these tools. Software that both needs to get created, debugged and updated to realize the intention of it. That’s what our time will be spent on as programmers.

Apple has a solid hardware business and massive profits from their App Store tax, they are not dependent on ad business in the way Google is. Very different incentives.

Sounds neat but what kind of range limits would that impose on each trip? Switching from one means of transportation to another, even if both are buses, increases the total travel time significantly. Not to mention all the hassle involved for passengers.

This 100%. It should be seen as critical infrastructure because of everything it can enable when run well.

That's how I read it XD "oh no, RL is dead too"

Sounds like you're fighting the weights. What would it take to align the setup with what the LLM expects?

Hard disagree. Python is so simple anyone can get up and running with coding in a few lines in the REPL.

Hallucinations and output quality are two different problems. Hallucinations are usually expressed in perfect sounding sentences by the LLM, that's what makes it so convincing to end users.

Yes, that's quite common.

I agree. And I also know how much of that experience comes from having a legal dept. that are collaborative and supportive of what the tech org wants to do. Which I suspect is quite rare.

I keep hearing that LLMs are trained on "Internet crap" but is it true?

Karpathy repeated this in a recent interview [0], that if you'd look at random samples in the pretraining set you'd mostly see a lot of garbage text. And that it's very surprising it works at all.

The labs have focused a lot more on finetuning (posttraining) and RL lately, and from my understanding that's where all the desirable properties of an LLM are trained into it. Pretraining just teaches the LLM the semantic relations it needs as the foundation for finetuning to work.

[0] https://www.dwarkesh.com/p/andrej-karpathy

Nice unlock to hyperinflate their way to $100B. I'd buy an AGI spoon but preferably before hyperinflation hits. I'd expect forks to outcompete the spoons though.

That's just a privately owned tech company then. Lots of companies never IPO.