vineyardmike

Anyone who says that shit publicly says crazier shit privately. Half the quotes floating around came from his resume and courses he was selling. So presumably the more polished material.

(2024) > for example in private i say “fat people” and “retard” and “trannies are disgusting.” the final leap is doing so online. if it costs me, this [new years] resolution was accomplished.

> quoted from ryanckulp.com

Yea I don’t think an apology is coming. The scrubbing of statements is contextually a funny plot twist though.

Billionaires use Gmail alongside impoverished people across the world.

Many tests penalize incorrect answers worse than blank answers.

As a famous example, you were incentivized to leave questions blank on the American SATs (until somewhat recently).

They (OpenAI) also made many more deals to acquire compute before they actually had the revenue to support it. They paraded around the world making deals with everyone and anyone. By doing it earlier, they got way better unit prices. Anthropic bought their compute at significantly higher prices by waiting until they actually had demand to fulfill.

Most of this hand-wringing on price will go away.

My assumption is that Anthropic, OpenAI, Kimi, etc all have a similar cost structure when serving models. The same size model roughly generates the same GPU usage whether you’re American or Chinese. I’d also guess that the model sizes across all SOTA models is similar, we just only see data for open models. The difference is most likely that American companies simply charge more because they have the dominant market position.

Remember not too long ago when Anthropic was charging $75/mt for Opus? Now that many models are in “opus tier”, their pricing is $25 - higher than competitors but close. The newest Kimi is $15. 40% lower to forgo “made in America” with American enterprise support staff is not crazy. Compare AWS to Hetzner or any other flagship enterprise service to the foreign and discount option. I assume that over time, we’ll see the commodification of models reducing prices even towards the raw GPU costs.

How is the "world" model different from plain dependency injection?

In addition to what the other comment said, this "world" model is great for hermetic testing of complex code, LLM written or not. We've seen existing projects that intercept the OS level syscall for testing, replayability, etc. Building it into the language runtime, hopefully with better ergonomics from the start than a syscall, would be a welcome addition broadly.

I just want every medical service to have this exact type of breakdown

Check out the "No Surprises Act"

https://www.cms.gov/priorities/key-initiatives/hospital-pric...

The goal is not for meta to take their market, the goal would be for meta to damage their competitors.

If meta releases an open-weight LLM that is not Chinese made, cheaper to run than the SOTA premiums, etc, it would lower the number of people paying for frontier labs models. We saw with with early LLAMA models, but they didn’t keep up in the race with v4.

Meta would benefit from this, not from increased revenue at the hands of open LLMs, but from reduced competition. Meta competes with Google for ad spend, and lowering the Google revenue (or increasing costs) from AI reduces the competitive advantage. OpenAI wants to build an ad engine, so same thing will apply there too - make it less-revenue-generating to compete. Meanwhile G, OpenAI, and Anthropic are huge talent sinks that they have to compete with, especially for ML talent which is core to Metas business goals (ads). Finally, Meta needs lots of GPUs to train their ad engine models. By reducing the revenue-per-GPU of these labs, they’re reducing demand on a core revenue generating supply they have to compete for.

That's basically a variant of a subscription, but one where the end user churns aggressively.

It's worse in many ways too - it's a lot harder to gauge interest as the developer to understand how well any update will sell, and if the updates "stack", then a user only pays for the newest update to get all older features free. It's also worse from a cashflow perspective for the developer (but better for consumer) since they have to pre-build the update before any chance of getting paid for it.

It needs to be safe (including from pollution), but you could absolutely just walk around the parking lot.

Some random person discovered a 60% across the board gain in all LLMs, using an extremely simple trick that none of the labs noticed in all these years of multi-trillion dollar growth

DeepSeek published a pretty well circulated paper on exactly this many months ago. It just hasn’t been attempted and shared publicly, asa retrofit, AFAIK.

Also, it’s no free lunch, the readme indicates that this “use images” hack is lossy and reduces success rates alongside the reduced cost. Most labs would focus on success increases regardless of price.

Nah it's BS. It's American-flavored capitalism that's the issue, and it's crushing the free-market.

I live in SF - the densest US city on the west coast, and the densest US city after NYC. We have city-owned "dark fiber" run through (most?) every street. Any ISP can offer service by renting the connection to my house, as long as they service "the last hundred feet" from pole to door and the billing.

I have about half-dozen ISPs that will give me from 1G-5G of service, all under $100/mo - (a great price in America). I pay for redundant >1g fiber connections to my house for less than the price of my parent's 50 mbps bill.

The issue is capitalism. In much of the US, the ISPs have lobbied and enforced "monopolies" by exclusive fiat of the jurisdiction, in some shape or form. 16 US states have laws that prevent the local government from maintaining or providing internet infrastructure like fiber lines, requiring private companies to maintain it all. Any free-market enthusiast will readily tell you that competition brings prices down, but capitalism is crushing the free-market, reducing competition for the benefit of the wealthy.

I only-kinda-jokingly say that I'd be willing to be the sacrifice and get a second mortgage on my house if it could buy the entire nation some new rights.

Our politicians routinely sell out smaller issues for "downpayment in a coastal metro" level of money. It's just about within reach of a middle-class urban adult to directly fund with some personal sacrifices.

I feel like we like to imagine that these corporations are budgeting big-bucks to bribe/lobby politicians, because they have more money than most humans can actually mentally picture, but their budgets are often closer to a small team of software engineers' salary. Meta spend ~$25M on all lobbying last year - and they're the top corporate spender. That's under 1 hours of revenue for them.

Google was back and was going to end up ultimately beating the rest because they have their own TPUs

Meanwhile Claude, ChatGPT, etc all now run at least partially on TPUs. At the same time, Anthropic, Meta, and others are buying TPU hardware directly for their own data centers.

Did they win with the best frontier model? Not with one they've made, but all the best frontier models do run on TPUs, and Google is making plenty of money along the way.

FWIW I agree with you, but it doesn't add much to the conversation to leave a comment saying so.

I also agree with the comment you're replying to as well - the vitriol and anger, along with the "this is just another blockchain bubble" type relies is really interesting. It's so surprising to see the variety of (negative) replies and beliefs people have, along with the general distaste/distrust for management. I guess it's also largely a sign of the times since a lot of ICs probably have a ton of anxiety about their career.

At my company, this was the explicitly stated and shared goal from management.

"We can't know all the parts of our business that AI can do a good job automating [because it's so new] but we also don't want to be the last to know and outcompeted along the way. Please throw AI at random parts of your job [and we're tracking this] so we can generate feedback from employees on where to invest in additional automation"

My company has since provided a ton of high-value little AI workflows, alongside a handful that didn't pan out. AI-assisted software development is a major change overall, but the general business-process updates from AI are a net-positive to me.

My curiosity is not the free AI summaries (which they can opaquely tune as necessary), but instead the renting of TPUs to Anthropic and OpenAI. Many of these contracts were announced last minute and seemed to involve a very desperate Anthropic. Based on the Anthropic/xAI data center contract, they’re willing to pay crazy markup to get immediate access to compute.

I want to know how impacted Gemini has been by that, because that will reveal a lot about their margins and revenue generating first party demand. Each MSFT earnings report they discuss the balance they’re dealing with between supplying GPUs to Azure customers and first party demand.

My pet theory is that Gemini is “losing” the LLM race because they’re preferentially selling the TPUs to competitors, while keeping just enough for themselves to stay competitive and build their own products.

No.

This isn’t a passive “walked by the window” thing that you might have unwittingly viewed. To actively search for open cameras by crawling every IP then creating a tool to see them, then choosing to watch the footage is a very active, deliberate choice. No one is viewing this footage without making a multi-step choice to view it.

They have a self-test you can take to compare yourself to the models, and one of the questions ends in “…even if some economists warn about bad outcomes from this”

That’s a crazy bias to throw into a question. Especially because it’s a relatively contested topic, from an economics research perspective.

What _is_ Google DeepMind doing

I feel like it has been pretty visible about what’s happening, between their press and products and financial statements. It’s just not what people are accustomed to expect.

First, Google has become a major compute provider for competitors, thanks to TPUs. They’ve talked about allocating TPUs to GCP instead of their first party products. I can only assume it’s because they’re collecting a higher margin, and it covers the cost of data center buildout - which they’ve been aggressively doing. I wouldn’t be surprised if they made the financial decisions to delay or slow training for Gemini 3.5 when they provided last minute compute to Anthropic this spring.

Second, Gemini has very directly not been focused on agentic coding, maybe 3.5 Flash being the change. They’ve built models they can deploy to watch YouTube videos, Nest cameras, scale to AI in search, understand fitness info in Fitbit, etc. They’re very clearly not focused around agentic/coding. They’ve put in a ton of efforts into multimodal data in and out, and they’re the only major lab working on video generation still. There was leak/rumor that their cofounder (brin) was getting involved in the model training to renew focus on agents so maybe this will change, and again 3.5 already feels different.

They literally had a productionized transformer model (translate) which was the inspiration for all of this. The famous paper was written about a productionized ML Model.

They also had a productionized LLM in search (known as 'mums') before the whole "AI Chatbot" craze.

They also had a chat-tuned LLM chatbot (LaMDA) in testing internally. It was shown off over a year before ChatGPT was created (2021)... and later released as an app (AI test kitchen) before ChatGPT was announced.

ChatGPT may have been the big industry moment, but Google was releasing LLMs to production before OpenAI.

For Claude specifically, (1) enterprises pay API rates on top of subscriptions, so subscriptions profitability questions are only relevant for smaller companies and indie devs. Many of whom probably have sporadic or low usage which helps balance some heavy users.

Again, for Claude, (2) it’s rumored that their API rates have around a 90% profit margin. It’s also claimed that the subscription limits get you around 10x tokens per monthly dollar vs buying them with API rates.

Edit: to drive it home. If a tokens true cost to anthropic is 1/10 of what they sell it for at API rates, and a subscription gets you tokens at 1/10 the price, that’s cost-neutral for the business if every subscription uses every token. They’re selling tokens at cost, not at a loss. Many subscription users won’t use their full allotment. That means serving some users doesn’t cost the business as much - which might push the subscription business from cost neutral to profitable.

Refusing to enact new laws around a thing most people don’t like, don’t want, and don’t care about (oh and is used for scams often) is quite different than a secret back door war.

If your country doesn't have any leading models, why not legalize distillation, either explicitly or implicitly?

(Chinese labs famously distilled American models, and that seems to be going well for them. They now have a competitive industry, home-grown talent choosing not to leave, and they now can truly compete without distillation).

The article addresses a pretty compelling reason...

Why would the makers of open models (mostly Chinese firms) continue to open them up, now that the value chain and economy shifts? Previously, it was a (Chinese) national goal to force the market to compress OpenAI/Anthropic margins (and compressing their revenue along the way), to ensure the Chinese had access to high quality models, and could afford to compete. Now there is an opportunity to usurp and be the international default, and claim the margin for themselves by closing their models.

Beyond that, there is likely an upper bound of capability-per-parameter, which means that there is an upper bound on "local" models, and once you need the cloud, why would the government not target clouds next?

Imagine you’re entirely pre-AI… to do some work, you read code, think, then write some code across a number of files. Usually then a small dance with compilation/unit tests to address anything broken. Along the way, you use your human judgement on style and quality, and midway through your change you might refactor something based on learned best practices (eg, when to use a static method, or helper class).

Today, even the dumbest AI agents can trivially loop through the final dance to get compilation, and often unit tests (depending on scope of failure). Big SOTA agents have OK code quality, but if left unattended or unchecked will still generate pretty sloppy repos after a while. That’s true even when using models like Opus which is ridiculously expensive in comparison.

When using the models in this fast “pair programming” style, I find that I (the human) mostly do all the “plan and think” work, and usually point the smaller agent towards specific files/directories, with specific targeted changes. It’s slower than 1-shot prompting an entire feature, but slightly faster than doing it manually, and I find the code is less “slop” because the changes are smaller and more human. It retains the agentic benefits of handing imports, compilation iteration, etc and can do basic cross-file plumbing. It’s also cheap and fast to do iterations like “wait make that method static” or “let’s update this to use <other util class>” and things like that. When the agent is slow to make localized edits, I find I’m less likely to push for minor nit-picks and style updates.

Recently I had switched to OpenCode to try out many of the Non-US-Frontier-Labs models. My unexpected favorite model to use was Mercury (a diffusion model). Not because it was “smart” but because it was stupid fast. It was more of a pair-programming experience instead of the SOTA agentic experience of prompting and waiting. Honestly, it was also way more fun and brought back some of the pre-AI coding experience while still getting some benefits of AI. It felt less of a slot machine where you prompt, wait, and hope it went in the right direction. It made me even use the tiny models like Gemini Flash Lite and GPT Mini/Nano more too.

Anyways, so excited for an open-weight model and I hope it performs well. I’ll be testing this ASAP.

I think one of life's big questions is defining the size of your sphere of responsibility.

Some people decide that sphere is really big, and they go on to be those historical figures. Others define it really big, and wallow in angst, aware and powerless to the suffering. Others still define it too small, and by the end of their life, find regret that they didn't try to help those within reach.

While I agree that AIs would do a good job…

Would you rather take instructions from a ruthless robot or ruthless flesh sack?

"Some features, including image generation, have daily usage limits because they rely on powerful server models. Increased access is available with most iCloud plus subscription plans".

Start at 1:07:00 in their announcement video. Craig is absolutely talking about "Apple Intelligence" as a whole in this segment.

Pragmatically, of course they'd need to add metering to any cloud available APIs that rely on large models. There's no way they will eat the cost of serving unlimited access to a cloud LLM to end users if they won't eat the cost of an image generation model.