scottjg

Kaiser is a vertically integrated healthcare provider. They provide insurance like UHC but they also require you, for the most part, to be treated in Kaiser medical facilities.

How can UHC use AI in the same way? They're an insurance company. They're not administering the actual healthcare?

usually the LLM will match against the style in the context, even if the words ask it to do otherwise. since your skill sounds like an LLM wrote it, I would be surprised if it didn't make things worse.

sorta like asking an LLM to get more creative with a visual design, that also never works.

i admit, you got me chuckling with that one.

interesting. that might be a fun intro project to MoltenVK. I hadn't dug into what was missing for Doom. I thought maybe the issue was that the intro/menu always ran in opengl mode or something. If it's just one missing op, that's way easier.

custom zoom75

it is a separate stack, but that probably doesn't matter much. a user process (in my case, qemu) can communicate with a driverkit driver. the user process can also map memory through the driver, which is how this pci passthrough system works.

i don't think the issues with the project really are specific to driverkit.

i don't know for sure, but i suspect what makes the tinygrad stuff slow isn't the macos host driver itself. i think they're doing something very similar to what i'm doing, which is just mapping the PCI BARs to userspace, then they have a bunch of python code that drives the GPU.

this is only speculation, but i think the big thing that makes tinygrad slow is that the tinygrad inference engine has not really been optimized much for all these open LLM models. probably most of the work has gone towards optimizing the stack for george's self-driving hardware company. since you can't just run the existing CUDA kernels on their engine, that makes things a lot tougher, engineering-wise.

i am actually curious if my project could share a macos host driver with them. i think it would need some changes, but it seems like there's a lot of overlap

thanks!

there also appears to be a generic pci passthrough path. we were discussing it on the qemu-devel list: https://lore.kernel.org/qemu-devel/C35B5E97-73F2-4A60-951B-B...

in fairness to the LLM critics, every time i ran into a minor speed bump in this project, it told me it probably just wasn't possible to get it to work well. the LLM did pretty actively discourage me from trying to get the whole thing working.

that said, since i was willing to ignore that aspect of it, it did accelerate getting the work done by a lot. it seems like it understands system programming really well, and did a good job navigating the qemu codebase. i have ~20 years of systems programming experience so i already knew what had to be done here. it didn't really guide the project much, but it did write a lot of the code.

the exact numbers in the graph are 17019ms vs 142ms. so you're right, it's not 120x, it's 119.85x.

two semi interesting things to note around this:

1. Virtualization.framework seems to support some form of GPU passthrough from the host (granted, not eGPU - it's for the integrated GPU). I think the primary use case is having macOS guests get acceleration, while still sharing GPU time with the host. There is also a patch that recently hit QEMU mainline that supports using the "venus server" with virtio-gpu to support a similar functionality for Linux guests under Hypervisor.framework.

2. Apple internally has some kind of PCI Passthrough support available in Virtualization.framework. It seems like the code is shipped to customers in the framework, but it relies on some kind of kext or kernel component that isn't shipped in retail macOS. I can't say if that's intended to ever be released to customers, but clearly someone at Apple has thought about this the feature.

I very recently ran the numbers on these GPUs for an upcoming blog post. The token generation performance is bad, but the prefill performance is _really_ bad.

For a Qwen 3.6 35B / 3B MoE, 4-bit quant:

- parsing a 4k prompt on a M4 Macbook Air takes 17 seconds before generating a single token.

- on an M4 Max Mac Studio it's faster at 2.3 seconds

- on an RTX 5090, it's 142ms.

RTX 5090 uses more power than an M4 Max Mac Studio but it's not 16x more power.

we can only hope

a great post that definitely inspired this one. i link to it in the first paragraph of my blog post.

i thought my post was already too long to include this, but to your point, you can run AI inference in this setup and the performance can be pretty good.

you're lucky you didn't get stuck with an S3 ViRGE.

this is mostly right, but it's not true that you can just sign up at any time. there's an open enrollment period for the aca marketplace and if you miss it, you won't have the opportunity to buy health insurance until next year.

runway construction is only part of the story, i think. in May, there was a number of complete ATC meltdowns causing ground stops. if you look at the stats, the majority of delays at newark in May are attributed to ATC.

personally, i was on a flight in May from SFO to EWR and my flight flew 2h30m towards Newark, then back to SFO when EWR stopped accepting landings: https://www.msn.com/en-us/travel/news/united-flight-that-was...

granted this was an especially egregious situation and not the norm, but it feels like these types of issues are on the rise based on my anecdotal experience. there were a number of full ground stops at newark due to ATC in the weeks after this. it was national news.

i frequently fly in and out of newark

may is the latest data currently available.

In May, Newark airport flights were on time 49% of the time: https://www.transtats.bts.gov/ot_delay/OT_DelayCause1.asp?20...

Maybe in aggregate flights have fewer delays but every single flight I’ve taken this year has been delayed (on top of the padded flight times the article mentions). I’ve flown about half a dozen trips.

I also hate the argument that the free market should solve the pricing problem. Airlines have exclusivity on airport gates. Any frequent flier on the SFO -> EWR route knows that if you want to save money you can book an Alaska flight instead of United but Alaska has significantly fewer gates and usually gets delayed when arriving waiting for one. Flights aren’t exactly equal commodities and even if the airlines were well-run, contracts for these gates are locked in.

Pricing stats here also fail to account for business class vs economy pricing. Business class prices on tickets have skyrocketed, way outstripping purported CPI. In some cases prices have doubled or more since COVID.

i have no doubt that other countries have some problems in their healthcare systems too, but i think you are downplaying a few key points:

1) united healthcare made 90 billion dollars gross profit in the last 12mo, and that's only one health insurance company. claiming that it's not a great business at a 2-3% profit margin ignores the scale of money involved, and ignores that the customer for health insurance is truly captive.

2) you're right that america has very high prices in healthcare. doesn't it seem bad that private insurance companies are incentivized to make things cost as much as possible so they can skim that 2-3% off the top? insurance companies negotiate and set prices for services and pharmaceuticals. they now own the pharmacy benefit management companies that would normally be incentivized to negotiate for lower prices.

i would expect in a public health care system that rejects procedures, they would follow consistent guidelines and rules. american health insurance companies will arbitrarily reject a percentage of procedures that they know they should be accepting in order to keep their profit margin in the right range.

i think it's hard for me to see the argument that health insurance companies are a net-positive or even net-neutral party in the united states. i don't think it's a coincidence that we have some of the highest prices and some of the worst outcomes.

i think there are a lot of folks who would be willing to have a 27% discount (allow for ~3% card processing fee) and forego those features.

if apple was saying you had to support their payment processor alongside others (so you could opt into paying +27% and getting easy cancellations), that would be one thing, but they don't allow you to have any other options available in the app, which i think is where the anticompetitive complaints start to feel more valid.

don't the incentives seem wrong? they're limited to 10% margin so what's the incentive for them to keep costs down for consumers? they only make more money when the costs go up.

out of curiosity, which wifi microcontrollers are better for battery life?

i don't know what the penetration of it looks like, but on the vsphere/esxi side there are also a number of really expensive addon features that i have not seen reproduced in open source software.

1. vmotion + storage vmotion - you can live migrate a vm from one hypervisor host machine to another. you can also live migrate the underlying storage (good if you want to consolidate storage servers, rebalance disk load, etc). with some caveats, you can do all of this without any downtime in the vm. it's not just a simple suspend on one host, resume on another host. a memory snapshot is migrated while the vm is still running on the first host, and when the amount of dirty pages starts to converge, they flip the vm over to the new host. similar idea for storage vmotion.

2. fault tolerance - for single cpu vms, you can use vmware's record-replay technology to execute a secondary vm in a "shadow" mode which replicates all of the nondeterministic events across the network. if one hypervisor host dies, the other can take over with no downtime. this is great when you need to add HA for a legacy application.

3. vsan - generally you run these systems with some sort of shared storage (nfs or iscsi attached SAN, or something like that). a SAN can be really expensive and a single point of failure. vmware can create a "virtual san" from a cluster of your esxi hypervisor hosts. as you can imagine, it has all sorts of HA features and can rebalance workloads to improve performance.

there are more, but that's just a few interesting features.

it was pretty noticeable to me. i don't think it's the groundwater specifically but they started treating the water with chlorine as part of the change, which you can definitely taste. i installed a filter under the sink and at least that fixed the problem for me... but it did make me reflect on what's in our water, in general.

also consider that we issued more than 350k student visas this year. if the h1-b cap is 85k then we are giving immigrants some of best college-level education in the world and then refusing to hire >75% of them. this has never made sense to me.

how old are you?

Paul is famous for his essay about Lisp - arguing that his startup beat other startups because Lisp is such a better language than C++ or Java which his competitors were using.

Unclear to me if this is really the meaningful reason that Paul's company succeeded (I'd say obsession with programming languages can be a counter-indicator for productivity), but it clearly resonated with a lot of people since that essay is arguably the onramp into Paul's fame. A lot of programmers read that essay back in 2001. His popularity as an investor who understood engineers arguably catapulted YCombinator