hajile

I might buy this if those languages were actually faster. Instead, implementations of scheme with no commercial backers run circles around scripting languages with huge budgets.

Common Lisp can match and exceed languages like Java without the backing of a megacorp.

Carp Lisp is basically identical to C in performance and old PS games were shipped in a proprietary dialect (GOAL) and had very good performance.

Lisp has proven itself capable great performance at every abstraction level.

Last I checked, both of these were built on top of Common Lisp/SBCL.

function can do what a macro does, but it must run every single time while the macro can run just one time during the final code generation.

Let's say you have some Java-style a + b. It needs to work on native strings (concatenation) along with various kinds of ints and floats. The issue is that each one of these works differently internally, so the system is going to look at the incoming symbols and metasymbols (inferred type data) and is going to dynamically change between different functions like ADD_STR, ADD_INT8, ADD_INT32, ADD_FLOAT32, etc. If it did this at runtime by introspecting the type with some `switch(incomingType)` statement which involves all kinds of extra data and branches that clog the cache and create branches (two of the worst things you can do for performance). Instead, the macro (though they probably don't call it that) looks at everything and hardwires the correct output.

This hardwiring and its associated performance is the difference between a function and a macro. A macro `foo!(a b)` can inline into the current function while the function alternative must save all the registers to the stack and create a new stack frame which is very expensive.

The beauty of lisp macros is how functions and macros look the same. You may think that `(+ a b)` is the same as your favorite languages' `a + b`, but most lisps also allow `(+ a b c)` too. `+` is a macro rather than a function and it knows that `(+ a b c)` needs to convert to something more like `(+ a (+ b c))` and a typed lisp may go further to something like `(+f64 a (+f64 b c))`.

Your lisp criticism is orthogonal to the idea of lisp itself. Lisps like Coalton (lambda calculus) or Shen (sequent calculus) are strongly typed.

AI is an interesting point too where being closer to a raw AST is likely an advantage because it can focus more on the semantics instead of the syntax.

They might be profitable for the exact two months SpaceX is giving them billions worth of free compute doesn’t seem convincing to me.

This is like when a vacuum doesn’t pick something up after a few tries. The user picks the thing up, looks at it, then puts it back down and tries again until they finally give up and move it to the trash.

If you can’t design a solution and instead waste days and who knows how much money in tokens instead of just turning on your brain for a few minutes, you are in the wrong profession.

XML wasn’t the best alternative then either. S-expressions have been around since the 50s and are better than either XML or JSON.

I fear even a lot of bright, motivated people will be so discouraged by AI doomers that they won’t bother trying to learn.

I'm wondering about something different. FPGA seems ideal for an AI chip because you can simply flash the latest model. The downsides are low density and low clockspeed. It seems that you can only fit 100-300M parameters in even very large FPGA, but that seems like it would be enough for most finetuning.

I'm thinking of a situation where you do the initial model calculations in hardware on the Taalas chip then hand that off to the FPGA to do the LoRA subset of calculations in hardware that can be continuously re-tuned to keep the model up-to-date. This would probably reduce throughput (or at least increase latency), but would save tons of money by allowing you to use the chips longer.

These people are professionals that acknowledge IPC is a stupid metric.

Even if you think IPC is a stupid metric (it's not), ARM is still wiping the floor in total performance, performance per area, and performance per watt.

If you switch your statement to SIMD throughput, now ARM NEON has the lower IPC and x86 looks like space age technology. They're optimized for different workloads.

Fujitsu also makes a 512-bit SIMD that has very good performance, but that large SIMD eats area and power while giving very little back to consumers. Honestly, SME co-processors or 6-wide SVE are much closer to "space age" than simply making a wider packed SIMD.

x86 vendors recognized that they could recoup the majority of efficiency that Apple Silicon has without buying an architectural license for ARM.

AMD actually bought an architectural license and Intel used to have one too. They could (and should) undercut ARM entirely by making x86+RISC-V cores. You can talk about Intel's E-cores or AMD C-cores, but they still don't compare in the key metrics.

AMD was shipping faster integrated GPUs than the M1 Pro before the M1 ever hit shelves.

Which GPUs are you talking about? 4800U (Vega 8) was around 50% slower than M1's GPU. 5700G released in April 2021 and also had the same Vega 8 GPU design. They didn't see a bump until RDNA2 in Rembrandt in 2022 (by which point Apple was already on M2).

nothing objectively prevents the x86 architecture from being power-efficient.

Citation needed. This is like saying nothing prevents branch delay slots from being good or VLIW/EPIC/Itanium is just as good as anything else. uop cache, extra pipeline stages for decoding, and memory ordering prediction are enough by themselves to use more power when under load. If you have actual proof that x86 workarounds have absolutely ZERO energy/area/latency, then show it.

That's Nvidia's work, no credit is due to Microsoft or Apple for reshaping the server market.

Linus Torvalds pointed out that ARM server wouldn't take off until devs could run ARM natively on their laptops. Apple provided those laptops.

MS claims Cobalt 200 lowers carbon footprint by 40% vs x86 (which should mostly reflect power consumption per unit of work). ARM accounted for over a third of Azure deployments in 2025 and should increase as C200 becomes available.

The Graviton and Grace chips that displaced x86 servers did it because they are slower, cheaper and less feature-dense. Graviton for the bare minimum of Raspberry Pi-tier web serving

That hasn't been true for a long time now. Graviton5 in particular is a big deal. It removes the NUMA issues resulting in much lower latencies (and higher performance) for a lot of things. It gives 2.6x more cache per core and a general ~25% performance uplift (SAP said it was up to 60% for their workloads) which is more than the difference between graviton4 and Zen5 for most workloads. Graviton accounts for more than 50% of all new servers in AWS and basically all of their 1000 largest customers are using them.

Apple was entirely incapable of pivoting their mobile chipsets to the server scale, surprising nobody that had paid attention to Apple's godawful raster/GPGPU acceleration stack. The Ultra hardware looked like a dog's dinner compared to x86 arches like CDNA.

Apple wasn't incapable -- they didn't want to because they aren't in the server market. That is the entire reason X-Elite/Nuvia exists. GPU stack is completely orthogonal to making server CPUs (you can make good CPUs without making good GPUs).

The government isn't going to be making chip fabs go any faster which is the biggest limitation here.

The second big issue is that it takes months to fab chips meaning your hardware AI is months to maybe a year or more behind the times when it lands.

I do think it makes sense for something like a medical scanner where the model simply doesn't need constant updates, but that doesn't need government involvement to ship.

There is a reasonable kernel of an idea here, but only if you dial expectations WAY back. The 10 years speculation is just wrong though. Even in 10 years, their 8B param model isn't going to be in consumer devices.

6nm is just 7nm++ and the process will be a decade old in a few months. In the decade since, we've only had a slightly less than 3x increase in transistor density and that's including EUV, BSPD, and GAAFET (which means progress is likely going to slow down even more).

Even if we hit another 3x increase, their 815mm2 design will still be a bit over 90mm2. For comparison, the entire M5 Pro/Max CPU die is just 61.7nm.

If our current progress somehow holds (not likely), even 20 years from now the 8B model would be 30mm2. You need 30 years of dead consistent progress to get it down to an includable 10mm2.

As you can see, this doesn't make sense to invest in. As to the stuff like voice recognition or basic vision, these can often fit within 100m parameter models which would be around 10mm2 on their current 6nm design. That's doable today in custom edge computing devices.

The other possible use is cheap fallback models for AI companies. Moving to N2 and shrinking chips to 600mm2 to improve yields a bit would give about 50B parameters with 3 chips plus another FPGA-ish programmable chip for continuing training and interconnects for everything. You'd need hundreds of thousands of chips produced for that exact AI model just to get costs below $100,000 per board.

That seems like a lot of money for the AI model you are essentially giving away, but maybe it still beats the power and price of GPU server racks.

The better solution would be making part of the chip cluster use something like FPGA which can be reprogrammed.

Text to speech or diagnostics equipment where the core model is relatively small and never changes seems like the ideal application. You might be able to fit something in the 25-30B range in 2nm to 14A, but it would need a way to update.

Large models are simply out of the question in my opinion. If you need 400+ different chip designs, it’ll be billions of dollars to tape out before you even make the first chip.

Taalas HC1 AI uses Llama 3.1 8B, but takes up a massive 53B transistors and 815mm2 on TSMC N6 (nearly at the reticle limit of 858mm2). N2 is a little less than 3x as dense (110MTr/mm2 vs 313MTr/mm2).

This chip would still be 272mm2 on N2 which is an eye-watering $30k/wafer and bigger than a 9950x or Nvidia 5070.

This just isn't feasible. Some of the latest-gen LLMs seem to have 5-10T parameters or about 1000x more. I don't know that taping out just one chip makes economic sense let alone the 300-1000 chips required for a cutting-edge model. Things like continuing education so your model knows about the latest NPM packages or world news is super important, but seems like it would require new chips.

There are a TON of uses for an 8B parameter models on the edge, but this is WAY too big to put on the edge of anything. Something like a 10mm2 100m parameter voice model might be feasible on the edge, but only for expensive devices, but most of those are TSMC 28nm (up to 29MTr/mm2) or GF FDX22 (up to 40MTR/mm2) which would increase the AI chip to the point where it would absolutely dominate the BOM.

If I can run it locally, it is certainly more safe than running on some cloud server somewhere.

M1 released 6 years ago, but AMD/Intel still can't get close to ARM cores in IPC. Anandtech was observing that Apple had better IPC in their phone chips YEARS before M1. Lots of people discredited it as "apples and oranges" because the ISAs were different, but investigative teams from Intel and AMD absolutely HAD to know the truth.

This has both a technical and human component.

On the human side, top x86 execs refused to see any threat coming. They must have thought Apple couldn't overcome the x86 software moat, thought the chips were for servers, consoles, or some other non-PC device, or perhaps they simply couldn't believe what their investigative teams told them.

At the same time, we're 6 years post-launch. The proof of ARM's capability is clear. x86 server marketshare is about to hit just 50% and Microsoft is pushing ARM hard as a replacement for x86. Either all the x86 engineers are completely incompetent and incapable of learning from years of ARM designs or there are aspects of x86 that makes copying those designs infeasible.

Demand for 512+gb hasn't gone away though. I suspect we'll see M5/M6 Ultra later this year with 512gb-1tb of ram going for the biggest premium we've ever seen from the company.

We've already hit RAM power and size limits (about 40k electrons which is the limit before we get noise messing up the amplifier).

If a model needs 2x more memory, but serves the same number of customers, the cost is going to go up per customer to cover the increased hardware and power costs. Companies are starting to implement AI limits to keep costs under control.

Anthropic and OpenAI are rumored to be considering cutting inference prices trying to retain customers as LLMs commoditize and race to the bottom. It reminds me of the Chinese bike wars where bike-share companies were losing massive amounts of money, but kept running sales and lowering prices in an attempt to compete and drive out their competitors. The end of that story was a bunch of major bankruptcies and giant bike graveyards.

Nvidia's hard pivot to "in the near future, everyone will run their AI at home" seems to indicate that they also see the market shifting. We've already had AI ingest everything out there. The real challenge becomes how to better optimize their algorithm to get more useful data in less space.

If you are trying to get more than 64gb of RAM or doing tons of inferencing, you're getting a Max or Ultra anyway.

It might be M6 Ultra and I think the real reason for stopping selling top-tier units was to avoid mid-generation price hikes and increasing demand for the more expensive next-gen systems that I assume will come with 512gb (maybe 1TB) of RAM and a massive markup to match.

Most of the areas without good internet barely take home $100/m and a lot of them take home less than that.

If you hooked up every single cargo ship on the planet to starlink, you'd only add around 100k connections and average wages on most of those vessels is $5-8/hr (very few US/EU sailors these days) for a handful of people which tells you how much businesses actually care about their workers who do dangerous jobs.

There are two different questions. Are cables cheaper and is Starlink profitable in the long term?

Launching a starlink satellite costs $1.5-2M per satellite for the current design. Each satellite lasts around 5 years and they need 42k of them.

Do the math and you get $13B to $17B per year ignoring inflation and ignoring the massive ground infrastructure that is also needed (costing billions more per year).

Residential plans range from $55-175/month and will make up almost all subscriptions. Most people will get the $85 middle plan, but we'll round up to $100/mo or $1200/yr.

Breaking even with no profit requires 10-15 million subscribers.

As starlink claims to have around 12k subscribers and just 10.5k satellites, I'd guess the system is profitable.

The cable question is still compelling though. Rural fiber costs around $25k per mile. This means that an average $15B/yr buys 600,000 miles of installed fiber every single year.

My conclusion is that the ideal solution would be Starlink keeping total satellites closer to the current 10k number ($4-5B per year) and spending the other $10B/yr running fiber to new areas.

Their subscriber base would drop as new fiber got installed in rural areas overlooked as "not profitable enough", but stabilize at still profitable enough from travelers and truly inaccessible places.

This is putting the world at risk of Kessler Syndrome.

The even bigger hurdle is selling token based pricing to normal (non-dev) users.

"You pay an indeterminant amount of money to ask a question and you might not even get the response you want without spending even more money" doesn't appeal to most people who aren't gamblers and explaining how "thank you" at the end of a long exchange can be expensive due to context is an even harder thing for an average person to swallow.

Token cost going up/down like a yo-yo also doesn't help. Normal users NEED fixed costs and don't want to expend energy constantly keeping up with the AI meta. "My subscription lasted much longer last month" isn't a winning problem either.

I think Apple is correct that Local LLM for most things is the future.

AI with blockchain. Maybe we can mix in IoT and VR for the ultimate in buzzword synergy.

It reminds me of the Chinese bike wars where everyone was slashing prices trying to keep marketshare until the bubble burst and everyone lost billions.

https://www.youtube.com/watch?v=FQrEDq8KPiU

In the coding realm, I think we'll be seeing 35, 70, and 150B models sold where you pay a few hundred to a few thousand dollars up front and get a year of monthly/bi-monthly updates where they've trained it on new coding documentation and repos.

Humans have good spatial memory and having a handful of statically-positioned desktops in a 2D plane makes navigation intuitive and consistent.

The real issue is how the ORDER of the desktops changes all the time which messes with that spatial memory and kills a lot of the productivity improvements. A consistent straight line would still be worse than a grid, but still MUCH better than the current situation.

That's some really fast goalpost moving.

If AI could outperform humans, Anthropic would NEVER release that model. Instead, they'd use it to create a new google, photoshop, office, windows, etc for cheap then undercut all those companies and taking over the entire software industry.

I remember when React's vdom renderer was faster than anyone and it wasn't even close. InfernoJS uses a vdom and is still one of the fastest JS frameworks out there, so I don't know that diffing is always inherently slower.