mikeayles

So for people wondering if it can be used to accelerate LLM inference, sadly not.

I've been trying to hit 100,000tokens/s with a 3.28m dumb model, and even this is an order of magnitude too large to benefit.

It appears to be focussed more on latency, than throughput. Happy to be corrected?

you may find this useful: https://phaestus.app/blog/blog0031

Edit: Forgot I also got doom running in openscad: https://www.mikeayles.com/blog/openscad-doom/

and doom running in openscad in the browser at https://doom.mikeayles.com/

I benchmarked Claude Code and GitHub Copilot on the same model (Haiku 4.5) with and without RAG-powered semantic search across 60 queries on a real codebase.

RAG didn't make search more accurate on Claude Code, but it cut token consumption by 28%. On Copilot, it cut time to resolution by 44% and improved F1 by 19.5%.

The bigger finding: controlling for model, tool design alone accounts for a 30pp recall gap between the two tools. Benchmark code and data are open source.

Rewriting the backend Bitwise Cloud, my semantic search for embedded systems docs Claude Code plugin from Python to Go.

The problem was the ML dependencies. The backend uses BGE-small-en-v1.5 for embeddings and FAISS for vector search. Both are C++/Python. Using them from Go means CGO, which means a C toolchain in your build, platform-specific binaries, and the end of go get && go build.

So I wrote both from scratch in pure Go.

goformer (https://www.mikeayles.com/blog/goformer/) loads HuggingFace safetensors directly and runs BERT inference. No ONNX export step, no Python in the build pipeline. It produces embeddings that match the Python reference to cosine similarity > 0.9999. It's 10-50x slower than ONNX Runtime, but for my workload (embed one short query at search time, batch ingest at deploy time) 154ms per embedding is noise.

goformersearch (https://www.mikeayles.com/blog/goformersearch/) is the vector index. Brute-force and HNSW, same interface, swap with one line. I couldn't justify pulling in FAISS for the index sizes I'm dealing with (10k-50k vectors), and the pure Go HNSW searches in under 0.5ms at 50k vectors. Had to settle for HNSW over FAISS's IVF-PQ, but at this scale the recall tradeoff is fine.

The interesting bit was finding the crossover point where HNSW beats brute-force. At 384 dimensions it's around 2,400 vectors. Below that, just scan everything, the graph overhead isn't worth it. I wrote it up with benchmarks against FAISS for reference.

Together they're a zero-dependency semantic search stack. go get both libraries, download a model from HuggingFace, and you have embedding generation + vector search in a single static binary. No Python, no Docker, no CGO.

Is it better than ONNX/FAISS? Heck no. I just did it because I wanted to try out Go.

goformer: https://github.com/MichaelAyles/goformer

goformersearch: https://github.com/MichaelAyles/goformersearch

Fantastic comment, thanks! A review of the state of BEV's was actually going to be one of my next articles (hopefully after the additives).

Are you happy for me to drop you an email to review a draft when I'm ready?

I do now: https://www.mikeayles.com/rss.xml

Good spot, and gruez is right about the caption too (fixed both, thanks).

The car's L/hr figure was wrong. At 45 mpg (imperial) and 70 mph cruise, a car burns ~7 L/hr, not 3. That makes the flow rate ratio ~4x, which is consistent with 5x per mile and the truck travelling 20% slower.

The ~3 L/hr I originally had is what you'd see as an average over a mixed driving cycle — ~30 mph mean across urban, suburban, and motorway. I was carelessly mixing the cars combined-cycle flow rate with the truck's cruise-only figure in the same row.

The truck doesn't have this problem because a long-haul artic genuinely spends most of its operating hours in that narrow 50-60 mph cruise band. "Average fuel burn rate" and "fuel burn rate at cruise" are nearly the same number. For a car they're very different, transient acceleration, idling in traffic, and low-speed urban driving all drag the average flow rate down well below the motorway figure.

Probably the first thing to consider is the trucks have their speed calibrated periodically to ensure the accuracy of their tachographs (in the UK at least) so a truck doing 90kmph may show as 100kmph+ in a passenger car, I know my Volvo is 7% out, and my Seat is closer to 10% out.

That said, depending on the truck, there's fuses you can pull, ECU remaps and even for the older trucks with the magnetic sensor in the gearbox, the trick is/was to stick a magnet on the sensor (with a bit of string, so you can pull it off remotely if you get pulled over). All of these methods are becoming less feasible, as things like the aggregate wheel speed sensors used for ABS get used, you can't just fool one thing now.

As for the weight limit problem, that's a whole other rabbit hole!

Sorry, got mixed up there, will amend, the 60 is for +3.5t!

Edit: Nope, despite the vehicles only being able to propel themselves to 90kmph, the speed limit is indeed 60mph (in England and Wales, Scotland is a more sensible 56mph)

https://www.gov.uk/speed-limits

I have done a deep dive here: https://www.mikeayles.com/blog/on-vehicle-hydrogen-generatio...

Short answer, it takes more energy to generate than the energy it produces.

You can do things like only producing electrical power from the alternator when decelerating, ensuring no load comes off the engine, but that would require accumulation as you're not actually burning fuel then either.

But running the numbers on the power requirements, I reviewed one commercially available system (at 12v 14A) and calculated that the HHO they are able to produce is 0.037% by energy going into the engine vs regular fuel.

When presented with 0.037% of the fuel substituted, their 10-15% claim on fuel savings becomes a bit of a red flag.

Fair point that a full TCO comparison would be more complete, and it's something I'm planning to cover in a later post. But the capital cost argument actually reinforces the conclusion rather than undermining it. Hydrogen fuel cell trucks are currently 2-3x the price of diesel equivalents, roughly comparable to battery electric. The infrastructure CAPEX is dramatically higher for hydrogen (£2-5M per station vs. transformer upgrades for depot charging). And the fuel cost gap means hydrogen has higher opex too. Diesel wins today on upfront cost, yes, but hydrogen doesn't beat BEV on capital or operating costs in any current scenario I've seen. Happy to be shown numbers that say otherwise.

The big thing I haven't covered yet is HVO, which provides the WTT CO₂ saving at a slight fuel cost surcharge, which matters if a fleet is mandated to reduce their CO₂. The TCO assuming a £100k diesel truck and £200k lifetime fuel cost, a 10% HVO surcharge brings the TCO to £320k, versus a £250-350k BEV truck that costs maybe £80k in electricity over the same life. That's £320k with an 80-90% CO₂ reduction from a drop in fuel you can put in your existing trucks tomorrow, versus £280k for battery electric with zero tailpipe. Both of those are available now, with existing infrastructure. Hydrogen is asking you to spend £300k+ on the truck, £150k+ on fuel, and hope someone builds a station within range of your routes.

Good catches, both of them. The hydrogen pathway should include grid transmission before the electrolyser, you're right. In practice it doesn't change the overall ratio much (multiplying by 0.95 on both sides) but the diagram should be consistent.

The BEV cost not adjusting with the electricity slider is a bug, not a choice. I'll fix both. Thanks.

Thanks. Both good questions, and they come up a lot.

To be clear, I'm fully behind decarbonising freight. It's one of the hardest sectors to clean up and it needs serious attention. But hydrogen for road transport requires jumping in with both feet (due to infrastructure requirements) when there are dozens of smaller, commercially proven steps that get you equivalent results. Better route planning, driver training, aerodynamic retrofits, hybrid and battery electric for shorter routes, even just reducing empty running.

These aren't exciting and they don't get press releases, but they compound. The industry could cut emissions meaningfully with changes that pay for themselves today, without waiting for a national hydrogen infrastructure that doesn't exist yet.

On surplus offshore wind: the economics only work if you assume the electricity is genuinely surplus, meaning there's literally no other use for it. In practice, the UK grid still runs gas plants for roughly 40% of generation. Every MWh of offshore wind that goes into an electrolyser instead of displacing gas is a missed decarbonisation opportunity. "Surplus" renewable electricity is a future state, not a current one, and even when we get there, interconnectors, grid storage, and demand response will compete for those MWh. The electrolyser only makes sense after all of those higher value uses are saturated.

On £1.50/kg: that would genuinely change the fuel cost picture, getting you to roughly 12-15p per mile which is competitive with diesel. But the distribution problem doesn't go away at any price point. You still need compression or liquefaction, transport, and a national network of dispensing stations. The UK has 11 public hydrogen stations. Even free hydrogen doesn't help if there's nowhere to fill up. The grid is already everywhere. Adding a charger to a depot is a transformer upgrade. Adding a hydrogen station is a £2-5M civil engineering project.

The place where cheap green hydrogen gets really exciting is exactly the applications where you can't just plug in: steel, ammonia, seasonal storage, maritime. Those don't need a distributed national refuelling network, they need point to point bulk delivery to industrial sites and ports, which is a much more tractable logistics problem.

I was reading the NASA truck aerodynamics thread earlier and realised that commercial freight is one of those fields that touches everyone's daily life (everything you own arrived on a truck) but sits in a complete knowledge blindspot for most people.

I work in fleet fuel efficiency and wrote up the foundational mental model, covering why trucks weigh what they weigh, why they're all doing exactly 56mph, why diesel is so hard to replace, and why 1% fuel savings matters when you're burning 43,000 litres a year.

This is the first in a series, there's already a 2-part deep dive on hydrogen up as well. Tried to keep it accessible without dumbing it down.

Correct, the engines will have a peak reported torque at one particular RPM, usually quite low (maybe 900RPM off the top of my head), at higher RPM, the peak torque may only be 90% of the peak torque, but at 50% more RPM, you end up with (100 * 0.9 * 1.5)=35% more 'power' despite the torque being lower.

This is detailed in J1939 (canbus protocol) PGN (parameter group number) EC1 (engine configuration 1) if you fancy a rabbit hole.

The vertical clusters will be a combination of gearing and cruise control, exactly. At the vehicle speed limit in the top gear (usually 12) most trucks will cruise around 1200rpm (exception being the scanias with supercruise). It's likely this truck did a lot of urban driving or was in average speed zones.

You also get a lot of drivers (suprisingly more younger than older drivers) that are happy to set it at 53mph, get the green tick for the telemetry to try and get a bonus, but then have a really low stress drive. It creates a bigger speed differential so they can be overtaken by other trucks easier, and realistically results in 10-20mins over a 10hr shift, to some, a 10hr chilled out shift is better than 9hr40mins of pushing it.

Correct, RPM and Torque (%)

edit: images may be down for 5mins, migrating my blog to astro.

To clarify, the air bags isolate the cab from the chassis.

There is also suspension between the axles and the chassis which is 99% of the time air on the rear, leaf spring front.

I haven't come across a cab that is suspension isolated from the frame of a conventional, even though the axles are on air. Theoretically as the driver is in the sweet spot of a much longer wheelbase, rather than sitting directly over an axle.

Although this has dropped off the FP now, I wanted to finish my response, but wanted to follow up with some data:

https://www.mikeayles.com/torque-rpm-time.png

So, here is approximately a weeks worth of data (around 700l of diesel), looking at the histogram on the right hand side, theres an awful lot of time spent at 0-10% this is around 60% of the trucks time.

https://www.mikeayles.com/torque-rpm-volume.png

However, when you multiply the time by the fuel consumption within that same time period, you get a volumetric chart, where we can see that 60% of the time, accounts for only 2% of the fuel use.

So, for idling, whilst it's noisy, produces CO2, a 5% improvement at highway speed will get you so much more of a fuel and co2 saving than eliminating idle. although, both, do both.

A lot of the time, the idling you may hear next to a truck that's parked up on break may actually be a refrigeration unit. Most fleets have telematics that would give a driver a red flag if they idled excessively, a lot of trucks even turn themselves off (although it's easy to bypass).

On the market, there's battery electric APU's that drive AC units, like: https://voltaair.com/product/iq-no-idle-ac-unit-for-trucks/#...

But unless they are mandated, an idling truck uses <1 litre per hour, assuming AC over summer months only (as they will have a webasto style diesel heater for the winter), results in about $1,000 per year in idle fuel. Assuming a fleet has their vehicles on a 3 year rotation, the APU needs to be $3k or less, with zero failures for it to be economically worthwhile. (additionally, the charge for those batteries needs to come from somewhere, you can do retarder/brake enabled alternator charging which would help though).

So unless mandated by carbon emissions targets, the lower risk option is just to burn diesel.

However, at around 1k litres per year, that's around 2.5tonnes of CO2, so if cost isn't a lever that can be pulled to change this behaviour, CO2 may be.

This is a bit counter intuitive, but Euro style cabovers are much quieter inside than a 'conventional'. Plus cabovers can have air spring suspension, they're often more comfortable. The downside is much less space inside, which is a big deal for the long haul drivers in the states, whereas in europe, they rarely do more than a month at a time.

As someone who works in commercial fleet fuel economy (powertrain, not aero, but you keep an eye on the whole stack since technologies compound), it's worth noting how far the industry has moved since the 1970s NASA Dryden work.

The sheer amount of "snake oil" bolt-on products sold to fleets in the 90s and 00s means the industry now demands rigorous third-party validation. The best open source testing I've seen comes from Mesilla Valley Transportation Services (https://www.m-v-t-s.com/certified-technologies/). They don't rely on ECM dashboard readouts or pump receipts. They run a control truck and test truck simultaneously on a track or controlled highway loop so weather, air density, and wind are identical. Anemometers on the trucks algorithmically factor out wind speed/direction, tyre temp, and weight. Tests are designed to answer the actual question rather than tick a standard's boxes. Their chief test engineer was ex NASCAR, IndyCar and F1.

My team actually tested a dual-fuel system with MVTS. The engineering was sound, testing proved a slight thermal efficiency improvement, but cheap US diesel versus the cost of the alternative fuel meant the ROI was non-existent. The physics worked, the math killed the project. (The economics work in Europe, thankfully.)

A lot of what's being discussed in this thread is already standard too. Mirrorless cabs are more the rule than the exception now (as pjc50 linked above), close-coupled trailers with gap fairings, wheel covers, under-trailer aero, all commonplace. A couple of current ones worth knowing about: the Scanias with the "Super" powertrain run a really aggressive overdrive at around 900rpm at the 90km/h (56mph)EU limit, where a conventional truck sits around 1200rpm. Requires huge low-end torque but claims 5-8%. The Mercedes Actros L claims 3% from its slippery front end alone. It is absolutely hideous though, so make of that trade-off what you will.

The biggest factor though is driver variability. All the aero and powertrain hardware in the world doesn't matter if the driver thinks they are the stig. The biggest shift in the last decade has been removing human inconsistency from fuel economy entirely.

Automated Manual Transmissions are now completely dominant in modern fleets. They shift perfectly for fuel efficiency every single time, totally capping the penalty of a bad driver. Layer on top of that Predictive Cruise Control. Modern trucks use GPS and 3D topographical maps to "see" miles ahead. The truck's computer knows exactly when to back off the throttle before cresting a hill to coast over the top, and exactly how much momentum to carry into the next dip. A human driver relying on their eyes simply cannot compete with a powertrain that knows the exact gradient of the road three miles ahead.

Ha, no way, I also built this: https://www.mikeayles.com/#zookeeper

https://zookeeperapp.com/

I wasn't incentivized enough to actually make it into a proper native app or submit to either store, but it's cool to see something taken to the next step.

This looks awesome, great job.

How effective at circuit generation are LLM's with the language? I tried similar, and could get syntactically correct files, but the content always had errors: https://www.mikeayles.com/#tokn

What are your thoughts on atopile and tscircuit?

https://gemini3.devpost.com/

Targeting long running multimodal agents, think I tick most of the boxes in the brief!

I also am building Phaestus, a 0 to product system, that builds entire products from scratch.

https://www.mikeayles.com/#phaestus-wip

It is capable of creating a PCB (and outputs gerbers, bom, pick and place files), an enclosure (written in SCAD, outputs an STL for printing), and firmware, which it's able to compile using a pio runner on railway and provides a binary, but also has a webserial flasher for ESP32's.

There is a blog here, but i've been focussing on getting things finished, as I built it for a hackathon ending today.

https://phaestus.app/blog

I need to update the blog & writeup, because I have the first product it created, a bluetooth remote control. It wasn't without issues, but I have a working PCB, in an enclosure that was printed from it's design, running firmware it generated.

https://photos.app.goo.gl/4r4XtQ65rx9iBUVp9

I built a Pokédex for zoo trips for my girlfriend:

https://www.zookeeperapp.com

https://www.mikeayles.com/#zookeeper-wip

It lets you take photos of all the animals you see to collect them, when you 'capture' a new animal, it gives you fun facts about them.

I seeded it with UK zoos, but there's no reason it can't work elsewhere.

It was built because the signage at a zoo we went to was terrible and we had no idea what some animals were, so it matches your photo with the list of animals to the best of its ability.

https://www.reddit.com/r/apple/comments/1ihufa0/doom_running...

If geohot is involved, it wouldn't suprise me if it was the first proof of concept for each new hardware variant.

So this isn't exactly putting pre-wired up blocks together, my intent behind phaestus is to essentially to get you from an idea to a prototype tangible product as fast as possible.

I'm targeting sub 5mins from first prompt to manufacturing exports, stl files for enclosure, gerbers for pcb manuf, bin file for firmware, bom for pcba.

E.g. if you wanted something that doesn't exist, but don't have the time, the skills or it's just not worth it. One silly example I had was a colour e-ink selfie fridge magnet. As far as I know, that doesn't exist, I could make it, but I can't be arsed. (so I could suprise my partner with a selfie, a picture of our dog, or anything, just a little treat for her for putting up with me).

With this, it'll pull in a ESP32-S3 Sense Xiao board, an e-ink module, a battery connector and a usb c charge connector. glue it all together, and there we go.

Should work if you wanted a rudimentary zigbee mesh communicator, pulls in a C6, a touchscreen, battery, probably a physical button or two. Once that block library starts filling up, it'll become more and more capable.

However, what I suspect you're after is more aligned with https://www.circuitsnips.com

I built circuitsnips to be the 'thingiverse' for electronics schematics.

Unfortunately it's been a bit neglected since so much of my free time has gone into phaestus, I did have great intentions to kicad up some official reference designs, so I can get rid of the github scraped bootstrap data as that was the sticky point both ethically and for the quality of the schematics, but there are only so many hours in a day.

I saw routing as equally as big of a headache as the schematic, so formalizing the layout to a grid means layout becomes a compilation problem, not a design problem.

My intent for phaestus isn't to design pcb's, it's to design entire products, and also to be friendly to non technical users who don't know what a PCB is, let alone do layout themselves.