bacon_blood

This is a software supply chain issue, but it likely doesn't interest Intel Mac users who are still using x86 homebrew and who can't empathize with the experience of the upstream developers for all the Mac software they use.

I just checked your profile and it seems like you're a command line software developer for Linux. I'm not sure why you're telling a Mac developer what Mac developers do and don't need, or what use cases are "legitimate".

I'm speaking from personal experience about what is broken about homebrew's developer tools for a software developer building and using lots of existing open source software for Mac. Please don't respond further unless you are absolutely confident your position is in good faith and supported by personal relevant experience.

No. This isn't fixed by open source. Open source projects also do binary releases (see brew cask), and those need to be universal as well. Someone needs to build those universal binaries, and homebrew has decided to be incapable as a platform to do that by refusing to ship universal binaries.

It is kinda mitigated for end users if you only run an x86 homebrew for the foreseeable future and never compile any software outside of homebrew, but that's a crappy answer.

Universal means you can seamlessly switch between architectures as necessary. Split arch homebrew, however, means you should probably keep one of them entirely out of your path until you need it, as your arch will silently change if you have the wrong arch bin in path (I encountered this in the wild). Having the wrong arch bin first in your path will also result in a bad cpu type error if you force arch on any parent process e.g. with arch -x86_64 (I encountered this in the wild).

Also, users expecting that their build tools will be x86 also results in hacks like checking the cpu type sysctl and forcing ARM, which in any build tool is likely to break the ability to build x86 binaries (I encountered this in the wild in several projects).

I've seen all of this in multiple open source projects. I had to stop using homebrew entirely for build tools as a result.

I disagree. Apple would spend the time to make universal binaries work (unlike homebrew, who has stated they won't support them). Not installing universal binaries has some bad implications for Rosetta for developers. Having mixed single processor binaries in PATH can cause architectures to switch unexpectedly for subprocess chains (like when running a build tool), or even get "bad CPU type" errors in some contexts if the wrong arch is first in PATH.

A GPU's compute core, to my limited understanding, is sort of like a CPU with a lot of threads that execute the same instruction at the same time (aka SIMT). It has a decode frontend, local memory cache, addressing, registers, and a lot of instructions available to dispatch - each cycle can do something wildly different from the previous cycle. Each distinct instruction requires some dedicated hardware.

To my knowledge, "tensor cores" and neural accelerators are modeled on something like a coprocessor with a very fast memory bus, and a bajillion of the same small execution unit that can do a single operation in parallel (like a 3x3 matrix multiply) on behalf of the main processor. Like if AVX512 was actually AVX 1,000,000 and only had one instruction, and that instruction did some kind of 3x3 matrix math.

Imagine if you had a very large specialized house made entirely of kitchens (instead of bedrooms, bathrooms, etc), and your roommates were all cooks, you could cook significantly more food at once than in the typical one kitchen small household. You also save power per meal cooked because all of the lights and electricity go to kitchens instead of other rooms.

So on a pipelined CPU, the processor has different pipeline stages that execute in order. An instruction may for example move from fetch, to decode, to load, to execute, to store. The execute step may be executed on a different part of the CPU depending on what kind of instruction it is. Basic arithmetic, floating point, and vector math (such as AVX) can be dispatched through different execution ports and run on different parts of the processor. So a processor may (per core) have a pipeline, an integer math unit, a floating point math unit, and two vector units. Operations running on the execution unit also take some variable amount of time to complete. Having ports to two execution units of the same type available makes it so the processor pipeline can dispatch a second long instruction of the same type before stalling.

I don't actually know how the hardware of a tensor accelerator works, but what I would imagine a "tensor core" to be, is thousands of identical execution units that can only do basic matrix math, and a basic pipeline that is much simpler than a typical CPU.

CPUs and GPUs have highly variable workloads and need a lot of specialized hardware on chip that may not always be in use. This wastes power and means you can't have any one task as densely or efficiently as a dedicated chip. If you're designing a dedicated chip, which has direct access to the main cpu memory (as the apple neural engine has), you can design the chip to directly (streaming) read a large matrix from memory, perform an operation on it, and store the result back into memory.

Normal CPUs and GPUs don't have this capability. They approximate matrix math with lots of individual instructions that just go through a pipeline, stall, cache miss, etc just to do a lot of floating point vector math and store the result back to memory. A dedicated chip can skip all the overhead and just tile efficient matrix math in thousands of execution units. That's why an NVIDIA A100 is 19 teraflops when doing normal floating point vector math, and 150 teraflops when doing fp16 matrix math. It has a section of chip dedicated to efficiently doing the required floating point operations en masse without overhead or extra cycles and cache for fetching instructions.

Most machine learning is using _NVIDIA_ GPUs, which themselves have a neural engine (tensor cores) for the last two generations. An NVIDIA A100 has around 19 Teraflops but 156 "tensor flops" (312 if you use sparse matrices).

In addition to being useful for training and inference, the consumer cards use tensor cores for things like mic filtering (RTX Voice) and neural upscaling (DLSS) in games.

General purpose GPU hardware is way more wasteful for matrix math, like maybe >10x waste on power and equally worse performance, than tensor cores.

Both x86 and arm clang can output for the other architecture, or universal.

Pass `-arch arm64` not `-march`

You can also `clang -arch x86_64 -arch arm64` to build for both at once.

You can even go a step further and run your clang as native from bazel, via `arch -arm64 clang`.

Put it all together and you have: `arch -arm64 clang -arch x86_64 -arch arm64`.

It may seem like I'm joking but I'm not.

The benchmark program itself is obviously broken on ARM, as Rosetta is jitting ARM behind the scenes, so you could write a program + compiler that emitted the same ARM as Rosetta. This means it's a problem with the program and not a problem with the M1. I'm not sure what's actually wrong with it yet.

Edit: messe found the issue in sibling thread

This article has a mistake. I actually ran the benchmark, and it doesn't return a valid result on arm64 at all. The posted numbers match mine if I run it under Rosetta. Perhaps the author has been running their entire terminal in Rosetta and forgot.

As I write this comment, the article's numbers are: (minify: 4.5 GB/s, validate: 5.4 GB/s). These almost exactly match my numbers under Rosetta (M1 Air, no system load):

    % rm -f benchmark && make && file benchmark && ./benchmark
    c++ -O3 -o benchmark benchmark.cpp simdjson.cpp -std=c++11
    benchmark: Mach-O 64-bit executable arm64
    minify : 1.02483 GB/s
    validate: inf GB/s

    % rm -f benchmark && arch -x86_64 make && file benchmark && ./benchmark
    c++ -O3 -o benchmark benchmark.cpp simdjson.cpp -std=c++11
    benchmark: Mach-O 64-bit executable x86_64
    minify : 4.44489 GB/s
    validate: 5.3981 GB/s

Update, I re-ran with the improvements from downthread (credit messe and tedd4u):

    % rm -f benchmark && make && file benchmark && ./benchmark
    c++ -Oz -o benchmark benchmark.cpp simdjson.cpp -std=c++11 -DSIMDJSON_IMPLEMENTATION_ARM64=1
    benchmark: Mach-O 64-bit executable arm64
    minify : 6.7234 GB/s
    validate: 17.7723 GB/s

That puts Intel at 1.16x and 1.07x for this specific test, not the 1.8x and 3.5x claimed in the article.

Also I took a quick glance at the generated NEON for validateUtf8 and it doesn't look very well interleaved for four execution units. I bet there's still M1 perf on the table here.

Go f1980efb92c011eab71aa61b68ccf58d845d1de7 on a Mini on 11.0.1

    goos: darwin
    goarch: arm64
    pkg: sync
    BenchmarkCond2       4849542        249.7 ns/op
    BenchmarkCond2-2     2639139        531.1 ns/op
    BenchmarkCond2-4     3112030        393.1 ns/op

Web framework development

Ad providers already silently notice context and provide relevant ads (manipulating the eventual result of your page load). Isn't that just as creepy? (especially when it happens inside e.g. Gmail)

Who's going to pay for web advertising anymore if just anybody can install a browser extension able to block ads? We should boycott Firefox.

What kind of world is this where I can't fund companies by learning about the best kind of paper towels or being retargeted about a product I already bought?

I recently saw an interesting thought about the state of piracy, and it being a symptom of a flawed business model. What's to say advertising in its current state is the future of Internet business models?

If enough users don't want to see the current kind of ads, don't they become far less effective?