Archit3ch

if someone would kindly quantise this down for the 64GB paupers

The Q4_K_M is 75GB. I'm exactly at 64GB, and I wouldn't quantize it further. Instead, do partial weight residency and stream the rest from SSD.

Reading "Custom CPU" I expected wafer.space, instead of FPGA.

I assume Nativ doesn't support SSD streaming like DwarfStar.

I expected a backrooms-style game where you are lost at an infinite airport...

Mixed feelings. Good for the defenders, also good for the attackers.

If Chinese models are distilled from Fable, does it mean that security questions will be answered at most to a level equivalent to the Opus fallback? Meanwhile the ~150 companies with access to Mythos have greater security capabilities. Assuming the Chinese models are not distilling Mythos.

1. iOS doesn't allow JIT for apps distributed through the App Store. However, WASM (including JIT) is allowed. Effectively, this would allow Julia to run on i(Pad)OS.

2. Technically, one could port the Julia compiler itself to WasmGC and get full REPL/eval semantics. However, App Store Guideline 2.5.2 states "Apps [...] nor may they download, install, or execute code which introduces or changes features or functionality of the app, including other apps.". It's not clear if this only applies to downloaded code or also local code (e.g. a playground app). If you attempt it, there is a good chance Apple will update their guidelines specifically to ban it. :P

2. We will be targeting WasmGC in the near future, so we need our core runtime and compiler data structures to be compilable without the assumption of necessarily having linear memory available.

Big if true. Does that mean we'll get a no-alloc iOS backend?

I'm very interested in your use case.

I looked into bit-slicing techniques, but they increase throughput at the cost of massive latency. This is not acceptable for realtime audio.

Having too many orders is a good problem to have (unless it gets cloned). Worst case, you pay an external contractor to sort compliance/fulfillment/whatever.

"Do things that don't scale!" is standard founder advice for a reason. The fact that no one else has put in the work means you get to sell $150 MIDI recorders... for a while. Which, fair enough, might just sustain a one-person business.

I can't wait for ASIC manufacturing to become as easy as ordering a PCB.

Obligatory "Does it answer security questions?".

With so many people implementing their own SSD streaming for specific combinations of model+hardware, maybe we should look into upstreaming to antirez/ds4 or llama.cpp...

You can keep the KV cache in (possibly Unified) RAM to avoid SSD writes entirely. Not sure if it would fit on a 32GB laptop, though.

Working on something similar targeting macOS on Apple Silicon, Unsloth split GGUF, compressed partial residency in unified memory (would make more sense on 128GB instead of my 64GB...), native Metal kernels, and RAM-only native compressed KV. Happy to put on GitHub when it's ready.

At this point comparing to Gemini is a free Bingo space.

As a developer, subscriptions come with a few problems:

- Users cannot open their old projects unless they pay the subscription.

- Lower uptake in general, which affects viral adoption ("my friends use this software, so will I"). This is important if you rely on word-of-mouth instead of ad spend.

- Need to keep shipping features to justify the subscription cost. There is less focus on stability/maintenance.

For developer productivity, you can move to a REPL workflow with edit-compile-test at the function level. Julia does this while staying on LLVM.

Of course, compile-the-world is still going to be slow, but there is no solution for that in the C++/Rust ecosystems either.

Is it possible to create an audio-grade capacitance multiplier in CMOS?

Surely this will succeed where $4B Modular failed!

A subset of python is python.

Mojo folks (rightly) disagree.

If low latency is your goal than you don't want JIT.

To be more specific, you don't want Java-style-JIT. Julia JIT (which functions closer to AOT in this case) is fine for low latency.

I'm travelling without my Zen 4 machine, or I could test it. ;) Oh well, Compiler Explorer is enough to look at these microbenchmarks on your own.

These simulations are single core to avoid core-to-core latency. Number of cores isn't relevant unless you want to run independent voices/channels and sum them at the end.

So you start with a very optimistic ~90 GFLOPs of 64-bit FMA on Zen 4. Unfortunately, not all operations are clean multiply-adds. Realistically, you'll need trigonometric functions and LUTs, which are quite slower. Btw, the tradeoff between when to compute vs LUT is very fragile and can change due to a ton of factors (notably integrator algorithm).

Then the data you are operating on won't fit cleanly in AVX-512 registers, requiring spills to L1 cache. Ok, still fast on a modern core.

Of course, the peak theoretical number assumes clean vectorization with double-pumped AVX-512... which also won't happen in practice. Classical DSP will fare better (https://www.youtube.com/watch?v=Ssq0a-YdamM) but SPICE integrators are inherently branchy and divergent. Especially for adaptive integrators, you'll waste a lot of operations trying to "lock in" at the exact time point where the waveform turns a corner. Apple Silicon is better at this messy, branchy code.

So yeah, it's possible-but-hard to hit hard-realtime under these conditions.

Call me when it stops making things up.

We are here.

Yes, it can still produce value.

Sure, that works. Caveats:

1. Your frontend needs to support emitting WASM. Ok for Rust, not for every language out there.

2. 2x-4x performance overhead.

If I understand this correctly, you could run this compiler on iPadOS (since it's plain C). But you cannot mark it's output executable, since that would require JIT entitlement.

I think given how fast computers have become relative to digital audio there is probably a good case to just make any "modular synth" run at 32-bit 480KHz or even 4.8MHz through every stage that could process the audio.

1. It should run at FP64 if you want to preserve filter resonances, etc.

2. At 10x/100x fixed-rate oversampling, even a modern "fast" CPU will have very few cycles per (higher-rate) sample to run the DSP for 1 "module" of the software modular. Forget about interconnected modules, multiple tracks, or polyphony. For this kind of "analog"-style processing, it's better to run adaptive-rate algorithms (think SPICE) instead of wasting compute on unnecessary extra audio samples.

the following audio processing is better done after converting the samples to FP32

Or, in some cases, FP64.

Pre-ordered one with gyro+accelerometer for livecoding.

ability to create a self-contained binary

so i am very happy to see a high level language get that potential. besides go only common lisp has that ability that i am aware of.

Julia can do it.