seertaak

Contrapunk is a cool name though.

How are you finding rust for audio development? I have a background in pro audio, and both for the audio and GPU render threads, I used a lot of arena allocators and ring buffers. Do you find yourself fighting with rust's strict semantics?

Gradus ad Parnassum! What a cool idea, and the fact it's counterpoint gives you a nice little time buffer for any DSP. Super cool

I guess it's possible that Russia and/or China delivered some hardware to the Iranians. Doesn't seem far fetched given the low international support for this "excursion". Both countries benefit from a US quagmire.

The irony of an IP scraper on an absolutely breathtaking, epic scale getting its secret sauce "scraped" - because the whole app is vibe coded (and the vibe coders appear to be oblivious to things like code obfuscation cuz move fast!)...

And so now the copy cats can ofc claim this is totally not a copy at all, it's actually Opus. No license violation, no siree!

It's fucking hilarious is what it is, it's just too much.

The irony of an IP scraper on an absolutely breathtaking, epic scale getting its secret sauce "scraped" - because the whole app is vibe coded (and the vibe coders appear to be oblivious to things like code obfuscation cuz move fast!)...

And so now the copy cats can ofc claim this is totally not a copy at all, it's actually Opus. No license violation, no siree!

It's fucking hilarious is what it is, it's just too much.

Interesting article; I've always been fascinated and intimidated by FPGA programming - it's one of the few remaining "dark arts" of software engineering.

VHDL’s delta cycle algorithm is its crown jewel. It gives you built-in determinism. Let us treasure it - Verilog doesn’t have anything like it. At the same time, you will agree with me that there is nothing too complicated about the concept.

I agree with you -- and thank you!

The deal was always simple: search engines had permission to crawl sites because they were going to be sending users to those sites. If they're hitting your site half a million times for every one user they send to your site, all they're giving you is higher costs.

Agree 100%.

Integer promotion? - Stroustroup pleads C source compat else stillborn.

Initializes lists suck mainly because of C source compat constraints, too. In fact, most things that suck in C++ came from B via C.

You made me laugh!...Bjarne indeed can't be accused of being a modest man. And by some accounts, he's quite a political animal.

But in fairness, when was D&E first published? Argued for auto there, long before their acceptance. Argued for implicit template instantiation - thank god the "everything-must-be-explicit" curmudgeons were vanquished there, too.

He's got a pretty good batting average - certainly better than Herb Sutter.

Can you share what aspects of the design you (and Stroustroup) aren't happy with? Stroustroup has a tendency of being proven right, with 1-3 decade lag.

I'll have a stab at this. I'll start with an attempt at justifying the remark that an agent which is a good coder will be good at other tasks.

1. Coding is, as a technical endeavour, relatively difficult (similarly for mathematics). So a model which performs well on this task can be expected to easily handle also-technical-but-slightly-easier tasks, like understanding (musical) harmony theory or counterpoint -- for much the same reason that human programmers/mathematicians/scientist don't struggle to understand those "easier" theories.

2. Reinforcement learning augments a base models ability to excel in something else that's "difficult", namely to "look ahead" and plan multiple steps in advance. That's literally how the training algorithm works, generating multiple paths at once, and rewarding intermediate steps in those paths which succeed in attaining the goal. And that skill, too, is extremely useful in other domains. An AI agent which learns that to break a problem into sub-problems, and then tackle each in turn methodically -- it stands to reason that it can apply that to, say, a business plan.

Note: 1 & 2 are not independent, nor are frontier models' excellence in these domains magical: it ultimately boils down to the availability of massive datasets (in particular for coding) and totally objective metrics (in the case of mathematics: solved math problems). That's the key ingrediant for reinforcement learning to be so effective.

So: the skills are transferrable because they're difficult, and require lots of planning. That models are so good at them is a fluke, and in a parallel world where humans created git repo after git repo of business plans, it might be that which we lean on to teach a reinforcement learning algorithm how to "reason" and "plan".

Now let's turn our attention to the "synergies" aspect, which I agree with. Let's say your agentic model, which is already excellent at reasoning and planning, acquires a new or improved capability which allows it to search the domain space, calculate, etc. much better than before -- this capability can now bear upon the plan, or be factored into the plan. For example, the model might be able to say "I don't need to worry about this particular subproblem for now; I can rely on my "mathematica" capability to deal with it when I absolutely need."

Or to put it differently: monkeys, like humans, are able to use (rudimentary) tools. They'll take a rock, and use it to crack open a coconut (or whatever). But a human being, with far superior reasoning and planning abilities, takes that tool, and uses it to make an even better tool -- and the result after many iterations of this process is civilization as we know it, while monkeys are still stuck trying to crack open nuts with rocks.

In my experience, an indicator that my interlocutor is (possibly) lying/BSing, when challenged for an explanation for something they did, is that they provide a list of reasons. The person who's telling the truth just gives one.

Is that really true? I would have expected by now that AI companies nowadays are doing RL on git histories, not just on the HEAD.

This -- and obviously David Ng's article -- are absolutely fascinating pieces of work.

I have a few (very naive) questions:

There is a widespread intuition, encapsulated in the very terms "feed-forward networks" and "deep neural networks", that computation in such networks is akin to a circuit wired in series. My "observation" is that residual layers offer an "escape hatch" from this, allowing layers (or sets of layers), to operate in parallel (and of course, something in between).

So here are my dumb questions:

1. Is my intuition about residual networks, at least in principle, allowing for in parallel layers, correct? Or am I missing something fundamental? Let's say the intuition is correct -- is it possible to measure the degree to which a layer operates in series or in parallel?

2. The formula for residual layers (at least to my mind) reminds of an Ornstein-Ühlenbeck time series process. If so, can we measure the degree of mean-reversion of a/several layer(s)? For me, this makes intuitive sense -- the goal of avoiding vanishing gradients feels similar to the goal of stationarity in time series processes.

3. Let's take as an article of faith the central idea of a tripartite network: input->latentspace block => reasoning block => latentspace->output block. Ng's intuition iiuc is that the reasoning block, more or less, wired in series. Intuitively, it feels like that is what it ought to be (i.e., a chain of calculations), though I'll add -- again hand-wavingly -- that OP's efforts appear to cast doubt on this conjecture. Are the two "translation" blocks wired "more" in parallel, then?

4. So what both Ng and OP did was to "tape together" the ostensibly reasoning layers -- in different ways but that's essentially it. Another thing you could do is to treat the input and output translation blocks as fixed. You now train a totally new model on a much smaller corpus of training data, only instead of feeding the input directly to your new model you feed it translated training data (similarly, your targets are now the activations at the entrance to the reasoning->output block. Let's assume it's exactly the same architecture in the middle as the standard netowrk, only it's initialized to random weights as per usual. Surely you should be able to pre-train that 6 layer reasoning network much, much faster. Has anyone tried this?

5. Having thus partitioned a very deep architecture into three distinct parts, there's no reason why you can't experiment with making the reasoning block wider or narrower. Has anyone tried that?

6. Another fun idea is to map a given input through input block and read the pre-reasoning activations. You now let that vector be a random variable and do a random walk through reasoning input space, and use this to "augment" your corpus of training data. Reasonable idea or bullshit?

Please remember, I'm only just (and belatedly) trying to wrap my head around how transformer architectures work -- I'm still waiting for my copy of "Build a Large Language Model (from scratch)"! I hope these questions aren't totally daft!

A UI design tool for TUIs -- made with Electron?... fun times!

I agree with the author that changes are needed to stymie the proliferation in malicious packages (almost certainly potentiated by the ubiquity of LLMs), but I wonder if this is the best way to go about it.

The author posits two scenarios allowing for bad actors to take control over packages to add malicious code: stealing the original maintainers credentials, and taking ownership over dormant packages.

I would guess the second is much more likely. At least, it's the only one I've witnessed myself (in Arch Linux - I now view AURs are somewhat risky). I acknowledge this is argument based on anecdote - but what isn't up for debate is that it's far easier to identity dormant packages with high fan-out than stealing someone's credentials.

The cool down approach addresses the symptom; it would be better to address the bad actors who are the cause: dormant packages, recently taken over by maintainers lacking prior reputations and with high (transitive) usage, should be viewed with high levels of suspicion by the package managers. We should assume the new maintainers are malicious, and require them to prove they are not.

That was a thought provoking read.

From a language design viewpoint, I wonder whether the current crop of async implementations - eg 'def' -> 'async def', f(x) -> 'await f(x)', 'return 1' -> 'async return 1' are making a mistake analogous to the decades-long opposition to type deduction, ostensibly for "clarity".

Let me phrase this as a question to prevent "social" arguments against a homogenous syntax for coroutine: if coroutines were await-invoked using the same syntax as regular functions were invoked, would that create some unresolvable ambiguity for an important use-case (and similarly for the other syntactic forms)?

My gut feeling is that the answer is no. Consider what a C++ compiler does with an awaitable function (this may be specific to stackless coroutines): they are transformed into state machines, each state lowers into something akin to a regular function, local variables accessed across states are stored in a dynamic environment. Ok, but then isn't a regular function simply a coroutine whose lowered state machine has a single state? And that being so, why distinguish syntactically between the two?

It seems silly, to my mind, to do all this "work" (in the compiler) to create an abstraction which allows the programmer to forget whether 'f(x)' returns synchronously or asynchronously, only to introduce a syntax which requires them to know or look up which of the two it is. What does it matter which of the two it is, if the whole point of the exercise was to turn callback spaghetti into linear code (and allow to leverage automatic variable lifetimes aka RAII)?

The whole point is that this fits on an ESP32, which has wifi. We're not quite at the point where it makes sense to run the whole thing locally - if you do try it, it will need a fan, and be loud etc.

For my part, I installed Nanoclaw on my Arch derived OS (I love Arch!), and it worked fine until the next day some update decided to revert the power management settings, and now my glorious assistant is dead.

There's something to be said for a barebones OS. No bullshit, no updates.

Also, playing with hardware watchdog timers and GPIOs and DACs can be so much fun.

How are JVM startup times nowadays? Frankly the need to ship with a JVM (modulo graal admittedly) is a drawback, eg for CLI tools - which the author listed as a requirement. Go's static executable story, combined with it's competitive performance - without C++ or rust contortions - is a strong combo when your focus is on startup time and deployment simplicity. (If, otoh, I cared a lot about GC I'd definitely prefer Java - eg for non-fast algo trading.)

Java is a fine language, tech stack, and ecosystem, but I agree with the author and parent commenter that this is a sweet spot for Go. Their decision to use it makes a lot of sense.

Congrats on creating and helming such a cool project!

Out of curiosity, does ghostty do the Quake terminal thing - I use yakuake for this, but it feels a bit long in the tooth.

I "pray" that you'll learn how work in an inclusive and non-toxic work environment.

What did I say? I lean on LLMs because I can't get help without being subjected to unnecessary degradation.

And what do you and others do? You immediately prove my point by saying things that amount to unnecessary degradation.

I'd be remiss to point out we went from "LLMs are vaporware" to "people are becoming slaves to their LLMs" awful quick.

[I'm scared] you are growing dependent on stilts that could disappear any moment.

First, I do control the RTX3070 I own, and that can actually do a pretty decent job nowadays with some of the 3B parameter models.

Second, maybe if people like you showed as much concern for the fact that LGBT people can expect family violence as you do for Dr. Strangelove scenarios, then people like me wouldn't have to lean on LLMs so heavily.

Third, it's hilarious that your response to a comment pointing out how difficult it was to get help from another human without being degraded, was to degrade me by calling me an LLM junkie. Maybe you should be worried that Gemini appears to have more capacity for empathy and self-awareness than you.

Fourth, given that you show absolutely zero concern or willingness to help when it comes to the difficulties faced by LGBT people or ADHDers, my advice to you is to take your fears and shove them up your ass.

Sure! First, here are references, in case you want to deep dive:

1. http://lucacardelli.name/Papers/Binary.pdf

2. https://www.researchgate.net/publication/221321423_Parasitic...

Second, asymmetric multimethods give something up: symmetry is a desirable property -- it's more faithful to mathematics, for instance. There's a priori no reason to preference the first argument over the second.

So why do I think they are promising?

1. You're not giving up that much. These are still real multimethods. The papers above show how these can still easily express things like multiplication of a band diagonal matrix with a sparse matrix. The first paper (which focuses purely on binary operators) points out it can handle set membership for arbitrary elements and sets.

2. Fidelity to mathematics is a fine thing, but it behooves us to remember we are designing a programming language. Programmers are already familiar with the notion that the receiver is special -- we even have a nice notation, UFCS, which makes this idea clear. (My language will certainly have UFCS.) So you're not asking the programmer to make a big conceptual leap to understand the mechanics of asymmetric multimethods.

3. The type checking of asymmetric multimethods is vastly simpler than symmetric multimethods. Your algorithm is essentially a sort among the various candidate multimethod instances. For symmetric multimethods, choosing which candidate multimethod "wins" requires PhD level techniques, and the algorithms can explode exponentially with the arity of the function. Not so with asymmetric multimethods: a "winner" can be determined argument by argument, from left to right. It's literally a lexicographical sort, with each step being totally trivial -- which multimethod has a more specific argument at that position (having eliminated all the candidates given the prior argument position). So type checking now has two desirable properties. First, it design principle espoused by Bjarne Stroustroup (my personal language designer "hero"): the compiler implementation should use well-known, straightforward techniques. (This is listed as a reason for choosing a nominal type system in Design And Evolution of C++ -- an excellent and depressing book to read. [Because anything you thought of, Bjarne already thought of in the 80s and 90s.]) Second, this algorithm has no polynomial or exponential explosion: it's fast as hell.

4. Aside from being faster and easier to implement, the asymmetry also "settles" ambiguities which would exist if you adopted symmetric multimethods. This is a real problem in languages, like Julia, with symmetric multimethods. The implementers of that language resort to heuristics, both to avoid undesired ambiguities, and explosions in compile times. I anticipate that library implementers will be able to leverage this facility for disambiguation, in a manner similar to (but not quite the same) as C++ distinguishes between forward and random access iterators using empty marker types as the last argument. So while technically being a disadvantage, I think it will actually be a useful device -- precisely because the type checking mechanism is so predictable.

5. This predictability also makes the job of the programmer easier: they can form an intuition of which candidate method will be selected much more readily in the case of asymmetric multimethods than symmetric ones. You already know the trick the compiler is using: it's just double-dispatch, the trick used for "hit tests" of shapes against each other. Only here, it can be extended to more than two arguments, and of course, the compiler writes the overloads for you. (And it won't actually write overloads, it will do what I said above: form a lexicographical sort over the set of multimethods, and lower this into a set of tables which can be traversed dynamically, or when the types are concrete, the compiler can leverage monomorphize -- the series of "if arg1 extends Tk" etc. is done in the compiler instead of at runtime. (But it's the same data structure.)

6. It's basically impossible to do separate compilation using symmetric multimethods. With asymmetric multimethods, it's trivial. To form an intuition, simply remember that double-dispatch can easily be done using separate compilation. Separate compilation is mentioned as a feature in both the cited papers. This is, in my view, a huge advantage. I admit, this I haven't quite figured out generics will fit into this -- at least if you follow C++'s approach, you'll have to give up some aspects of separate compilation. My bet is that this won't matter so much; the type checking ought to be so much faster that even when a template needs to be instantiated at a callsite, the faster and simpler algorithm will mean the user experience will still be very good -- certainly faster than C++ (which uses a symmetric algorithm for type checking of function overloads).

To go a bit more into my "vision" -- the papers were written during a time when object-orientation was the dominant paradigm. I'd like to relax this somewhat: instead of classes, there will only be structs. And there won't be instance methods, everything will be a multimethods. So instead of the multimethods being "encapsulated" in their classes, they'll be encapsulated in the module in which they're defined. I'll adopt the Python approach where everything is public, so you need to worry about accessibility. Together with UFCS, this means there is no "privileging" of the writer of a library. It's not like in C++ or Java, where only the writer of the library can leverage the succinct dot notation to access frequently used methods. An extension can import a library, write a multimethod providing new functionality, and that can be used -- using the exact same notation as the methods of the library itself. (I always sigh when I read languages, having made the mistake of distinguishing between free functions and instance methods, "fix" the problem that you can only extend a library from the outside using free functions -- which have a less convenient syntax -- by adding yet another type of function, an "extension function. In my language, there are only structs and functions -- it has the same simplicity as Zig and C in this sense, only my functions are multimethods.)

Together with my ideas for how the parser will work, I think this language will offer -- much like Julia -- attractive opportunities to extend libraries -- and compose libraries that weren't designed to work "together".

And yeah, Claude Code and Gemini are going to implement it. Probably in Python first, just for initial testing, and then they'll port it to C++ (or possibly self-host).

To my mind at least, it is different. I lean heavily on AI for both admin and coding tasks. I just filled out a multipage form to determine my alimony payments in Germany. Gemini was an absolute godsend, helping answer questions in, translate to English, draft explanations, emails requesting time extensions to the Jugendamt case worker.

This is super scary stuff for an ADHDer like me.

I have an idea for a programming language based on asymmetric multimethods and whitespace sensitive, Pratt-parsing powered syntax extensibility. Gemini and Claude are going to be instrumental in getting that done in a reasonable amount of time.

My daily todos are now being handled by NanoClaw.

These are already real products, it's not mere hype. Simply no comparison to blockchain or NFTs or the other tech mentioned. Is some of the press on AI overly optimistic? Sure.

But especially for someone who suffers from ADHD (and a lot of debilitating trauma and depression), and can't rely on their (transphobic) family for support -- it's literally the only source of help, however imperfect, which doesn't degrade me for having this affliction. It makes things much less scary and overwhelming, and I honestly don't know where I'd be without it.

Absolutely! -- and we could play this game for a long time ;)

The right way of looking at it is, there was tiny little interlude of something vaguely approaching the free market -- back when Volcker was in charge.

If you use Arch Linux, there's now an AUR package to install it -- made by yours truly.

https://aur.archlinux.org/packages/nanoclaw-git

Sorry, when you say "gloriously free market", do you mean whatever it takes EU, helicopter money (or, rewinding a decade, Greenspan put) US, or factory of the world China? :)

My point is that it's not a real market economy if the risk premium -- and in China's case, the exchange rate -- is rigged. And it has been, since the 90s.

EDIT: For clarity, I'm agreeing with you, since you were being facetious.

Funny and excellent comment!

And it ... works?

It gets the job done, but the existence of Cython, Numba, Pythran, PyPy and many, many, others are indication that this isn't a global optimum.

First, let's distinguish between two types of syntactic constructs: null and left denominations. (Terminology borrowed from Pratt parsers.) Null denominations can exist on their own, left denominations can't -- they are inherently chained (eg arithmetic expressions, statements in a block, or elements of a tuple), and allow a succinct, infix notation for variable-length constructs (no lispy parentheses hell).

Second, null denominations usually introduce names -- whether for variables, types, functions, lifetimes, macros, etc. One exception to this are free-standing value expressions (a bit weird; less so when they're the last expression in a block indicating the value returned by it). Another other exception would be directive type constructs - eg directives to import names from another module, directives to give hints to the compiler, etc. The last two exceptions are the most common ones: variable assignment and function invocation.

The golden rule of good language design, as I see it, is this: null denominations must begin with a fixed and unique token. The only permissible exceptions should be for assignment and function invocation; exceptions which exist because those use-cases appear so often in a typical program that requiring a prefix would be insufferable.

Julia breaks this rule for global variables. (Fair enough, Python also commits this error, but it's a mistake and a source of bugs!) But wait, Julia also has "const" and "local" binding constructs, where it follows the golden rule -- but now your syntax isn't consistent. So now you need to keep in your head these nuances -- and know the difference between a soft and hard scope -- when you want to write a function which modifies a function using macrology.

(As a point of taste on the choice of prefix token: introduction of variables through "local" is just as weird as C++'s "auto" -- and at least Bjarne Stroustroup had an excuse for that choice. Anyone who introduces a global variable in a local scope should be punished imho, so there's no need to say "this is a local variable", it's obvious from the fact that the name is introduced inside a function. Instead, my personal preference is to introduce constants through "let", and variables through "var". The former is well-known to anyone numerate, and the latter is ubiquitous in software engineering. Both read well; they're as close as possible as you can get to constructs in English.)

Julia breaks the golden rule again with its succinct, Mathematica-style notation for function definition. I get that it wants to appeal to Mathematica users, but Python already proved you don't need to do that. This is a programming language; brainy types, like mathematicians and physicists, aren't going to be flummoxed by an unfamiliar notation for function definition, or irritated by having to type a few extra characters.

I mention macrology; it's not just that. Let's say you want write a syntax highlighter -- you need to take into account all that weirdness. If null denominations have a fixed & unique prefix, parsing is easy-peasy. Want to add a capability to "inline" HTML code within Julia, react-style? You're going to run into similar issues. And so on...