pash

Right, that’s why I wrote, “I don’t mean that most neural networks are invertible functions.”

For a neural network that is not bijective, you can obtain an input that maps to a desired output by the following algorithm.

1. Start with a trained neural network. (The weights will not change throughout this procedure.)

2. Pick a random input.

3. Given an output for which you want to compute an associated input, feed the input into the network to compute the output.

4. Compute the loss of the computed output relative to the target output (e.g., mean-square error). If the loss is sufficiently small, you’ve found an input that maps to an output close to your target output and you’re done.

5. Otherwise, compute the gradient of the loss with respect to the input (e.g., by backprop).

6. Update the input according to a gradient-update rule. And go back to Step 3.

In theory, you can recover a “representative” prompt for the output of an LLM in this manner. For outputs that could have been generated by a large set of disparate prompts, obviously this won’t work well.

The obvious solution is to run things in reverse, inputting the AI-generated output to recover the prompt that generated it.

Most generative models can be run in reverse by algorithms that already exist [0], but you have to have the model weights. For closed-weight models, or for a process that can handle unknown models, you’d have to do some engineering.

But do we have the technology to build models that back out the prompt from suspected AI output? Yes.

0. I don’t mean that most neural networks are invertible functions. They’re not. But you can do backprop in reverse, from output to input, to train a model to generate an input to the original model that best predicts its output.

The inflection point was 2012, when AlexNet [0], a deep convolutional neural net, achieved a step-change improvement in the ImageNet classification competition.

After seeing AlexNet’s results, all of the major ML imaging labs switched to deep CNNs, and other approaches almost completely disappeared from SOTA imaging competitions. Over the next few years, deep neural networks took over in other ML domains as well.

The conventional wisdom is that it was the combination of (1) exponentially more compute than in earlier eras with (2) exponentially larger, high-quality datasets (e.g., the curated and hand-labeled ImageNet set) that finally allowed deep neural networks to shine.

The development of “attention” was particularly valuable in learning complex relationships among somewhat freely ordered sequential data like text, but I think most ML people now think of neural-network architectures as being, essentially, choices of tradeoffs that facilitate learning in one context or another when data and compute are in short supply, but not as being fundamental to learning. The “bitter lesson” [1] is that more compute and more data eventually beats better models that don’t scale.

Consider this: humans have on the order of 10^11 neurons in their body, dogs have 10^9, and mice have 10^7. What jumps out at me about those numbers is that they’re all big. Even a mouse needs hundreds of millions of neurons to do what a mouse does.

Intelligence, even of a limited sort, seems to emerge only after crossing a high threshold of compute capacity. Probably this has to do with the need for a lot of parameters to deal with the intrinsic complexity of a complex learning environment. (Mice and men both exist in the same physical reality.)

On the other hand, we know many simple techniques with low parameter counts that work well (or are even proved to be optimal) on simple or stylized problems. “Learning” and “intelligence”, in the way we use the words, tends to imply a complex environment, and complexity by its nature requires a large number of parameters to model.

0. https://en.wikipedia.org/wiki/AlexNet

1. https://en.wikipedia.org/wiki/Bitter_lesson

https://en.wikipedia.org/wiki/Sprezzatura

https://en.wikipedia.org/wiki/Wabi-sabi

Even for billion-parameter theories, a small amount of vectors might dominate the behaviour.

We kinda-sorta already know this is true. The lottery-ticket hypothesis [0] says that every large network contains a randomly initialized small network that performs as well as the overall network, and over the past eight years or so researchers have indeed managed to find small networks inside large networks of many different architectures that demonstrate this phenomenon.

Nobody talks much about the lottery-ticket hypothesis these days because it isn’t practically useful at the moment. (With the pruning algorithms and hardware we have, pruning is more costly than just training a big network.) But the basic idea does suggest that there may be hope for interpretability, at least in the odd application here or there.

That is, the (strong) lottery-ticket hypothesis suggests that the training process is a search through a large parameter space for a small network that already (by random initialization) exhibit the overall desired network behavior; updating parameters during the training process is mostly about turning off the irrelevant parts of the network.

For some applications, one would think that the small sub-network hiding in there somewhere might be small enough to be interpretable. I won’t be surprised if some day not too far into the future scientists investigating neural networks start to identify good interpretable models of phenomena of intermediate complexity (those phenomena that are too complex to be amenable to classic scientific techniques, but simple enough that neural networks trained to exhibit the phenomena yield unusually small active sub-networks).

0. https://en.wikipedia.org/wiki/Lottery_ticket_hypothesis

Sandvault [0] (whose author is around here somewhere), is another approach that combines sandbox-exe with the grand daddy of system sandboxes, the Unix user system.

Basically, give an agent its own unprivileged user account (interacting with it via sudo, SSH, and shared directories), then add sandbox-exe on top for finer-grained control of access to system resources.

0. https://github.com/webcoyote/sandvault

OK, let’s survey how everybody is sandboxing their AI coding agents in early 2026.

What I’ve seen suggests the most common answers are (a) “containers” and (b) “YOLO!” (maybe adding, “Please play nice, agent.”).

One approach that I’m about to try is Sandvault [0] (macOS only), which uses the good old Unix user system together with some added precautions. Basically, give an agent its own unprivileged user account and interact with it via sudo, SSH, and shared directories.

0. https://github.com/webcoyote/sandvault

Milton Friedman wouldn’t have approved of a basic-income scheme restricted to artists. He would have argued that restricting the benefit to artists would distort incentives for choosing a profession in a way likely to reduce social welfare, and that eligibility by profession is a “welfare trap”: it’s hard to stop being an artist and start being something else when it means losing your guaranteed income.

But Friedman would have supported a broad basic-income scheme. We know this because he did support one. It was his proposal in 1962 of a “negative income tax” [0] (in Capitalism and Freedom) that gave rise to the movement to replace traditional social welfare programs with simple schemes that just give money to poor people. (This movement led to the Earned Income Tax Credit [1] in the United States.)

Friedman’s negative income tax is equivalent to the contemporary notion of a guaranteed basic income (but not to a universal basic income, as only people earning below some threshold would receive it). Like most economists, Friedman believed that people (even poor people) can typically make their own economic choices better than a government program can make those choices for them. (He was likewise not opposed to redistributive policies per se.) That was the root of his advocacy for market-based mechanisms of organizing the economy.

0. The idea dates to at least the 1940’s, but Friedman’s book is typically credited with popularizing it. See, e.g, https://en.wikipedia.org/wiki/Negative_income_tax.

1. https://en.wikipedia.org/wiki/Earned_income_tax_credit

From what I understand, which is very incomplete, the leading hypothesis at the moment is that ingested prions are a bit hard to digest (because they’re malformed proteins), so they end up making it out of the gastrointestinal tract somehow, interacting with the nervous system via the intestinal lining or lymphatic system. Then they travel to the brain via nervous pathways, by-passing the usual blood–brain barrier.

But transmission of prions by ingestion is thought to be quite rare, as that mechanism suggests. Transmission by any means seems to be quite rare, even heritable transmission (e.g., vCJD). So that’s why it seems unlikely that whatever is happening in New Brunswick is CVD.

But if it’s not some minor mass hysteria, then maybe prions.

I’m not qualified to comment intelligently on what might be going on here, but I’d like to add some background color that the article lacks.

Creutzfeldt–Jakob Disease is a prion disease [0] for which there is no definitive diagnosis in vivo. A confident diagnosis can be made only after examining brain tissue under a microscope.

Prions are an unusual type of mis-folded protein that induce other proteins to take on a similar mis-folded shape when they come into contact with them. The mis-folded shape of the prion itself is what causes the mis-folding in adjacent proteins. It’s a chemical-bonding thing at the molecular level. It’s the shape of the prion that causes other proteins to take on a similar shape and become prions, etc.

Some prion diseases occur spontaneously (when a protein takes on a mis-folded configuration due to mis-transcription or random energetic impulses) and some are transmitted, typically by eating some part of an animal that contains prions, which then end up in your own body, inducing proteins in your body to take on prion configurations.

Prion diseases are the only known transmissible diseases that do not involve the replication of a pathogen’s genetic material in a host cell. The only known prion diseases affect nervous tissues, and in humans the only known prion diseases affect brain tissues.

I’m not an expert on prion diseases, but I’ve had a bit of a fascination with them since having to report on a bunch of USDA surveillance lectures on mad-cow disease (bovine spongiform encephalopathy, BSE) and to summarize a bunch of symposia on prion diseases in a previous life. The symptoms reported in the article sound very much like a prion disease, and the tests for CJD indicate that the doctors in the region suspect as much.

But we simply don’t have good tests for prion diseases in vivo. And prion diseases are not well understood in general, so it wouldn’t be surprising that a new one would present as something of a mystery.

It is also the case that I know very little about New Brunswick, but I will mention that prion diseases in humans are thought to be far more commonly acquired than spontaneous. The most common cause of acquisition is eating animals with endemic prion diseases; this is most often nervous tissue of venison, but rarely nervous tissue of cattle infected with BSE, which is present in Canada more than anywhere else (by a small margin).

It is also possible (but not likely) that a prion disease can arise de novo.

0. https://en.wikipedia.org/wiki/Prion_disease

Philip E Converse, The Nature of Belief Systems in Mass Publics (1964), 75 pages [0].

0. https://web.ics.purdue.edu/~hoganr/Soc%20312/The%20nature%20... [PDF]

We should be able to define our functions to accept the inputs they are designed for, and not try to handle every possible edge case.

Oh, look, somebody just re-discovered static typing.

For some definition of “sufficiently introductory”, I’d recommend starting with the first chapter of John Nolan’s book Stable Distributions [0] (20 pages), which presents the class of distributions to which sums of iid random variables converge and builds up to a version of the generalized CLT.

Note that this generalization of the classical CLT relaxes the requirement of finite mean and variance but still requires that the summed random variables are iid. There are further generalizations to sums of dependent random variables. John D. Cook has a good blog post that gives a quick overview of these generalizations [1].

0. https://edspace.american.edu/jpnolan/wp-content/uploads/site... [PDF]

1. https://www.johndcook.com/blog/central_limit_theorems/

I’m not sure what the author had in mind when he wrote the paper, but its light-hearted content reveals well how to structure a math paper: (1) state the problem and why the reader should care about it, (2) state how the paper builds on prior work, (3) summarize the main results, and then (4) develop them in theorem-proof style.

An economics paper in the same spirit (though of course economics addresses more practical problems than mathematics, in this case whether one should leave the toilet seat up or down) shows quite well how to present and develop an economic model [0].

0. https://jaypilchoi.com/wp-content/uploads/2021/06/Economic-I... [PDF]

The general version of this is called inverse transform sampling [0], which uses the fact that for the cdf F of any random variable X the random variable Y = F(X) has a standard uniform distribution [1]. Since every cdf increases monotonically on the unit interval, every cdf is invertible [2]. So apply the inverse cdf to both sides of the previous equation and you get F^-1(Y) = X is distributed like X.

Sampling from a standard uniform distribution and then using the inverse transform is the commonest way of generating random numbers from an arbitrary distribution.

0. https://en.m.wikipedia.org/wiki/Inverse_transform_sampling

1. https://en.m.wikipedia.org/wiki/Probability_integral_transfo...

2. Not every cdf is one-to-one, however, so you may need a generalized inverse.

The Fed’s estimates thru 2022 say real estate makes up 46.8% of net US wealth ($83.5 trillion of $178.4 trillion) [0].

0. Readably summarized at https://www.barrons.com/amp/articles/net-wealth-is-over-135-... .

Yes, it’s a bit surprising. The whole point of a constitution is to provide a groundwork and a set of meta-rules governing how the ordinary rules of society may be made and modified, and how disputes about those sorts of thing should be resolved. The judiciary is precisely the mechanism for entertaining and deciding these disputes.

The odd thing, really, is that it took a decade and a half, until 1803, for the Supreme Court first to find it necessary to tell another of the other branches of government that it had broken the rules.

Article III, Sec. 2, clearly, which begins:

The judicial Power shall extend to all Cases, in Law and Equity, arising under this Constitution …

Let’s not continue this conversation. The view I’m presenting is long established and can be found better and much more extensively made in a great many sources which no one should have any trouble finding, should he wish. You, I infer, do not wish.

The role of the courts in deciding the constitutionality of legislation was very much in the zeitgeist of the American experiment. Federalist, No. 78:

It is not otherwise to be supposed, that the Constitution could intend to enable the representatives of the people to substitute their will to that of their constituents. It is far more rational to suppose, that the courts were designed to be an intermediate body between the people and the legislature, in order, among other things, to keep the latter within the limits assigned to their authority. [0]

It is absurd to say that the Supreme Court “magicked out of thin air” the notion of judicial review in Marbury v. Madison. That was simply the Court's first assertion of the power the constitution gave it to wield, the wielding of which is the primary purpose of the judiciary as a separate but equal branch of American government.

0. https://en.m.wikipedia.org/wiki/Federalist_No._78#Judicial_r...

You’re right, sorry. Reserve Primary was the only fund whose NAV was below the Treasury’s qualifying threshold, so it did not participate in the guarantee program. I had the mistaken recollection that Reserve Primary’s NAV had returned above the $0.995 level by the 19th, but that was not the case.

(In fact, the SEC ended up suing Reserve Primary’s managers for making misleading statements after breaking the buck that they would restore a $1 NAV, which they did not do. Reserve Primary’s investors had an eventual recovery as of 2011 of 99.06% of par value.)

Of the funds that participated in the Treasury’s guarantee program, none were liquidated likely because the program succeeded in stemming the run on the money-market sector. As I’m sure you know, the whole point of the Treasury’s guarantee was to give money-market investors confidence that their money was safe so that they would not redeem their shares and the funds would not need to liquidate assets for payouts.

It worked, and I think it’s clear that the Treasury’s backstop, by ending the run, prevented fire-sales that likely would have led to the liquidation of further funds in the absence of government intervention. That was the point I was making in my original comment: despite the lack of a formal equivalent of deposit insurance for money-market funds, the government did step in to backstop investor’s money in the sector during the only episode in which there was a need for it.

Edit: Whoops, also noticed that in my original comment above, I twice mis-wrote money-market account. Everything we’ve been discussing relates to money-market funds (more formally, money-market mutual funds), not money-market accounts. The latter term is another name for savings accounts at a bank, which of course are FDIC-insured. (The two are related in that the cash put into both gets invested in short-term, low-risk government and commercial bonds, the markets for which are collectively called “money markets”.)

Two new writers to read! Interesting, if it’s really the case, that there are only a couple of twentieth-century British novelists whose cult reputations outstrip their wider reputations. I feel like there are a number of American authors who fit that description, e.g., John Barth, William Gaddis.

Yes, really. The Treasury guaranteed dollar-for-dollar redemptions for all investors’ holdings at the time the RPF broke the buck, following Lehman’s failure several days before [0, 1]. Only those who speculated on the value of the fund after that time lost money, and very little.

0. https://home.treasury.gov/news/press-releases/hp1161

1. https://elischolar.library.yale.edu/cgi/viewcontent.cgi?arti...

Money-market accounts are not covered by government deposit insurance. But money-market funds make a return for themselves by investing their customers’ cash in risky assets, similarly to a bank. If there’s a run on your money-market fund akin to the bank runs we’ve recently seen, the FDIC isn’t going to save you.

(Kinda. The other big distinction between a bank and a money-market fund is that the latter don’t indulge the farce of “demand deposits”. Money-market accounts are fixed-term securities, so you’ve agreed to lock up your money until some agreed upon future date, just like with a bond. That might be inconvenient for you, in comparison to a bank account, but it vastly reduces the complexity the fund faces in matching maturities of liabilities and assets to minimize run risk. As a result, together with stricter regulatory controls on their risk-taking, money-market funds fail much less often than banks do. The last time a US money-market fund broke the buck, in 2008, during the last financial crisis, the Treasury stepped in to make investors whole.)

Northrop Grumman announced in September that they would unveil the B-21 at Palmdale this week [0]. The location was not a secret.

0. https://www.globenewswire.com/news-release/2022/09/20/251965...

This is the essence of the method described in the link, and the ability to pull it off is called "mathematical maturity". How one acquires mathematical maturity is the matter of some debate.

Sorry, re-reading my comment I can see that it is likely only to add to the confusion. (The downvotes suggest so also! Apologies all around.)

I meant to make two points. First, that IMO it is misleading to say that a bank simply creates new money ex nihilo when making a loan, counter to what you wrote ("the bank does not need to have $10k sourced from somewhere else"). There is in fact a bound on the amount of new lending a bank can undertake, one related to its assets, and in that sense the money lent does "come from somewhere." My attempt at a (poorly) stylized accounting was meant to illustrate what sort of thing makes up a bank's cash assets and how they relate to lending, but I'll try a new approach below. Second, I was trying to clarify what "bank money" means, to explain how it gets created, as much of the broader confusion in this thread seems to be around what "[bank] money-creation" really entails.

Let me try to make my first point again. In modern banking systems, fractional reserve has been mostly replaced by capital requirements, which limit a bank to holding an amount of (risk-weighted) assets no greater than some multiple of its capital. A bank's capital is its assets less its liabilities, which we'll write A - L. Let's call the regulatory capital multiple M, then capital requirements essentially say that A <= M(A - L). When a bank makes a new loan, its assets and liabilities increase equally in the way you described. That means that for a loan of X, if the lending bank is not already bounded by capital requirements, that the capital adequacy inequality becomes X + A < M(A + X - (L + X)) = M(A - L). That is, the left side of the inequality increases but the right side of the inequality is unchanged, so the capital requirement binds more tightly. In particular, it must be that the loan amount X < M(A - L) - A; in other words, new loans are constrained to be less than the excess of regulatory capital over pre-existing assets. So a bank cannot just create arbitrarily large new loans; loans are constrained by capital, the positive component of which is assets. That's where loan money "comes from": a bank's assets.

The second point I meant to make about bank money-creation was not directed at your comment specifically. But I wanted to point out that "bank money" simply means the sum of all bank deposits, and that it, too, is constrained to be a finite number. In this sense also banks cannot poof into existence arbitrarily large loans (with corresponding deposits) in the way I read your comment to imply.

If you apply for a $10k loan from a bank then, once the loan is approved, the bank will simultaneously:

- increase your current account balance by $10k (credit your current account) - increase your loan account balance by $10k (debit your loan account)

That $10k deposit appeared from nowhere. You didn't 'put the deposit in', and it wasn't transferred from another customer or another bank. And the bank does not need to have $10k sourced from somewhere else.

This is not correct. A bank lends against its cash assets, not against nothing.

Bank accounting works like so: when you deposit $1 at the bank, the bank’s assets increase by $1 (the $1 you just gave it), and its liabilities also increase by $1 (the bank owes you your dollar back when you want it). A bank’s loans are financed by its assets; each dollar it lends is a dollar taken from the asset side of its balance sheet. When a bank lends you $1, it (1) decrements its asset balance by $1, (2) increments your bank account by $1, (3) creates a new asset representing a $1 loan on its own balance sheet, and (4) creates a new liability for you representing a loan of $1 owed.

Point is, a bank does not create money from absolutely nothing when making loans in the way you describe. All loans are financed by the bank’s assets, and a bank cannot originate a larger value of loans than the value of its cash assets. Bank “money-creation” refers to the fact that the total bank account balances in the economy increase when a bank makes a loan; this happens after what I described in the previous paragraph, since the $1 loan now increments your bank account, while nobody else’s bank account balance decreases.

(To see how this works in more detail, imagine the process begins with Alice depositing a one-dollar bill at the bank, and then the bank loaning Bob one dollar. Keep track of the distinction between who has the one-dollar bill and who has a one-dollar bank account balance. The loan proceeds as follows: first the bank gives Bob the one-dollar bill it took from Alice as a deposit, then Bob turns right around and deposits the one-dollar bill back at the bank. The result is that Alice and Bob now both have $1 bank account balances after the loan is made, whereas only Alice did before; the bank still has the one-dollar bill. Summing either bank account balances alone or balances plus dollar bills shows $1 more after the loan was made.)

Topology is a canonical example of "mathematics without numbers" (and indeed of "qualitative" rather than "quantitative" mathematics), so for anyone approaching it with an undergrad or lesser background in math, learning it feels a bit like starting with a clean slate. As benrbray mentions in a sibling comment, there aren't any prerequisites in the usual sense, as you won't need to draw on any reserve of mathematical facts built up in learning previous topics while studying it. But learning it does require the ability to assimilate axioms (and to intuit why they make sense as a starting point), and then the ability to reason about those axioms rigorously to draw out the interesting consequences.

That said, a background in some mathematical topics is useful for learning topology. The subject is rooted in a particularly abstract notion of distance, so it's useful to have some experience reasoning with somewhat less abstract notations of distance like metrics, the most familiar of which is the quite numerical Euclidean metric involving the familiar square root of the sum of squares. Familiarity with metric spaces in general (that is, spaces equipped with any axiomatically acceptable notion of a metric distance) is even more useful because it requires a similar sort of axiomatic reasoning as does topology.

Kolmogorov & Fomin's classic text on analysis, the Dover edition of which is under $12 on Amazon [0], has a good (albeit austere) introduction to topology, leading up to it via axiomatic set theory and metric spaces in the book's first chapters. On the one hand, I hesitate to recommend this book, because it was the text for a course that mercilessly exposed the shortcomings in my mathematical education to that time; on the other hand, I recommend it for precisely that reason. Having later assimilated the material, I do consider the book a very good introduction for anyone who either already has the mathematical maturity to do the book properly or for anyone who wants to gain that ability in the way that everybody who has done does: by staring at the same page for hours on end while working everything out on paper, down to the axioms if necessary, until you stop misunderstanding, and then start understanding.

That said, there surely are gentler introductions to the subject if you just want to get a rough idea of it.

[0] https://www.amazon.com/Introductory-Analysis-Dover-Books-Mat...

I was trying to avoid being too detailed, but the distinction is actually that linear types require function arguments to be used exactly once and affine types require that they be used at most once. So affine types allow constant functions, but linear types don’t. A good place to start for anyone interested in learning more is the Wikipedia article on substructural types [0].

And, yes, you’re right that these type systems effectively implement reference counting at the type level. In practice, linear and affine types are used alongside more general types to permit controlled copying and destruction, which tends to look like manual (but type-enforced) memory allocation and deallocation around the edges.

0. https://en.wikipedia.org/wiki/Substructural_type_system

Linear logic and affine logic (the latter of which SrPeixinho refers to in that Reddit post) let you use an assumption only once in a proof. Their counterparts in functional programming are called linear and affine types [0]; the arguments of functions with such types are restricted to being used only once in the function’s body. This constraint lets you do some interesting things in compiling and optimizing languages built around these types, and in particular it makes garbage collection trivial.

Linear and affine types are pretty hot right now in the small world of type theory and functional programming. SrPeixinho’s work with his collaborators is aimed at building a language for the Ethereum VM that might actually make it possible to write smart contracts that don’t blow up in your face. It’s good stuff. Search his posts on Reddit to see what he’s up to [1].

0. https://en.wikipedia.org/wiki/Curry–Howard_correspondence

1. https://www.reddit.com/user/SrPeixinho