Ari_Rahikkala

I'm a big fan of Scott's, but I haven't been happy to see how this story's failure to say what it means has in some ways only made it less relevant over the years as we've come closer to building something like the item that, on the surface, it talks about.

Not to say I think it's a bad thing at all that people have a name for this concern for giving up agency to an external system, even when that system really has your best intent in mind. You can call Claude a whispering earring and people will understand what you mean pretty well. The actual consequences you're looking out for are maybe a bit subtler than your brain atrophying, I really really never liked that detail in the story, but that's fine, people will get it. By the Death of the Author, the story's a great contribution as it is.

But it just feels like a loss that there really was a different thing that the author meant, and it got lost in translation. http://web.archive.org/web/20121007235422/http://squid314.li... clarifies it. It's quite similar to a Yudkowskyan fun theory concern (and IMO really not surprising that the story was released a couple of years after the fun theory sequence): If making the right choices and doing the right thing eventually comes down to a mechanical computation, and you've got all of your resource limitations and psychological hang-ups and such things out of the way, then there's no room left to do the right thing, and you've turned yourself into a mechanism.

I don't think that's really a possible state that any system with any trace of humanity left in it will ever enter, there's too many too large search processes in the way to fit in a finite universe. So it's a pretty abstract concern even among its peers. But it's one that's fun to talk about sometimes, and one lesson we've learned from the response to this story is that mysterious, easily misinterpreted parallels in fact do not point people in the right direction to see it.

Neat. Very similar to tree-based speculation as they point out, and they also point how to combine them.

Speculative decoding: Sample a linear output (next n tokens) from draft model, submit it to a verifier model. At some index the verifier might reject a token and say that no, actually the next token should be this other token instead ("bonus token" in this paper), and that's your output. Or if it accepts the whole draft, you still get a bonus token as the next token past the draft. Then you draft again from that prefix on.

Tree-based speculation: Sample a tree of outputs from draft model, submit whole tree to verifier, pick longest accepted prefix (and its bonus token).

Speculative speculative decoding: Sample a linear output from draft model, then in parallel both verify it with the verifier model, and produce a tree of drafts branching out from different rejection points and different choices of bonus tokens at those points. When the verifier finishes, you might have have a new draft ready to submit right away.

Combined: Sample a tree from the draft model, submit the whole tree to the verifier and in parallel also plan out drafts for different rejection points with different bonus tokens anywhere in the tree.

Almost every team that I’ve been talking to that is training a LLM right now talks about how they’re training a Chinchilla optimal model, which is remarkable given that basically everything in the LLM space changes every week.

I hope that either that's a miscommunication, or I'm wrong about how much of a red flag that seems to be.

The Chinchilla scaling laws allow you to relate, at a somewhat-better-than-rule-of-thumb level, the model size, training data size, and achieved performance of a LLM, without actually training one. So, if for instance you have a certain loss target, and a certain sized corpus of training data, you can use the scaling law to calculate what size of a model to train to hit the target. I can see that being useful to any team.

Chinchilla-optimality on the other hand means finding, for a set loss target, the combination of model size and training data size that minimizes training compute (which, roughly speaking, scales with just the product of those two numbers). But only training compute: Inference compute only scales with model size, regardless of training data. So Chinchilla-optimality is useful only if you expect training to take up most of your compute, i.e. if you are not expecting to actually use the model that much. I'm not in the field myself so I don't know how to quantify "that much", but it's definitely enough to keep those concepts distinct.

Call me an optimist, but I think prompt injection just isn't as fundamental a problem as it seems.

Having a single, flat text input sequence with everything in-band isn't fundamental to transformer: The architecture readily admits messing around with different inputs (with, if you like, explicit input features to make it simple for the model to pick up which ones you want to be special), position encodings, attention masks, etc.. The hard part is training the model to do what you want, and it's LLM training where the assumption of a flat text sequence comes from.

The optimistic view is, steerability turns out not to be too difficult: You give the model a separate system prompt, marked somehow so that it's easy for the model to pick up that it's separate from the user prompt; and it turns out that the model takes well to your steerability training, i.e. following the instructions in the system prompt above the user prompt. Then users simply won't be able to confuse the model with delimiter injection: OpenAI just isn't limited to in-band signaling.

The pessimistic view is, the way that the model generalizes its steerability training will have lots of holes in it, and we'll be stuck with all sorts of crazy adversarial inputs that can confuse the model into following instructions in the user prompt above the system prompt. Hopefully those attacks will at least be more exciting than just messing with delimiters.

(And I guess the depressing view is, people will build systems on top of ChatGPT with no access to the system prompt in the first place, and we will in fact be stuck with the problem)

"They" as in OpenAI, when they trained the tokenizer, just dumped a big set of text data into a BPE (byte pair encoding) tokenizer training script, and it saw that string in the data so many times that it ended up making a token for it.

"They" as in the rest of us afterward... probably just looked at the token list. It's a little over fifty thousand items, mostly short words and fragments of words, and can be fun to explore.

The GPT-2 and GPT-3 models proper were trained on different data than the tokenizer they use, one of the major differences being that some strings (like " SolidGoldMagikarp") showed up very rarely in the data that the model saw. As a result, the models can respond to the tokens for those strings a bit strangely, which is why they're called "glitch tokens". From what I've seen, the base models tend to just act as if the glitch token wasn't there, but instruction-tuned models can act in weirdly deranged ways upon seeing them.

The lesson to learn overall AIUI is just that you should train your tokenizer and model on the same data. But (also AIUI - we don't know what OpenAI actually did) you can also simply just remove the glitch tokens from your tokenizer, and it'll just encode the string into a few more tokens afterward. The model won't ever have seen that specific sequence, but it'll at least be familiar with all the tokens in it, and unlike never-before-seen single tokens, it's quite used to dealing with never-before-seen sentences.

Models like ChatGPT aren’t eligible for the Hutter Prize for a variety of reasons, one of which is that they don’t reconstruct the original text precisely—i.e., they don’t perform lossless compression.

Small nit: The lossiness is not a problem at all. Entropy coding turns an imperfect, lossy predictor into a lossless data compressor, and the better the predictor, the better the compression ratio. All Hutter Prize contestants anywhere near the top use it. The connection at a mathematical level is direct and straightforward enough that "bits per byte" is a common number used in benchmarking language models, despite the fact that they are generally not intended to be used for data compression.

The practical reason why a ChatGPT-based system won't be competing for the Hutter Prize is simply that it's a contest about compressing a 1GB file, and GPT-3's weights are both proprietary and take up hundreds of times more space than that.

I'm glad that you enjoyed my previous responses, but I want to clarify that I was not pretending to be a Linux terminal.

People who like to pooh-pooh generative AI systems as unable to be "truly creative" or to have "genuine understanding" tend to misunderstand them, which is a shame, because their actual fundamental limitations are far more interesting.

One is that behavior cloning is miscalibrated(https://www.lesswrong.com/posts/BgoKdAzogxmgkuuAt/behavior-c...): GPT-3 can be thought of as having been taught to act like a human by predicting human-written text, but it's incapable of recognizing that it has different knowledge and capabilities than a human when trying to act like one. Or, for that matter, it can roleplay a Linux terminal, but it's again incapable of recognizing for instance that when you run `ls`, an actual Linux system uses a source of knowledge that the model doesn't have access to, that being the filesystem.

Self-knowledge is where it gets particularly bad: Most text about systems or people describing themselves is very confident, because it's from sources that do have self-knowledge and clear understanding of their own capabilities. So, ChatGPT will describe itself with that same level of apparent knowledge, while in fact making up absolute BS, because it doesn't have self-knowledge when describing itself in language, in exactly the same sense as it doesn't have a filesystem when describing the output of `ls`.

Stories like this might as well be titled "Local change in Earth's magnetic field causes objects to float in the air". They shouldn't be read and then disputed because their details don't match the facts, they should be laughed out of the room based on the title alone. The forces they suppose to be relevant just aren't anywhere even close to the brute economic logic that actually governs how many people are willing to work in a field.

If companies want to employ more roofers, they should pay roofers more. It turns out that they will find more roofers that way. If they can't afford that... then we as a society didn't actually want that much roofing done in the first place. Or teaching, or farming, or programming, or whatever kind of job this story happened to be about again.

With the defaults of per_replica_batch=1, seq=2048 and gen_len=512, a completion takes about 20 seconds.

I'm not sure yet what settings I'll end up with if I decide to play with this more. per_replica_batch=3, seq=1024, gen_len=64 would give an experience roughly similar to the AI Dungeon that I'm used to, though less clever than the Dragon model, and a bit slower at about 10 seconds per batch.

I'm running it comfortably on my 3090, although it's a really snug fit for the VRAM, and that's with a number of fixes to significantly reduce its memory use from https://github.com/AeroScripts/mesh-transformer-jax .

I used to think that this was only ever going to be possible with some sort of incomprehensible type trickery that I would never be able to understand. Then TypeScript came along and showed me that no, actually, in a structural type system, it's just about as simple as you could imagine:

    type Foo = { kind: 'A', aItem: number } | { kind: 'B', bItem: string } | { kind: 'C' };
    
    type SubsetFoo = Foo & { kind: 'B' | 'C' }
    type SupersetFoo = Foo | { kind: 'D', dItem: boolean };

Hello, my name is Ari and I'm a type addict.

Thankfully, the impulse to waste time trying to come up with the most precise typing possible for everything hasn't been as strong with TypeScript as it used to be with Haskell for me. In part that's because TS's type system is so ridiculously expressive that I can say what I want to say without spending too long on it anyway, in part it's because the system's proud unsoundness and the ability for typings to simply be wrong means that I know not to stake my life on the types anyway. Besides, in my experience, the more precise I try to make the types at an interface, the more I need to cast in the internals. Better to find a balance that keeps both reasonable.

The Internet often makes me feel old, but seeing Photopia called a "newish" game certainly put some spryness in my joints again. It's not that it's not a great game - it is - but even in its medium, I do think it should be called a classic by now. A lot of things have changed since it was released.

Over the last several years of very occasionally playing interactive fiction, I've been particularly impressed by:

- Cactus Blue Motel, by Astrid Dalmady. 2016. A coming-of-age story with a bit of magical realism, written in Twine. Highly accessible, and it takes just minutes to give the game a try: http://astriddalmady.com/cactusblue.html

- Chlorophyll, by Steph Cherrywell. 2015. Also a coming-of-age story, but mostly a rip-roaring scifi adventure. Could well make a good introduction to the more modern views on interactive fiction.

- Coloratura, by Lynnea Glasser. 2013. Carpenterian horror from an unusual perspective. Swept the awards in the IF community when it came out.

- Eat Me, by Chandler Groover. 2017. A twisted fairytale that's thoroughly obsessed with food - the richer and the more varied, the better. A great showing of how much a writer who's willing to go far enough with it can do with prose style.

Aw. Well, it was worth a shot, that kind of mild homonym abuse is kind of direction that these sorts of titles usually tend to take. Fortunately the real story is more interesting.

My guess is it's either about a picture of Donald Knuth, or Knuth's reward checks, being put in picture frames.

"Nitpickers will complain that this isn’t the same as pickle.load(), since you have to pass a Class<T> token to choose what type of thing you want ahead of time. However, nothing is stopping you from passing Serializable.class and branching on the type later, after the object has been loaded."

Is that actually true in Java? It seems to me that the way that you'd implement that load() method is by using generics to inspect the class you were passed, figuring out what data it wants in what slot, and pulling that data in from the input. You could hold on to the input and return a dynamic proxy, and you would be able to see when someone calls a getFoo() on that proxy, but then you wouldn't know what the type of the foo it expects is. And I don't know whether you could even make your proxy assignable to T.

Could be it just came across the right pair of eyes. It's quite reminiscent of the old EvilIx problem in Haskell, where you can break memory safety without doing anything explicitly marked unsafe: https://mail.haskell.org/pipermail/haskell-cafe/2006-Decembe...

This turned out to be quite an angry post, so a quick preface: All of this is a knee-jerk response to a single sentence in your post. So, you know, please take with a grain of salt, not personally, etc..

I have to admit, it's a little bit depressing to see someone just use a straight-out lump of labor argument against immigration.

I can deal with people who believe immigrants tend to be criminal and who are concerned for their family's safety - they're wrong, but they're afraid, and fear's got a way of overriding statistical reason. I can deal with people who believe immigrants will bring poor institutions from their countries of origin - I think their evidence is weak, but I realize there's a lot at stake in keeping the global North rich and well-run, so I can see at least some reason for caution. I can even deal with the racists by letting my eyes glaze over for a while. People who think that brain drain hurts the country of origin too much... believe in a greater amount of responsibility that individuals have for their nations of birth than I do, but okay, at least they're coming from what they believe is a position of compassion. And so forth - I can sympathize with, understand, or at least ignore most arguments against more open immigration, even if I disagree with them.

But just "stop importing poor foreigners who undercut American wages"? What do you think makes your work valuable in the first place? I mean, I assume that like with most people in a modern economy, very little of what you consume comes from what you yourself produced. Your work is valuable because other people have demand for its results, and will exchange some of their own surplus for it. The more surplus there is to go around, the more there is for you to get. You might have to specialize more to compete in a bigger market, but you're surely capable of it. Even the poor generally are. If nothing else, in case of immigration to the USA, the natives tend to have the distinct advantage of being literate and generally conversable in English.

So please at least have the rational selfishness to ask for more surplus! Yes, the one guy who moves in to your tiny geographic area and works in the minuscule sliver of what you produce in the economy will make your life more complicated, because now you'll have to compete with them. But for that guy, there'll be thousands of others whose greater productivity you will now get to enjoy. Maybe if you were the Immigration Czar, you could be very selfish and decide that that person doesn't get to move in but everyone else does. But if you want to make a policy that lets all selfish people benefit, then you have to let everyone face more competition and get more surplus.

To in fact not undercut my own point: I'm not claiming that that's the whole story. You might believe that maybe there will in fact be less surplus overall in the long run if you open up immigration, or you might believe that the fruits of greater productivity will go to people who don't deserve them. Again, those are concerns that I can in fact see reason in. Not that many people actually believe that they live in a world where the only difference between labor in a rich country and labor in a poor country is that the former are more productive because of where they live. In more advanced territory, I might even have to deploy moral arguments, like how unconscionable it is to not let people - individuals of just as much moral worth and agency as you or I, or indeed us behind the veil of ignorance - work in your country because they were born on the wrong side of a border. But I won't do it for lump-of-labor. It doesn't show enough understanding of economics to be worth a moral argument.

I don't know. Maybe if you've been reading bad science fiction with immortal geniuses running around everywhere, it might seem that living to 120 and having an IQ of 130 is leaving a lot on the table, but when you compare those numbers to what we actually get, I think calling them "modest" is missing the point. Those aren't numbers that say "I'm more or less content with my lot", those are numbers that say you wish you were as smart as the smartest person you ever met, and that the papers wrote stories about your birthdays. They're an argument against the idea that transhumanism isn't something that people actually want.

And of course, the implication is obvious: If everyone did on average live to 120 and get an IQ of 130 on our tests, then everyone would be wanting to live 200 years, and quite a few people would probably want an IQ of 160 on the old tests. That is, unless that was the point where people just switched to transhumanism as simplified humanism: Having the choice to live longer and to understand the universe better are probably always a good thing, regardless of how long you've lived and how much you understand.

Just to make sure I'm not completely confused here: "Information can't be lost" and "you can't arrive to the same state of the universe through two different paths" are two ways to state exactly the same thing, right? Regardless of the the details of the rest of your physics (though you do need various notions to build up that far - time, with at least a past and a present, the ability to call states the "same" or not, etc.)

For instance, in cellular automata, the way that you would state that same concept is "the update function is injective". In our universe's physics, AIUI injectivity is involved somewhere in the definition of unitarity.

I'll admit to not being a physicist and to being easily confused by the subject, so I'll just use a silly little metaphor to check whether I'm thinking straight, and try to avoid terminology I don't fully understand (which, since I don't have a particularly strong understanding of even special relativity, means not using relativity terms at all):

Think of the universe as a rubber sheet over a table, being stretched in all directions. My understanding is that cosmic background radiation is on the rubber, stretching and moving with it just like all the matter is. If background radiation were on the table instead, there would be exactly one spot on the rubber that saw no motion relative to it, while spots very far away from that spot would see a huge amount of motion. If you went out far enough, you'd be seeing hard radiation from half the sky.

Or, as another view, though I find this one harder to think about, since here the expansion of the universe seems to confuse rather than clarify things: The background radiation you see at any given point is simply the photons radiated inward from a sphere of a very large radius centered on that point. Every point has its own such sphere, and if an observer sees a dipole component in its background radiation, it means that that observer is in motion relative to the average motion of the matter that those photons originally came out of.

Only locally, though. The next galaxy cluster over has a uniquely privileged inertial reference frame, too, but it's in motion (and accelerating!) relative to ours.

AIT applies to:

- Votes with at least three options. A simple majority vote between two options does in fact satisfy all of Arrow's fairness criteria (if only vacuously).

- Ranked voting systems, i.e. ones where voters only give a rank ordering of their options, not any more information.

Quadratic voting is neither of those: It applies only to binary decisions, but for those decisions, it tries to be more fair than Arrow's criteria call for, by getting out not only information about which option voters prefer, but how strongly they prefer it. Whether it could be extended to more complex votes than binary ones, I don't know.

Just for the sake of having another solution for the same problem, here's the basically-zero-thought-required brute-force search solution in C: https://gist.github.com/arirahikkala/ef1a07dbd5a8ca3a077c03c...

Not as fast as dietrichepp's solution, but not terrible either, and probably the winner in terms of memory use. Since there's no state to keep track of, adding parallelism with OpenMP was trivial, too.

In particular: The unit of GDP is in fact dollars/year (even if people rarely bother reporting the precise unit), whereas market cap is just dollars. They're as incomparable as velocity and position.

You did misunderstand. Read the next sentence:

The purpose of this frame is to allow an endpoint to gracefully stop accepting new streams (perhaps for a reboot or maintenance), while still finishing processing of previously established streams.

Fewer words, create a more powerful message.

Fewer commas, create a sentence that makes more sense.

It's Motion Sickness Simulator 2012! No, seriously, I've never had motion sickness from FPS games but this one made me feel pretty uncomfortable. Those who do get motion sickness should consider themselves warned.

That aside... it's not the first game/toy about illustrating relativistic effects that I've seen before, for instance there's http://lightspeed.sourceforge.net/ and a funny little Flash game that I've tried: http://www.testtubegames.com/velocityraptor.html . Neither of them were very much fun.

This game was not very much fun either, but there was some promise, because at the end once I'd gathered all the orbs by moving slowly and methodically, I enjoyed skating around the level and trying to go as fast as possible without bumping into things. Hopefully someone will use the engine (they say they're releasing it next year) to do something really good.

There's a terrible horrible no good really bad solution to this problem that I was surprised to discover one day when pointlessly experimenting with monad transformers:

Don't write type declarations.

Well, there's another part to it, too: Always open up data constructors using the record syntax. Record punning makes that bearable to use and usually briefer than typing out every argument of a big constructor. Combine that with using record updates rather than constructors wherever possible and appropriate, and you can minimise the unrelated knock-on effects of data constructor changes pretty well.

But! The point I was digressing from was, avoiding type declarations can save you so much trouble when you're making changes that affect 90% of the types and 5% of the code. That includes adding or removing parameters from type constructors or type classes, renaming types, and turning single types into typeclasses and vice versa. If you never said Array in a type declaration, and you suddenly realise you need more generic array types, you just import Data.Array.IArray rather than Data.Array and you're done. If you never said IntMap in a type declaration, and you suddenly realise you need more generic int-keyed finite maps, you can just import Data.EnumMap rather than Data.IntMap and you're done. If you never said your Foo type constructor takes two arguments in a type declaration, and you suddenly realise that it should, you change the type declaration and you're done.

Sure, when you're writing a function, you start with a type declaration, and let the arrows guide your path until you're done. And when you know you've got the meaning of something precisely right, you go ahead and fix it with a type so the compiler will know to look for errors elsewhere. But when you're not sure everything you're saying is the right thing, try to say as little as possible. That way you won't need to eat your words later.