garlic_enjoyer

AI is not so elegant, to make a comparison similar to Berry, as the humble, sterile Microsoft Word.

This is much like Berry drawing the line at a typewriter instead of a computer. To call a computer "sterile, untouched, factory made" and still use a typewriter is comical. Now let me tell you why etching into stone is the best medium.

People can have their preferences, but this kind of "I'm old fashioned and here's why that's better" posts are just nostalgia bait.

For papers that make grand claims, they usually have poignant limitations. In this paper, it's clear that the conclusion is extrapolated from a small set of observable information. That's usually the recipe for poor conclusions, and they are up front about where their findings sit.

That's not to say these findings aren't valuable though. Their blog post summary is just a bit more hype inducing than the underlying paper.

The way I feel about this is that the emergent properties of LLMs seem to reflect our own human faculties. Whether that's because we built them in our image or whether this is a common mode of consciousness is definitely outside the scope of this paper.

Assuming they have been transcribed, yes. The key idea that makes LLMs special is the attention mechanism. Maintaining attention over volumes of data is boring for most humans.

Also, to be pedantic, just taking about LLMs in this context is a tad reductive. There are many deep learning models involved in archival work that aren't language models.

I encourage you to read into this post for more context on what I mean: https://news.ycombinator.com/item?id=48675179

"There's nothing to apologize to. They're weights."

  - The Meat