earslap

It is not a vulnerability though. It is by design. Docker also modifies iptables directly and bypasses most soft firewalls on the machine - which is also by design.

How do you even get usable microsecond precision sync info from a server thousands of kilometers away? The latency is variable so the information you get can't be verified / will be stale the moment it arrives. I'm quite ignorant on the topic.

probably not very relevant in a zoom meeting but there are people hypersensitive to the eye movements of someone reading from a teleprompter as well! When I figure someone is reading from a script with their eyes moving, it really affects my focus for some reason. Here is someone trying to find the ideal setup to prevent it: https://www.youtube.com/watch?v=8LkRMtWfhn4

Sometimes I hear a 10-20hz pulsing sound (so pretty fast, chirping sound) emanating from around my brainstem / back of neck at the level of my ears and a slight feeling of some liquidy movement inside. I always thought it was connected to CSP movement but maybe it isn't. But if it isn't I can't see what else it can be as it happens when I'm completely stationary.

There are also documented effects of your camera quality, and even where your camera is placed (as you tend to look at your screen during a call so your gaze is always relative to it).

I think the company named Topaz had a photoshop plugin to remove "jpeg artifacts" - I don't know if they are using a neural model for it though.

first you need to find a way to differentiate human thinking from machine thinking. you need to basically show how human thought is not the result of statistical learning and inference but something completely different. if it is not possible, the distinction is moot, something that only appeals to emotion.

we are working on <impossible problem stumping humanity>. We have considered the following path to find a solution. Are we on the right track? Only answer Yes or No.

(1 week of GPUs whirring later)

AI: Your

(that will be $1 million, thank you)

good

It is more obvious when taken to extreme: With the current feedforward transformer architectures, there is a fixed amount of compute per token. Imagine asking a very hard question with a yes/no answer to an LLM. There are infinite number of cases where the compute available to the calculation of the next token is not enough to definitively solve that problem, even given "perfect" training.

You can increase the compute for allowing more tokens for it to use as a "scratch pad" so the total compute available will be num_tokens * ops_per_token but there still are infinite amount of problems you can ask that will not be computable within that constraint.

But, you can offload computation by asking for the description of the computation, instead of asking for the LLM to compute it. I'm no mathematician but I would not be surprised to learn that the above limit applies here as well in some sense (maybe there are solutions to problems that can't be represented in a reasonable number of symbols given our constraints - Kolmogorov Complexity and all that), but still for most practical (and beyond) purposes this is a huge improvement and should be enough for most things we care about. Just letting the system describe the computation steps to solve a problem and executing that computation separately offline (then feeding it back if necessary) is a necessary component if we want to do more useful things.

Not only that but also LLMs "think" in a latent representation that is several layers deep. Sure, the first and last layers make it look like it is doing token wrangling, but what is happening in the middle layers is mostly a mystery. First layer deals directly with the tokens because that is the data we are observing (a "shadow" of the world) and last layer also deals with tokens because we want to understand what the network is "thinking" so it is a human specific lossy decoder (we can and do remove that translator and plug the latent representations to other networks to train them in tandem). There is no reason to believe that the other layers are "thinking in language".

When I'm talking about velocity, I'm not talking about coding fast, but being able to write DRY, flexible yet easily maintainable code that can weather future requirements / refactorings easily. Personally, I'm also talking in the context of my own projects so nobody is breathing down my neck or pressuring me with time. I just want to write good code that is a joy to maintain for years to come.

Haxe is amazing, has macros etc. A force multiplier if you are a solo developer for sure. But damn, you feel kinda alone if you are using it. Not everything is an npm install away which negates your velocity gains from using a saner language.

yes but still, a local model, a lightning in a bottle that is between GPT3.5 and GPT4 (closer to 4), yours forever, for about that price is pretty good deal today. probably won't be a good deal in a couple years but for the value, it is not that unsettling. When ChatGPT first launched 2 years ago we all wondered what it would take to have something close to that locally with no strings attached, and turns out it is "a couple years and about $10k" (all due to open weights provided by some companies, training such a model still costs millions) which is neat. It will never be more expensive.

I only ever used cache to find what google thought was in the site (at the time of crawling) as these days it is common to not find that info in the updated page. For everything else, there is the Internet Archive.

IIRC the progressions are generated by a small neural network trained on Bach chorales.

I can't believe people are willingly installing rootkits of dubious origin on their computers to play games (and paying for the privilege). I know cheating is an issue, but still.

I would not trust Kazakhstan to honor the TLD registrations if this took off and made some noise. Reminds me of Libya taking ownership of all those trendy .ly domains claiming you have to obey Libyan laws and regulations to operate them. Still a fun idea taken quite far!

Ah yes, I totally agree. I was inspecting the method as a stopgap solution (especially because it does not require retraining or any other special tricks) until researchers figure out "planning" in a broader sense. It is very inefficient otherwise, but in the meantime, is just simple sampling with a couple parameters to tune from the output softmax the best we can do? is there no low hanging fruit there?

For the existing models is beam-search like methods hopeless due to combinatorial explosion? Are there no smart ways to improve it? Evaluating multiple futures will be slow but if it means that the model can give vastly better output, it might be a worthwhile trade-off in some cases. I feel like our standard way of sampling the output of the LLMs is a bit too simplistic and my hunch is that it should be possible to get a lot more out of them even if it means losing speed.

I had that in one ear for 2-3 years. By pulsatile tinnitus I assume you mean the loud "whooshing" sound that is in sync with your heart beat (hearing the blood flow basically). Doctors were pretty ignorant about the phenomenon, I gave up. It was not constant either, would go away for a few hours, start again for another few hours - repeat. One day, it stopped and never came back. Pretty mysterious.

Music is not constrained to... publishing musicians, and "real music" is not defined by them. The fact that novelists are not depending on LLMs to write their next book does not mean that they are not (or will not be) impactful. From a product point of view, you have an audience of billions of non-musicians to begin with. You surely can't pretend to have the final say on people's relationship with how they like to enjoy their music or other sound content.

Besides, even a novelty can evolve to something not imaginable today. The fact that we can go that far with text only at this early stage is a sign of things to come. When creating something, there is a place for expressing your intent with language. Maybe not all of it all the time, but we'll see how things will evolve.

Did not realize Bun had (even if rudimentary) macros - bundle time executing code support. That is pretty neat! https://bun.sh/docs/bundler/macros

history shows that it is generally very hard to not slip up here and there, especially if you are not expecting to be a huge target.

IIRC the killer "feature" that gave these previewers traction (ACDSee, IrfanView etc.) was that you could just preview a bunch of images in a folder using your arrow keys. So you'd just load one and use arrow keys to see the other images in the same folder. With the built-in options, you'd have to double click images one by one (and close their windows one by one) which was a horrible UX compared to what these provided.

Haxe is pretty established, but not well known. Perpetually the underdog. And totally awesome. It also has macros where you can transform the AST! You can do some amazing compile time stuff that very small number of languages support, and the features are pretty mature. If you want a hacker's language but don't want a lisp give Haxe a shot.

There is this mostly "sterile" art presented in the page (which is cool!), and there also is the demoscene which at least needs a mention.

The way I worded it, it might seem wrong - and I agree with you. When I said "constant" I meant without any optimizations to speed up shorter contexts, so with full designed context, architecturally, it is constant. You can pad shorter active contexts with zeroes and avoid attending to empty spaces as an optimization, but that is just an optimization, not an architectural property. If you want "more computation" you fill the context with relevant data (chain of thought, or n-shot stuff), which is the "trick" Karpathy alluded to (it provides more context to attend to), and I agree with that analysis.

The autoregressive transformer architecture has a constant cost per token, no matter how hard the task is. You can ask the most complicated reasoning question, and it takes the same amount of computation to generate the next token compared to the simplest yes / no question. This is due to architectural constraints. Letting the LLM generate "scratch" data to compute (attend to relevant information) is a way of circumventing the constant cost limitation. The harder the task, the more "scratch" you need so more relevant context is available for future tokens.