IX-103

Doesn't that mean these 33k tokens can be cached, since they don't depend on the input? The model can just start generation on the 33k+1th token.

I wonder why Chrome's built-in AI wasn't mentioned, but the Claude Chrome plugin was. Were they not able to trick Gemini, or was it not tested?

Yeah, I'm not sure how competitive it is without any specs. Just from it being "inference only" that puts it on the same level as Google's 2015 TPUv1.

There are schemes where you don't need key pairs for each user (assuming the government has some way of authenticating users). Private State Tokens use blinded tokens for this.

It doesn't prevent tokens from being stolen or sold, but the token issuer only accepts each token once and can limit the rate that tokens are issued and control how fast they expire, giving decent control over how practical using stolen or sold tokens are.

It doesn't necessarily have to be white mold either. Anything that generates a persistent immune response can keep it going once it triggered. Like really bad allergies.

So your "evidence" is the transcript of an interview that references an indictment containing information that hasn't been publicly substantiated.

Even if they're more productive, the amount of work to be done has also increased significantly. Not only is there so much work to be done on AI integrations but all of the other products have bug trackers exploding with new privacy and security vulnerabilities that need to be addressed.

Some of the security people I work with claim that without the productivity gains from AI, there is no way they could keep up with the bugs.

Theoretically, that's true. In practice the gains were limited to the wealthy who use that money to fund increasingly deranged start-ups* until the market crashes. Then rinse and repeat. The failed start-ups effectively waste any increased productivity leaving everyone about where they were before.

*They fund these start-ups to get a good return on investment so they can get even more money. As the economy overheats the number of places to invest with a reasonable return falls so they are left with the high-risk stuff to invest in. I'm not sure what they want the money for, though, since they could already afford most of the things I would find useful...

How do you handle object disambiguation with YOLO? All the examples I've played with have the problem where if two "cars" get too close to each other then the tracking IDs keep switching between them, meaning we'd need an additional kinetic model for disambiguation.

Ick. The entire article starts from the fundamentally flawed premise that "you want a function that takes a blob of memory as an argument". Then they discuss bytewise access into structures..

Passing around void pointers is simply not a safe thing to do in C++. You can't do anything with a void pointer, so you're probably going to cast it as something else. Use that type instead, so that your caller knows they need to pass a valid pointer to that type. If the pointer has the wrong alignment then that will result undefined behavior. If you need to support multiple pointer types, use templates.

And, unless there are some really weird circumstances, you actually don't want to access your structures bytewise. Offsets can shift with compiler flags/versions. If you want serialization , please use a serialization library that correctly handles all of the odd cases. These can be quite efficient.

I've only actually had to munge bytes in a class once. Somebody decided that a previously POD class that was passed between processors with different memory spaces needed a virtual function, so I had to overwrite the vtable when I received it to make it valid.

Why not both?

Chess games are in their training set, other games are not.

But there's nothing stopping AI from developing domain expertise. If you fine tune a model based on the records of all previous work (effectively "shadowing" the existing workers) then it can easily learn this domain knowledge. The only difference is that AI companies have gone after programming domain knowledge first. Others will come later.

I don't see how it would be possible for inference costs to dominate training costs, even after amortization.

Training involves multiple passes over the entire training dataset, ideally in large batches where you can perform inference on as many samples as possible simultaneously and then perform backpropagation to adjust the model weights (which is about as expensive as inference).

Let's consider the size of the dataset we're dealing with here. The dataset likely consists of practically every piece of digitized text they can get their hands on (including that extracted from audio and video). We know Google has digitized a large portion of the books in existence as part of their "search book contents" feature and we have no reason to believe they're not using it alongside their cache of 90+% of the internet to train their models. We're talking about 100s of millions of books each with an average of 100,000s of tokens. The internet has 10s to 100s of billions of pages on it with who knows how many tokens on average. This is a huge dataset that we've got to go through hundreds of times.

Second, let's consider the effect of batching and how it sets requirements for our hardware. We know that larger batch sizes converge faster, are more stable, and produce better models. So if you want a good model you need large batch sizes. This means that you need machines several orders of magnitude more powerful than you use for inference. From what I heard Google uses clusters of 100s of the their TPUs all located in a single rack for training. These clusters are organized in a customized computing architecture to maximize memory locality between cores (really critical for efficient back-propagation). Further, you can't use reduced precision weights for training like you can for inference, so there are no shortcuts.

Finally, the initial training stage is followed by reinforcement learning stages - this is key development in how AI models have improved in the past year. This may mean going through a curated set of traces (either synthetic or captured from users) and adjusting the weights based on experienced outcome.

Overall there's so many orders of magnitude more work and more hardware requirements for training that I find it improbable that inference dominates. The number of "inference" steps in training is freaking ridiculous and includes such factors as the "number of words ever written".

You forgot the /s.

Or did you forget all the journalists kicked out of the Pentagon, the exclusion of the associated press over their use of the internationally recognized name for the Gulf of Mexico, the threatened cancellation of FCC licenses of ABC and CBS regarding their reporting, etc.

Jetski/Antigravity is a better piece of software. The Gemini CLI codebase looks like someone tried to vibe-code a Claude code clone in nodeJS, as it's simply not maintainable.

That being said, until recently Gemini CLI was better. It had support for persistent policies on what code could run without asking and had good extension hooks to allow you write extensions that influence policy (to perform complex logic like rewriting tool calls before they are executed).

Antimatter/Jetski only recently added support for remembering what commands are "always allowed" between sessions, the extension framework (excuse me, "plugins") has fewer features, and hooks have much less power than with Gemini CLI (and can't come bundled with extensions).

In my experience, its the opposite. The AI is very good at writing code, but it is unreliable at any kind of design. I use it as a fancy form of autocomplete. I give the broad strokes: "add a method here and change all but this one caller to use the new method", "Apply this design pattern here for this change but don't do this other thing". It completes the task reasonably well and sometimes even remembers to run the code formatter and check that tests pass.

If I ask it to me produce a design, I'll almost always end up with something unworkable or inefficient.

Though if you push it hard enough then it can sometimes give you a good description of what existing code does and how it does it (which can be easily verified).

I work for a company that has been using Mythos for vulnerability detection in our software. The results we're getting are revolutionary to the point that our software security teams are heavily overloaded addressing the deluge of thousands of real bugs/vulnerabilities and design flaws across our billions of lines of code.

For comparison, we are invested heavily the the AI space to the point where Anthropic is one of our competitors. We were already using state of the art models to find flaws in our code, but Mythos was just so much better at finding real vulnerabilities it's not even funny.

Yes, exactly. Moore's law says that in less than 10 years you will be able to fit today's state of the art models on your phone. If you add in all of the computationally and memory neutral improvements and breakthroughs that we will accumulate over the next 10 years then it will be both far more capable and far more reliable than today's models.

An AI assistant you can trust and bring with you is coming, and almost nothing can stop it.

Control-C is the usual for that.

They're working on it. I think they even have a "beta" for Android/Chrome on CiderV. From what I heard it's slow and doesn't work with most of the existing tooling (want to reformat your source files? Too bad).

They just announced the Googlebook (a laptop), not to be confused with Google Books (their service for selling ebooks). It sounds like the mismanagement is right at normal levels.

It's going to the administration overhead. If you have to document everything and argue for every medical procedure and deal with 20+ different processes for filing claims then it takes time. And, as a provider, you have to pay someone to spend that time if you want to get paid.

It doesn't help that our healthcare billing systems are so outdated and broken. I once had a doctor visit denied with the reason code that it should charge the other insurance (for people on multiple plans). I was only on one plan, but my wife was on two. The doctor and I went through all the paperwork - my name was right, my birthday was right, my policy number was right and when I got notice of the rejection it had my name on it. Eventually we traced it to an error - not in my insurance company, not in the company that handles claims in this areas for my insurance, but instead in some middle-man company that was responsible for transferring claims between the two. Nevermind that all three companies claimed to be BlueCross BlueShield. This took over a year to resolve.

The TI-85 also didn't have a lot of the built-in statistical functions that the TI-83 had.

I also was the one person with a TI-85 in a school of 83s. But by the time I took the statistics class I knew enough BASIC to write my own programs to replicate the functionality that was missing.

The average person is slightly more female than male and has 2.1 children, but they do benefit from defense contracts since it makes up a small percentage of their salary.

'readSync' does two different things - tells the OS we want to read some data and then waits for the data to be ready.

In a good API design, you should exposed functions that each do one thing and can easily be composed together. The 'readSync' function doesn't meet that requirement, so it's arguably not necessary - it would be better to expose two separate functions.

This was not a big issue when computers only had a single processor or if the OS relied on cooperative multi-threading to perform I/O. But these days the OS and disk can both run in parallel to your program so the requirement to block when you read is a design wart we shouldn't have to live with.

They don't want the appearance of tampering. They don't really care about tampering itself because they make their money either way.

So as long as you are subtle about it they won't even investigate, as a public investigation would make it seem like tampering was common.

I can't wait for the day that killing a human-any human-is considered a war crime.

The GDPR is vague and unworkable as written. It fundamentally restricts all data processing with a few, vague exceptions.

What is data processing essential for the services being provided? Many publishers assumed that getting paid was an essential part of providing a service, and it was not until 3 months before the implementation deadline that the committee clarified that getting paid is not included when you are being paid by a third party.

How are you to know whether or not the user is an EU citizen (and thus subject to the GDPR)? Is making that determination a service essential for providing your service? The answers apparently were "You don't" and "No", which would effectively make companies assume that the GDPR applies to everyone on the planet.

The GDPR also is fundamentally opposed to how things currently work in the internet, making almost all advertising on the web illegal overnight. It was too big of a change to happen at once, so it effectively only loosely enforced in practice.

I like the idea of the GDPR, but the implementation sucks.

How did that work out for the Palestinians?