winternewt

Capacity for inference isn't a cost issue, it's an availability issue. There just isn't enough hardware out there.

From the article:

What is happening here is that leading AI labs are charging not only for inference but also for research in model architecture, training data collection and curation, model training cost (which can be tens or even hundreds of millions of dollars), paying their employees and recovering the marketing costs.

That's what's being subsidized.

Adding an in-product tip to recommend running /clear when re-visiting old conversations (we shipped a few iterations of this)

I feel like I'm missing something here. Why would I revisit an old conversation only to clear it?

To me it sounds like a prompt-cache miss for a big context absolutely needs to be a per-instance warning and confirmation. Or even better a live status indicating what sending a message will cost you in terms of input tokens.

Instead of just dropping all the context, the system could also run a compaction (summarizing the entire convo) before dropping it. Better to continue with a summary than to lose everything.

What version of Claude Code is this? I don't have the /cost command mentioned here.

Unless the bubble bursts and tons of failing AI companies dump used graphics cards on the market.

And if you don't want to buy a Mac? A 80 GB NVidia GPU costs $10,000K (equivalent to 30 years of ChatGPT Plus subscription) and will probably be obsolete in 5-7 years anyway. What are my options if I want a decent coding agent at a reasonable price?

No need to speculate. The article was published in 2019 and the movie is now six years old.

When I was a kid reading Marvel comics, Spiderman seemed like a bad and relatively uninteresting super hero. His problems were trivial personal issues and he rarely left NYC, fighting local crime or villains like Green Goblin who were just broken humans. Meanwhile the Fantastic Four were in regular contact with government institutions, went into space, and fought interstellar aliens. They just seemed better and more important. Not to mention the Avengers.

As an adult, while Marvel isn't my favorite thing anymore, I find Spiderman to their most interesting superhero.

The use case I always envisioned from the sidelines was that I could ditch a poorly designed, garbage-collected mess of a language (JavaScript) for something typesafe, with predictable performance, cache locality by default (as in not making everything a reference), no GC, generics, etc. But WASM won't be there until it has first-class access to the DOM, in my opinion.

You are allowed to throw from a destructor as long as there's not already an active exception unwinding the stack. In my experience this is a total non-issue for any real-world scenario. Propagating errors from the happy path matters more than situations where you're already dealing with a live exception.

For example: you can't write to a file because of an I/O error, and when throwing that exception you find that you can't close the file either. What are you going to do about that other than possibly log the issue in the destructor? Wait and try again until it can be closed?

If you really must force Java semantics into it with chains of exception causes (as if anybody handled those gracefully, ever) then you can. Get the current exception and store a reference to the new one inside the first one. But I would much rather use exceptions as little as possible.

Destructors are vastly superior to the finally keyword because they only require us to remember a single time to release resources (in the destructor) as opposed to every finally clause. For example, a file always closes itself when it goes out of scope instead of having to be explicitly closed by the person who opened the file. Syntax is also less cluttered with less indentation, especially when multiple objects are created that require nested try... finally blocks. Not to mention how branching and conditional initialization complicates things. You can often pair up constructors with destructors in the code so that it becomes very obvious when resource acquisition and release do not match up.

That's not a feature that the developer has control over. All they can do is try to develop a good tool.

The worst part about MS Office isn't the direct user experience, because I can usually choose to use other software. The worst part is that I and everybody else are subjected to the documents that Office produces. Their defaults and their UX inevitably produce stuff that is hard to read and inconsistent, unless you fight the software really hard and make sacrifices with your desired output. And there's no escape from it. Another specimen of Word's 2.5 cm margins, 200-character lines in poorly designed knockoff Helvetica will probably find its way to my mailbox before the end of the day.

What are you implying by linking to that article?

You're not explaining why the trifecta doesn't solve the problem. What attack vector remains?

I see where you're coming from. But I often find that when I have some idea or challenge that I want to solve, I get bogged down in details (like how do I build that project)... before I even know if the idea I _wanted_ to solve is feasible.

It's not that I don't care about learning how to build Rust or think that it's too big of a challenge. It's just not the thing I was excited about right now, and it's not obvious ahead of time how sidetracked it will get me. I find that having an LLM just figure it out helps me to not lose momentum.

You misunderstand my point. I'm not saying that the purpose of Hermès is to sell bags. I'm saying that the _product_ that Hermès sells is status, and the product of a restaurant is a dining experience.

Maybe not slower once it has warmed up, though for memory-bandwidth bound use cases I would still say the lack of mutable records has you fighting the language to get reasonable cache locality (and everybody will hate your code for not being good Java). The fact that everything is a pointer kills the CPU execution pipeline and cache.

But even for I/O bound applications it still feels slow because excessive memory usage means more swap thrashing (slowing down your entire OS), and startup time suffers greatly from having to fire up VM + loading classes and waiting for the JIT to warm up.

I can start a C/C++/Rust based web server in under a second. The corresponding server in Java takes 10 seconds, or minutes once I have added more features.

The article got off on the wrong foot from the start by separating the purpose from the product. To my mind the purpose is the product and always will be.

It's actionable if you have some imagination. Raise funds for a nonprofit. Start lobbying on both sides of the aisle. Enlist an advertising company to show the dystopian future if something like chat control comes into effect, poll for focus groups and target them. Find ways to undermine and expose the forces that are pushing for authoritarian legislation.

Promote a better constitution that protects people from laws like this?

I think we already ticked that box

Humans and LLMs are deterministic in the sense that if you would rewind the universe, everything would happen the same way again. But both humans and LLMs have hidden variables that make them unpredictable to an outside observer.

I thought it was "there's no position in chess that requires more than 218 moves to reach."

Every corporate meeting should start with reminding ourselves that we're all going to die. And it most likely won't be from anything happening at the office.

The amount of shady things going on in this domain (AI trained on stolen content, no proper attribution, not proper way to audit what's going out to third party servers etc.) should be a huge red flag for any professional developer.

If you already have your entire information infrastructure in Office 365 (including all email, Excel sheets with material non-public information etc) I think this point is moot. Why would MS abuse information only from Copilot and not the rest of its products when the legal agreements permit them to do neither?

You read my mind. City builder games are unrealistic in a bad way that defies intuition and prevents you from modeling a real city. This has been my huge frustration with Cities Skylines, which looks great but where ambulances take three months to reach patients.

How do they avoid lossy compression?

The trick with L4 is they treat IPC basically like syscalls. Arguments are left on CPU registers instead of serializing them to a message buffer. The only significant work performed is a change of virtual memory map. The called process continues execution within the time slice of the caller, instead of waiting for the thread scheduler or using synchronization primitives. While some of this could possibly be achieved with Linux, a lot of the optimization is ingrained into the calling convention and so would require changes to the user-mode source code.