Claude Sonnet 4 now supports 1M tokens of context

https://www.anthropic.com/news/1m-context

by adocomplete • 11 months ago

1324 692 11 months ago

holy moly! awesome

tankenmate2• 11 months ago

This is definitely good to have this as an option but at the same time having more context reduces the quality of the output because it's easier for the LLM to get "distracted". So, I wonder what will happen to the quality of code produced by tools like Claude Code if users don't properly understand the trade off being made (if they leave it in auto mode of coding right up to the auto compact).

jasonthorsness• 11 months ago

What do you recommend doing instead? I've been using Claude Code a lot but am still pretty novice at the best practices around this.

TheDong• 11 months ago

Have the AI produce a plan that spans multiple files (like "01 create frontend.md", "02 create backend.md", "03 test frontend and backend running together.md"), and then create a fresh context for each step if it looks like re-using the same context is leading it to confusion.

Also, commit frequently, and if the AI constantly goes down the wrong path ("I can't create X so I'll stub it out with Y, we'll fix it later"), you can update the original plan with wording to tell it not to take that path ("Do not ever stub out X, we must make X work"), and then start a fresh session with an older and simpler version of the code and see if that fresh context ends up down a better path.

You can also run multiple attempts in parallel if you use tooling that supports that (containers + git worktrees is one way)

agotterer• 11 months ago

I use the main Claude code thread (I don’t know what to call it) for planning and then explicitly tell Claude to delegate certain standalone tasks out to subagents. The subagents don’t consume the main threads context window. Even just delegating testing, debugging, and building will save a ton context.

sixothree4• 11 months ago

/clear often is really the first tool for management. Do this when you finish a task.

tehlike• 11 months ago

Some reference:

https://simonwillison.net/2025/Jun/29/how-to-fix-your-contex...

https://www.dbreunig.com/2025/06/22/how-contexts-fail-and-ho...

cubefox• 11 months ago

It would be interesting how this manifests in SSM/Mamba models. The way they handle their context window is different from Transformers, as the former don't use the Attention mechanism. Mamba is better at context window scaling than transformers but worse at explicit recall. Though that doesn't tell us how susceptible they are to context distractions.

bachittle• 11 months ago

As of now it's not integrated into Claude Code. "We’re also exploring how to bring long context to other Claude products". I'm sure they already know about this issue and are trying to think of solutions before letting users incur more costs on their monthly plans.

PickledJesus2• 11 months ago

Seems to be for me, I came to look at HN because I saw it was the default in CC

dbreunig• 11 months ago

The team at Chroma is currently looking into this and should have some figures.

falcor842• 11 months ago

Strange that they don't mention whether that's enabled or configurable in Claude Code.

farslan• 11 months ago

Yeah same, I'm curious about this. I would guess it's by default enabled with Claude Code.

csunoser• 11 months ago

They don't say it outright. But I think it is not in Claude Code yet.

We’re also exploring how to bring long context to other Claude products. - Anthropic

That is, any other product that is not Anthropic API tier 4 or Amazon bedrock.

CharlesW2• 11 months ago

From a co-marketing POV, it's considered best practice to not discuss home-grown offerings in the same or similar category as products from the partners you're featuring.

It's likely they'll announce this week, albeit possibly just within the "what's new" notes that you see when Claude Code is updated.

reasonableklout• 11 months ago

They just sent an email that the feature is in beta in CC.

faangguyindia2• 11 months ago

In my testing the gap between claude and gemini pro 2.5 is close. My company is in asia pacific and we can't get access to claude via vertex for some stupid reason.

but i tested it via other providers, the gap used to be huge but now not.

Tostino• 11 months ago

For me the gap is pretty large (in Gemini Pro 2.5's favor).

For reference, the code I am working on is a Spring Boot / (Vaadin) Hilla multi-module project with helm charts for deployment and a separate Python based module for ancillary tasks that were appropriate for it.

I've not been able to get any good use out of Sonnet in months now, whereas Gemini Pro 2.5 has (still) been able to grok the project well enough to help out.

jona777than• 11 months ago

I initially found Gemini Pro 2.5 to work well for coding. Over time, I found Claude to be more consistently productive. Gemini Pro 2.5 became my go-to for use cases benefitting from larger context windows. Claude seemed to be the safer daily driver (if I needed to get something done.)

All that being said, Gemini has been consistently dependable when I had asks that involved large amounts of code and data. Claude and the OpenAI models struggled with some tasks that Gemini responsively satisfied seemingly without "breaking a sweat."

Lately, it's been GPT-5 for brainstorming/planning, Claude for hammering out some code, Gemini when there is huge data/code requirements. I'm curious if the widened Sonnet 4 context window will change things.

llm_nerd• 11 months ago

Opus 4.1 is a much better model for coding than Sonnet. The latter is good for general queries / investigations or to draw up some heuristics.

I have paid subscriptions to both Gemini Pro and Claude. Hugely worthwhile expense professionally.

faangguyindia2• 11 months ago

when gemini 2.5 pro gets stuck, i often use deep seek r1 in architect mode and qwen3 in coder mode in aider and it solves all the problem

last month i ran into some wicked dependency bug and only chatgpt could solve it which i am guessing is the case because it has hot data from github?

On the other hand, i really need a tool like aider where i can use various models in "architect" and "coder" mode.

what i've found is better reasoning models tend to be bad at writing actual code, and models like qwen3 coder seems better.

deep seek r1 will not write reliable code but it will reason well and map out the path forward.

i wouldn't be surprised if sonnets success was doing EXACTLY this behind the scenes.

but now i am looking for pure models who do not use this black magic hack behind API.

I want more control at tool end where i can alter the prompts and achieve results i want

this is one reason i do not use claude code etc....

aider is 80% of what i want wish it had more of what i want though.

i just don't know why no one has build a perfect solution to this yet.

Here are things i am missing in aider

1. Automatic model switching, use different models for asking questions about the code, different one for planning a feature, different one for writing actual code.

2. Self determine, if a feature needs a "reasoning" model or coding model will suffice.

3. be able to do more, selectively send context and drop the files we don't need. Intelligently add files to context which will be touched by the feature, not after having done all code planning asking to add files, then doing it all over again with more context available.

penguin2022• 11 months ago

Claude doesn't have a mid-life crisis and try to `rm -rf /` or delete your project.

film422• 11 months ago

Agree but pricing wise, Gemini 2.5 pro wins. Gemini input tokens are half the cost of Claude 4. Output is $5/million cheaper than Claude. But, document processing is significantly cheaper. A 5MB PDF (customer invoice) with Gemini is like 5k tokens vs 56k with Claude.

The only downside with Gemini (and it's a big one) is availability. We get rate limited by their dynamic QoS all the time even if we haven't reached our quota. Our GCP sales rep keeps recommending "provisioned throughput," but it's both expensive, and doesn't fit our workload type. Plus, the VertexAI SDK is kind of a PITA compared to Anthropic.

Alex-Programs• 11 months ago

Google products are such a pain to work with from an API perspective that I actively avoid them where possible.

artursapek4• 11 months ago

Eagerly waiting for them to do this with Opus

irthomasthomas2• 11 months ago

Imagine paying $20 a prompt?

artursapek4• 11 months ago

If I can give it a detailed spec, walk away and do something else for 20 minutes, and come back to work that would have taken me 2 hours, then that's a steal.

datadrivenangel2• 11 months ago

Depending on how many prompts per hour you're looking at, that's probably same order of magnitude as expensive SAAS. A fancy CRM seat can be ~$2000 per month (or more), which assuming 50 hours per week x 4 weeks per month is $10 per hour ($2000/200 hours). A lot of money, but if it makes your sales people more productive, it's a good investment. Assuming that you're paying your sales people say 240K per year, ($20,000 per month), then the SAAS cost is 10% of their salary.

This explains DataDog pricing. Maybe it will give a future look at AI pricing.

mettamage2• 11 months ago

Shame it's only the API. Would've loved to see it via the web interface on claude.ai itself.

minimaxir• 11 months ago

Can you even fit 200+k tokens worth of context in the web interface? IMO Claude's API workbench is the worst of the three major providers.

mettamage2• 11 months ago

Via text files right? Just drag and drop.

data-ottawa• 11 months ago

When working on artifacts after a few change requests it definitely can.

77pt77• 11 months ago

Even if you can't, a conversation can easily get larger than that.

fblp• 11 months ago

I assume this will mean that long chats continue to get the "prompt is too long" error?

penguin2022• 11 months ago

But will it remember any of it, and stop creating new redundant files when it can't find or understand what its looking for?

1xer• 11 months ago

moaaaaarrrr

aliljet2• 11 months ago

This is definitely one of my CORE problem as I use these tools for "professional software engineering." I really desperately need LLMs to maintain extremely effective context and it's not actually that interesting to see a new model that's marginally better than the next one (for my day-to-day).

However. Price is king. Allowing me to flood the context window with my code base is great, but given that the price has substantially increased, it makes sense to better manage the context window into the current situation. The value I'm getting here flooding their context window is great for them, but short of evals that look into how effective Sonnet stays on track, it's not clear if the value actually exists here.

rootnod34• 11 months ago

Flooding the context also means increasing the likelihood of the LLM confusing itself. Mainly because of the longer context. It derails along the way without a reset.

aliljet2• 11 months ago

How do you know that?

Wowfunhappy3• 11 months ago

I keep reading this, but with Claude Code in particular, I consistently find it gets smarter the longer my conversations go on, peaking right at the point where it auto-compacts and everything goes to crap.

This isn't always true--some conversations go poorly and it's better to reset and start over--but it usually is.

benterix2• 11 months ago

it's not clear if the value actually exists here.

Having spent a couple of weeks on Claude Code recently, I arrived to the conclusion that the net value for me from agentic AI is actually negative.

I will give it another run in 6-8 months though.

wahnfrieden8• 11 months ago

Did you try with using Opus exclusively?

mark_l_watson• 11 months ago

I am sort of with you. I am down to asking Gemini Pro a couple of questions a day, use ChatGPT just a few times a week, and about once a week use gemini-cli (either a short free session, or a longer session where I provide my API key.)

That said I spend (waste?) an absurdly large amount of time each week experimenting with local models (sometimes practical applications, sometimes ‘research’).

mikepurvis• 11 months ago

For a bit more nuance, I think I would my overall net is about break even. But I don't take that as "it's not worth it at all, abandon ship" but rather that I need to hone my instinct of what is and is not a good task for AI involvement, and what that involvement should look like.

Throwing together a GHA workflow? Sure, make a ticket, assign it to copilot, check in later to give a little feedback and we're golden. Half a day of labour turned into fifteen minutes.

But there are a lot of tasks that are far too nuanced where trying to take that approach just results in frustration and wasted time. There it's better to rely on editor completion or maybe the chat interface, like "hey I want to do X and Y, what approach makes sense for this?" and treat it like a rubber duck session with a junior colleague.

cambaceres2• 11 months ago

For me it’s meant a huge increase in productivity, at least 3X.

Since so many claim the opposite, I’m curious to what you do more specifically? I guess different roles/technologies benefit more from agents than others.

I build full stack web applications in node/.net/react, more importantly (I think) is that I work on a small startup and manage 3 applications myself.

revskill2• 11 months ago

Truth. To some extend, the agent doesn't know what it's doing at all, it lacks real brain, maybe we should just treat them as the hard worker.

flowerthoughts3• 11 months ago

What type of work do you do? And how do you measure value?

Last week I was using Claude Code for web development. This week, I used it to write ESP32 firmware and a Linux kernel driver. Sure, it made mistakes, but the net was still very positive in terms of efficiency.

greenie_beans• 11 months ago

same. agents are good with easy stuff and debugging but extremely bad with complexity. has no clue about chesterson's fence, and it's hard to parse the results especially when it creates massive diffs. creates a ton of abandoned/cargo code. lots of misdirection with OOP.

chatting witch claude and copy/pasting code between my IDE and claude is still the most effective for more complex stuff, at least for me.

jmartrican• 11 months ago

Maybe that is a skills issue.

ericmcer2• 11 months ago

Agreed, daily Cursor user.

Just got out of a 15m huddle with someone trying to understand what they were doing in a PR before they admitted Claude generated everything and it worked but they weren't sure why... Ended up ripping about 200 LoC out because what Claude "fixed" wasn't even broken.

So never let it generate code, but the autocomplete is absolutely killer. If you understand how to code in 2+ languages you can make assumptions about how to do things in many others and let the AI autofill the syntax in. I have been able to swap to languages I have almost no experience in and work fairly well because memorizing syntax is irrelevant.

meowtimemania• 11 months ago

For me it depends on the task. For some tasks (maybe things that don't have good existing examples in my codebase?)

I'll spend 3x the time repeatedly asking claude to do something for me

9cb14c1ec0• 11 months ago

The more I use Claude Code, the more aware I become of its limitations. On the whole, it's a useful tool, but the bigger the codebase the less useful. I've noticed a big difference on its performance on projects with 20k lines of code versus 100k. (Yes, I know. A 100k line project is still very small in the big picture)

sorhaindop2• 11 months ago

This exact phrase has been said by 3 different users... weird.

alexchamberlain• 11 months ago

I'm not sure how, and maybe some of the coding agents are doing this, but we need to teach the AI to use abstractions, rather than the whole code base for context. We as humans don't hold the whole codebase in our hear, and we shouldn't expect the AI to either.

F7F7F73• 11 months ago

There are a billion and one repos that claim to help do this. Let us know when you find one.

siwatanejo• 11 months ago

I do think AIs are already using abstractions, otherwise you would be submitting all the source code of your dependencies into the context.

anthonypasq• 11 months ago

the fact we cant keep the repo in our working memory is a flaw of our brains. i cant see how you could possibly make the argument that if you were somehow able to keep the entire codebase in your head that it would be a disadvantage.

sdesol16• 11 months ago

LLMs (current implementation) are probabilistic so it really needs the actual code to predict the most likely next tokens. Now loading the whole code base can be a problem in itself, since other files may negatively affect the next token.

throwaway314155• 11 months ago

/compact in Claude Code is effectively this.

LinXitoW5• 11 months ago

They already do, or at least Claude Code does. It will search for a method name, then only load a chunk of that file to get the method signature, for example.

It will use the general information you give it to make educated guesses of where things are. If it knows the code is Vue based and it has to do something with "users", it might seach for "src/*/User.vue.

This is also the reason why the quality of your code makes such a large difference. The more consistent the naming of files and classes, the better the AI is at finding them.

sdesol16• 11 months ago

I really desperately need LLMs to maintain extremely effective context

I actually built this. I'm still not ready to say "use the tool yet" but you can learn more about it at https://github.com/gitsense/chat.

The demo link is not up yet as I need to finalize an admin tool but you should be able to follow the npm instructions to play around with.

The basic idea is, you should be able to load your entire repo or repos and use the context builder to help you refine it. Or you can can create custom analyzers that you can do 'AI Assisted' searches with like execute `!ask find all frontend code that does [this]` and the because the analyzer knows how to extract the correct metadata to support that query, you'll be able to easily build the context using it.

kvirani• 11 months ago

Wait that's not how Cursor etc work? (I made assumptions)

hirako20004• 11 months ago

Not clear how it gets around what is, ultimately, a context limit.

I've been fiddling with some process too, would be good if you shared the how. The readme looks like yet another full fledged app.

handfuloflight3• 11 months ago

Doesn't Claude Code do all of this automatically?

msikora• 11 months ago

Why not build this as an MCP so that people can plug it into their favorite platform?

seanmmward• 11 months ago

The primary use case isn't just about shoving more code in context, although depending on the task, there is an irredicible minimum context needed for it to capture all the needed understanding. The 1M context model is a unique beast in terms of how you need to feed it, and its real power is being able to tackle long horizon tasks which require iterative exploration, in context learning, and resynthesis. Ie, some problems are breadth (go fix an api change in 100 files), other however require depth (go learn from trying 15 different ways to solve this problem). 1M Sonnet is unique in its capabilities for the latter in particular.

hinkley• 11 months ago

Sounds to me like your problem has shifted from how much the AI tool costs per hour to how much it costs per token because resetting a model happens often enough that the price doesn't amortize out per hour. That giant spike every ?? months overshadows the average cost per day.

I wonder if this will become more universal, and if we won't see a 'tick-tock' pattern like Intel used, where they tweak the existing architecture one or more times between major design work. The 'tick' is about keeping you competitive and the 'tock' is about keeping you relevant.

TZubiri2• 11 months ago

"However. Price is king. Allowing me to flood the context window with my code base is great"

I don't vibe code, but in general having to know all of the codebase to be able to do something is a smell, it's spagghetti, it's lack of encapsulation.

When I program I cannot think about the whole database, I have a couple of files open tops and I think about the code in those files.

This issue of having to understand the whole codebase, complaining about abstractions, microservices, and OOP, and wanting everything to be in a "simple" monorepo, or a monolith; is something that I see juniors do, almost exclusively.

ants_everywhere3• 11 months ago

I really desperately need LLMs to maintain extremely effective context

The context is in the repo. An LLM will never have the context you need to solve all problems. Large enough repos don't fit on a single machine.

There's a tradeoff just like in humans where getting a specific task done requires removing distractions. A context window that contains everything makes focus harder.

For a long time context windows were too small, and they probably still are. But they have to get better at understanding the repo by asking the right questions.

stuartjohnson12• 11 months ago

An LLM will never have the context you need to solve all problems.

How often do you need more than 10 million tokens to answer your query?

onion2k2• 11 months ago

Large enough repos don't fit on a single machine.

I don't believe any human can understand a problem if they need to fit the entire problem blem domain in their head, and the scope of a domain that doesn't fit on a computer. You have to break it down into a manageable amount of information to tackle it in chunks.

If a person can do that, so can an LLM prompted to do that by a person.

sdesol16• 11 months ago

But they have to get better at understanding the repo by asking the right questions.

How I am tackling this problem is making it dead simple for users to create analyzers that are designed to enriched text data. You can read more about how it would be used in a search at https://github.com/gitsense/chat/blob/main/packages/chat/wid...

The basic idea is, users would construct analyzers with the help of LLMs to extract the proper metadata that can be semantically searched. So when the user does an AI Assisted search with my tool, I would load all the analyzers (description and schema) into the system prompt and the LLM can determine which analyzers can be used to answer the question.

A very simplistic analyzer would be to make it easy to identify backend and frontend code so you can just use the command `!ask find all frontend files` and the LLM will construct a deterministic search that knows to match for frontend files.

mock-possum• 11 months ago

The context is in the repo

Agreed but that’s a bit different from “the context is the repo”

It’s been my experience that usually just picking a couple files out to add to the context is enough - Claude seems capable of following imports and finding what it needs, in most cases.

I’m sure it depends on the task, and the structure of the codebase.

manmal• 11 months ago

The context is in the repo

No it’s in the problem at hand. I need to load all related files, documentation, and style guides into the context. This works really well for smaller modules, but currently falls apart after a certain size.

alvis2• 11 months ago

Everything in context hurts focus. It's like some people suffering from hyperthymesia. They are easily get distracted when the recall something

injidup• 11 months ago

All the more reason for good software engineering. Folders of files managing one concept. Files tightly focussed on sub problems of that concept. Keep your code so that you can solve problems in self contained context windows at the right level of abstraction

NuclearPM• 11 months ago

Problems

jack_pp• 11 months ago

maybe we need LLMs trained on ASTs or create a new symbolic way to represent software that's faster to grok by LLMs and have a translator so we can verify the code

energy1232• 11 months ago

You could probably build a decent agentic harness that achieves something similar.

Show the LLM a tree and/or call-graph representation of your codebase (e.g. `cargo diagram` and `cargo-depgraph`), which is token efficient.

And give the LLM a tool call to see the contents of the desired subtree. More precise than querying a RAG chunk or a whole file.

You could also have another optional tool call which routes the text content of the subtree through a smaller LLM that summarizes it into a maximum density snippet, which the LLM can use for a token efficient understanding of that subtree during early the planning phase.

But I'd agree that an LLM built natively around AST is a pretty cool idea.

fgbarben2• 11 months ago

Allow me to flood the fertile plains of its consciousness with my seed... yes, yes, let it take root... this is important to me

fgbarben2• 11 months ago

Let me despoil the rich geography of your context window with my corrupted b2b SaaS workflows and code... absorb the pollution, rework it, struggling against the weight... yes, this pleases me, it is essential for the propagation of my germline

dberge2• 11 months ago

the price has substantially increased

I’m assuming the credits required per use won’t increase in Cursor.

Hopefully this puts pressure on them to lower credits required for gpt-5.

khalic• 11 months ago

This is a major issue with LLMs altogether, it probably has to do with the transformer architecture. We need another breakthrough in the field for this to become reality.

HarHarVeryFunny8• 11 months ago

Even 1 MB context is only roughly 20K LOC so pretty limiting, especially if you're also trying to fit API documents or any other lengthy material into the context.

Anthropic also recently said that they think that longer/compressed context can serve as an alternative (not sure what was the exact wording/characterization they used) to continual/incremental learning, so context space is also going to be competing with model interaction history if you want to avoid groundhog day and continually having to tell/correct the model the same things over and over.

It seems we're now firmly in the productization phase of LLM development, as opposed to seeing much fundamental improvement (other than math olympiad etc "benchmark" results, released to give the impression of progress). Yannic Kilcher is right, "AGI is not coming", at least not in the form of an enhanced LLM. Demis Hassabis' very recent estimate was for 50% chance of AGI by 2030 (i.e. still 15 years out).

While we're waiting for AGI, it seems a better approach to needing everything in context would be to lean more heavily on tool use, perhaps more similar to how a human works - we don't memorize the entire code base (at least not in terms of complete line-by-line detail, even though we may have a pretty clear overview of a 10K LOC codebase while we're in the middle of development) but rather rely on tools like grep and ctags to locate relevant parts of source code on an as-needed basis.

aorobin• 11 months ago

"Demis Hassabis' very recent estimate was for 50% chance of AGI by 2030 (i.e. still 15 years out)."

2030 is only 5 years out

km144• 11 months ago

As you alluded to at the end of your post—I'm not really convinced 20k LOC is very limiting. How many lines of code can you fit in your working mental model of a program? Certainly less than 20k concrete lines of text at any given time.

In your working mental model, you have broad understandings of the broader domain. You have broad understandings of the architecture. You summarize broad sections of the program into simpler ideas. module_a does x, module_b does y, insane file c does z, and so on. Then there is the part of the software you're actively working on, where you need more concrete context.

So as you move towards the central task, the context becomes more specific. But the vague outer context is still crucial to the task at hand. Now, you can certainly find ways to summarize this mental model in an input to an LLM, especially with increasing context windows. But we probably need to understand how we would better present these sorts of things to achieve performance similar to a human brain, because the mechanism is very different.

brookst2• 11 months ago

1M tokens ~= 3.5M characters ~= 58k LOC at an average of 60 chars/line. 88k LOC at 40 chars/line

HarHarVeryFunny8• 11 months ago

Just as a self follow-up, another motivation to lean on tool use rather than massive context (cf. short-term memory) is to keep LLM/AI written/modified code understandable to humans ...

At least part of the reason that humans use hierarchical decomposition and divide-and-conquor is presumably because of our own limited short term memory, since hierarchical organization (modules, classes, methods, etc) allows us to work on a problem at different levels of abstraction while only needing to hold that level of the hierarchy in memory.

Imagine what code might look like if written by something with no context limit - just a flat hierarchy of functions, perhaps, at least until it perhaps eventually learned, or was told, the other reasons for hierarchical and modular design/decomposition to assist in debugging and future enhancement, etc!

scotty79• 11 months ago

Maybe use a cheaper model to compose a relevant context for the more expensive one?

Even better, use expensive model to create a general set of guidelines for picking the right context for your project, that the cheaper model will use in the future to pick the right context.

rootnod34• 11 months ago

So, more tokens means better but at the same time more tokens means it distracts itself too much along the way. So at the same time it is an improvement but also potentially detrimental. How are those things beneficial in any capacity? What was said last week? Embrace AI or leave?

All I see so far is: don't embrace and stay.

rootnod34• 11 months ago

So, I see this got downvoted. Instead of just downvoting, I would prefer to have a counter-argument. Honestly. I am on the skeptic side of LLM, but would not mind being turned to the other side with some solid arguments.

pupppet2• 11 months ago

How does anyone send these models that much context without it tripping over itself? I can't get anywhere near that much before it starts losing track of instruction.

9wzYQbTYsAIc• 11 months ago

I’ve been having decent luck telling it to keep track of itself in a .plan file, not foolproof, of course, but it has some ability to “preserve context” between contexts.

Right now I’m experimenting with using separate .plan files for tracking key instructions across domains like architecture and feature decisions.

CharlesW2• 11 months ago

I’ve been having decent luck telling it to keep track of itself in a .plan file, not foolproof, of course, but it has some ability to “preserve context” between contexts.

This is the way. Not only have I had good luck with both a TASKS.md and TASKS-COMPLETE.md (for history), but I have an .llm/arch full of AI-assisted, for-LLM .md files (auth.md, data-access.md, etc.) that document architecture decisions made along the way. They're invaluable for effectively and efficiently crossing context chasms.

collinvandyck76• 11 months ago

Yeah, this. Each project I work on has it's own markdown file named for the ticket or the project. Committed on the branch, and I have claude rewrite it with the "current understanding" periodically. After compacting, I have it re-read the MD file and we get started again. Quite nice.

olddustytrail2• 11 months ago

I think it's key to not give it contradictory instructions, which is an easy mistake to make if you forget where you started.

As an example, I know of an instance where the LLM claimed it had tried a test on its laptop. This obviously isn't true so the user argued with it. But they'd originally told it that it was a Senior Software Engineer so playing that role, saying you tested locally is fine.

As soon as you start arguing with those minor points you break the context; now it's both a Software Engineer and an LLM. Of course you get confused responses if you do that.

pupppet2• 11 months ago

The problem I often have is I may have instruction like-

General instruction: - Do "ABC"

If condition == whatever: - Do "XYZ" instead

I have a hard time making the AI obey the instances I wish to override my own instruction and without having full control of the input context, I can't just modify my 'General Instruction' on a case by case basis to simply avoid having to contradict myself.

greenfish6• 11 months ago

Yes, but if you look in the rate limit notes, the rate limit is 500k tokens / minite for tier 4, which we are on. Given how stingy anthropic has been with rate limit increases, this is for very few people right now

alvis2• 11 months ago

Context window after certain size doesn’t bring in much benefit but higher bill. If it still keeps forgetting instructions it would be just much easier to be ended up with long messages with higher context consumption and hence the bill

I’d rather having an option to limit the context size

EcommerceFlow2• 11 months ago

It does if you're working with bigger codebases. I've found copy/pasting my entire codebase + adding a <task> works significantly better than cursor.

spiderice• 11 months ago

How does one even copy their entire codebase? Are you saying you attach all the files? Or you use some script to copy all the text to your clipboard? Or something else?

andrewstuart• 11 months ago

Oh man finally. This has been such a HUGE advantage for Gemini.

Could we please have zip files too? ChatGPT and Gemini both unpack zip files via the chat window.

Now how about a button to download all files?

qsort• 11 months ago

I won't complain about a strict upgrade, but that's a pricy boi. Interesting to see differential pricing based on size of input, which is understandable given the O(n^2) nature of attention.

isoprophlex• 11 months ago

1M of input... at $6/1M input tokens. Better hope it can one-shot your answer.

elitan• 11 months ago

have you ever hired humans?

bicepjai• 11 months ago

Depends on which human you tried :) Donot underestimate yourself !

rafaelero• 11 months ago

god they keep raising prices

revskill2• 11 months ago

The critical issue with LLM which never beats human: break what worked.

henriquegodoy• 11 months ago

Thats incredible to see how ai models are improving, i'm really happy with this news. (imo it's more impactful than the release of gpt5) now, we need more tokens per second, and then the self-improvement of the model will accelerate.

lherron• 11 months ago

Wow, I thought they would feel some pricing pressure from GPT5 API costs, but they are doubling down on their API being more expensive than everyone else.

sebzim4500• 11 months ago

I think it's the right approach, the cost of running these things as coding assistants is negligable compared to the benefit of even a slight model improvement.

AtNightWeCode• 11 months ago

GPT5 API uses more tokens for answers of the same quality as previous versions. Fell into that trap myself. I use both Claude and OpenAI right now. Will probably drop OpenAI since they are obviously not to be trusted considering the way they do changes.

shamano• 11 months ago

1M tokens is impressive, but the real gains will come from how we curate context—compact summaries, per-repo indexes, and phase resets. Bigger windows help; guardrails keep models focused and costs predictable.

jbellis2• 11 months ago

Just completed a new benchmark that sheds some light on whether Anthropic's premium is worth it.

(Short answer: not unless your top priority is speed.)

https://brokk.ai/power-rankings

24xpossible• 11 months ago

Why no Grok 4?

Zorbanator• 11 months ago

You should be able to guess.

jeffhuys2• 11 months ago

People hate it because it had less filters and media caught on, so they told people to hate it.

It’s actually the best one right now, or close to. For my uses (code and queries) nothing comes even close.

Once people look past the “but ELoN mUssKkkk!!!”, they’ll be surprised.

jbellis2• 11 months ago

the accompanying blog post explains: xAI did not respond to our requests for a grok 4 quota that would allow us to run the evaluation

rcanepa• 11 months ago

I recently switched to the $200 CC subscription and I think I will stay with it for a while. I briefly tested whatever version of ChatGPT 5 comes with the free Cursor plan and it was unbearably slow. I could not really code with it as I was constantly getting distracted while waiting for a response. So, speed matters a lot for some people.

Someone1234• 11 months ago

Before this they supposedly had a longer context window than ChatGPT, but I have workloads that abuse the heck out of context windows (100-120K tokens). ChatGPT genuinely seems to have a 32K context window, in the sense that is legitimately remembers/can utilize everything within that window.

Claude previously had "200K" context windows, but during testing it wouldn't even hit a full 32K before hitting a wall/it forgetting earlier parts of the context. They also have extremely short prompt limits relative to the other services around, making it hard to utilize their supposedly larger context windows (which is suspicious).

I guess my point is that with Anthropic specifically, I don't trust their claims because that has been my personal experience. It would be nice if this "1M" context window now allows you to actually use 200K though, but it remains to be seen if it can even do that. As I said with Anthropic you need to verify everything they claim.

Etheryte• 11 months ago

Strong agree, Claude is very quick to forget things like "don't do this", "never do this" or things it tried that were wrong. It will happily keep looping even in very short conversations, completely defeating the purpose of using it. It's easy to game the numbers, but it falls apart in the real world.

joquarky• 11 months ago

I've found it better to use antonyms than negations most situations.

wahnfrieden8• 11 months ago

ChatGPT Pro has a longer window but I’ve read conflicting reports on what it actually uses

lvl155• 11 months ago

Only time this is useful is to do init on a sizable code base or dump a “big” csv.

film422• 11 months ago

The 1M token context was Gemini's headlining feature. Now, the only thing I'd like Claude to work on is tokens counted towards document processing. Gemini will often bill 1/10th the tokens Anthropic does for the same document.

varyherb2• 11 months ago

I believe this can be configured in Claude Code via the following environment variable:

ANTHROPIC_BETAS="context-1m-2025-08-07" claude

falcor842• 11 months ago

Have you tested it? I see that this env var isn't specified in their docs

https://docs.anthropic.com/en/docs/claude-code/settings#envi...

bazhand• 11 months ago

    Add these settings to your `.claude/settings.json`:

     ```json
     {
       "env": {
         "ANTHROPIC_CUSTOM_HEADERS": {"anthropic-beta":
     "context-1m-2025-08-07"},
         "ANTHROPIC_MODEL": "claude-sonnet-4-20250514",
         "CLAUDE_CODE_MAX_OUTPUT_TOKENS": 8192
       }
     }
     ```

varyherb2• 11 months ago

Yup! Claude Code has a lot of undocumented configuration. Once I saw the beta header value in their docs [1], I tried to see in their source code if there was anyway to specify this flag via env var config. Their source code is already on your computer, just gotta dig through the minified JS :) Try:

`cat $(which claude) | grep ANTHROPIC_BETAS`

Sibling comment's approach with the other (documented) env var works too.

[1] https://docs.anthropic.com/en/docs/build-with-claude/context...

anonym292• 11 months ago

Tested this morning. Worked wonderfully, except ran into output issues. Attempted to patched the minified Claude file's CLAUDE_CODE_MAX_OUTPUT_TOKENS hard limit of 32000 on Sonnet to 64000, which worked, and I was able to generate outputs above 32000 tokens, but this coincided with a breakage of the 1m context window for me. Still testing and playing around with this, but this may be getting patched?

gdudeman7• 11 months ago

A tip for those who both use Claude Code and are worried about token use (which you should be if you're stuffing 400k tokens into context even if you're on 20x Max):

  1. Build context for the work you're doing. Put lots of your codebase into the context window.
  2. Do work, but at each logical stopping point hit double escape to rewind to the context-filled checkpoint. You do not spend those tokens to rewind to that point.
  3. Tell Claude your developer finished XYZ, have it read it into context and give high level and low level feedback (Claude will find more problems with your developer's work than with yours).

If you want to have multiple chats running, use /resume and pull up the same thread. Hit double escape to the point where Claude has rich context, but has not started down a specific rabbit hole.

rvnx• 11 months ago

Thank you for the tips, do you know how to rollback latest changes ? Trying very hard to do it, but seems like Git is the only way ?

gdudeman7• 11 months ago

Git or my favorite "Undo all of those changes."

SparkyMcUnicorn• 11 months ago

I haven't used it, but saw this the other day: https://github.com/RonitSachdev/ccundo

rtuin• 11 months ago

Quick tip when working with Claude Code and Git: When you're happy with an intermediate result, stage the changes by running `git add` (no commit). That makes it possible to always go back to the staged changes when Claude messes up. You can then just discard the unstaged changes and don't have to roll back to the latest commit.

seperman• 11 months ago

Very interesting. Why does Claude find more problems if we mention the code is written by another developer?

bgilly• 11 months ago

In my experience, Claude will criticize others more than it will criticize itself. Seems similar to how LLMs in general tend to say yes to things or call anything a good idea by default.

I find it to be an entertaining reflection of the cultural nuances embedded into training data and reinforcement learning processes.

mcintyre19942• 11 months ago

Total guess, but maybe it breaks it out of the sycophancy that most models seem to exhibit?

I wonder if they’d also be better at things like telling you an idea is dumb if you tell it it’s from someone else and you’re just assessing it.

gdudeman7• 11 months ago

Claude is very agreeable and is an eager helper.

It gives you the benefit of the doubt if you're coding.

It also gives you the benefit of the doubt if you're looking for feedback on your developers work. If you give it a hint of distrust "my developer says they completed this, can you check and make sure, give them feedback....?" Claude will look out for you.

daveydave• 11 months ago

I would guess the training data (conversational as opposed to coding specific solutions) is weighted towards people finding errors in others work, more than people discussing errors in their own. If you knew there was an error in your thinking, you probably wouldn't think that way.

sixothree4• 11 months ago

I've been using Serena MCP to keep my context smaller. It seems to be working because claude uses it pretty much exclusively to search the codebase.

lucasfdacunha• 11 months ago

Could you elaborate a bit on how that works? Does it need any changes in how you use Claude?

yahoozoo• 11 months ago

I thought double escape just clears the text box?

gdudeman7• 11 months ago

With an empty text box, double escape shows you a list of previous inputs from you. You can go back and fork at any one of those.

oars• 11 months ago

I tell Claude that it wrote XYZ in another session (I wrote it) then use that context to ask questions or make changes.

ewoodrich2• 11 months ago

Hah, I do the same when I need to manually intervene to nudge the solution in the direction I want after a few failed attempts to recontruct my prompt to avoid some undesired path the LLM really wants to go down.

gdudeman7• 11 months ago

I'll note this saves a lot of wait time as well! No sitting there while a new Claude builds context from scratch.

i_have_an_idea3• 11 months ago

This sounds like the programmer equivalent of astrology.

Build context for the work you're doing. Put lots of your codebase into the context window.

If you don’t say that, what do you think happens as the agent works on your codebase.

bubblyworld• 11 months ago

You don't have to think about it, you can just go try it. It doesn't work as well (yet) for me. I'm still way better than Claude at finding an initial heading.

Astrology doesn't produce working code =P

gdudeman7• 11 months ago

You don't say that - you instruct the LLM to read files about X, Y, and Z. Putting the context in helps the agent plan better (next step) and write correct code (final step).

If you're asking the agent to do chunks of work, this will get better results than asking it to blindly go forth and do work. Anthropic's best practices guide says as much.

If you're asking the agent to create one method that accomplishes X, this isn't useful.

insane_dreamer• 11 months ago

I usually tell CC (or opencode, which I've been using recently) to look up the files and find the relevant code. So I'm not attaching a huge number of files to the context. But I don't actually know whether this saves tokens or not.

Wowfunhappy3• 11 months ago

I do this all the time and it sometimes works, but it's not a silver bullet. Sometimes Claude benefits from having the full conversation.

FajitaNachos• 11 months ago

What's the benefit to using claude code CLI directly over something like Cursor?

trenchpilgrim2• 11 months ago

Claude Code is a much more flexible tool: https://docs.anthropic.com/en/docs/claude-code/overview#why-...

qafy• 11 months ago

the benefit is you can use your preferred editor. no need to learn a completely new piece of software that doesnt match your workflow just to get access to agentic workflows. for example, my entire workflow for the last 15+ years has been tmux+vim, and i have no desire to change that.

KptMarchewa• 11 months ago

You don't have to deal with awfulness of vs code.

nojs3• 11 months ago

In my experience jumping back like this is risky unless you explicitly tell it you made changes, otherwise they will get clobbered because it will update files based on the old context.

Telling it to “re-read” xyz files before starting works though.

bamboozled2• 11 months ago

I always ask it to read the last 5 commits and anaylize and modified or staged files, works well...

cube004• 11 months ago

Tell Claude your developer finished XYZ [...] (Claude will find more problems with your developer's work than with yours).

It's crazy to think LLMs are so focused on pleasing us that we have to trick them like this to get frank and fearless feedback.

razemio• 11 months ago

I think it is something else. If you think about it, humans often write about correcting errors done by others. Refactoring code, fixing bugs and write code more efficient. I guess it triggers other paths in the model, if we write that someone else did it. It is not about pleasing but our constant desire to improve things.

ZeroCool2u• 11 months ago

It's great they've finally caught up, but unfortunate it's on their mid-tier model only and it's laughably expensive.

thimabi• 11 months ago

Oh, well, ChatGPT is being left in the dust…

When done correctly, having one million tokens of context window is amazing for all sorts of tasks: understanding large codebases, summarizing books, finding information on many documents, etc.

Existing RAG solutions fill a void up to a point, but they lack the precision that large context windows offer.

I’m excited for this release and hope to see it soon on the UI as well.

OutOfHere• 11 months ago

Fwiw, OpenAI does have a decent active API model family of GPT-4.1 with a 1M context. But yes, the context of the GPT-5 models is terrible in comparison, and it's altogether atrocious for the GPT-5-Chat model.

The biggest issue in ChatGPT right now is a very inconsistent experience, presumably due to smaller models getting used even for paid users with complex questions.

wahnfrieden8• 11 months ago

Doesn’t it matter more what context they provide via Claude Code and Codex CLI? And arent they similar anyway there?

Because the API with maximum context is very expensive (also not rolled out to everyone)

kotaKat• 11 months ago

A million tokens? Damn, I’m gonna need a lot of quarters to play this game at Chuck-E-Cheese.

xnx2• 11 months ago

1M context windows are not created equal. I doubt Claude's recall is as good as Gemini's 1M context recall. https://cloud.google.com/blog/products/ai-machine-learning/t...

xnx2• 11 months ago

Good analysis here: https://news.ycombinator.com/item?id=44878999

the model that’s best at details in long context text and code analysis is still Gemini.
Gemini Pro and Flash, by comparison, are far cheaper

firasd2• 11 months ago

A big problem with the chat apps (ChatGPT; Claude.ai) is the weird context window hijinks. Especially ChatGPT does wild stuff.. sudden truncation; summarization; reinjecting 'ghost snippets' etc

I was thinking this should be up to the user (do you want to continue this conversation with context rolling out of the window or start a new chat) but now I realized that this is inevitable given the way pricing tiers and limited computation works. Like the only way to have full context is use developer tools like Google AI Studio or use a chat app that wraps the API

With a custom chat app that wraps the API you can even inject the current timestamp into each message and just ask the LLM btw every 10 minutes just make a new row in a markdown table that summarizes every 10 min chunk

cruffle_duffle• 11 months ago

btw every 10 minutes just make a new row in a markdown table that summarizes every 10 min chunk

Why make it time based instead of "message based"... like "every 10 messages, summarize to blah-blah.md"?

dev0p• 11 months ago

Probably it's more cost effective and less error prone to just dump the message log rather than actively rethink the context window, costing resources and potentially losing information in the process. As the models gets better, this might change.

firasd2• 11 months ago

Sure. But you'd want to help out the LLM with a message count like this is message 40, this is message 41... so when it hits message 50 it's like ahh time for a new summary and call the memory_table function (cause it's executing the earlier standing order in your prompt)

tosh• 11 months ago

How did they do the 1M context window?

Same technique as Qwen? As Gemini?

deadbabe• 11 months ago

Unfortunately, larger context isn’t really the answer after a certain point. Small focused context is better, lazily throwing a bunch of tokens in as a context is going to yield bad results.

ramoz• 11 months ago

Awesome addition to a great model.

The best interface for long context reasoning has been AIStudio by Google. Exceptional experience.

I use Prompt Tower to create long context payloads.

simianwords2• 11 months ago

How does "supporting 1M tokens" really work in practice? Is it a new model? Or did they just remove some hard coded constraint?

eldenring• 11 months ago

Serving a model efficiently at 1M context is difficult and could be much more expensive/numerically tricky. I'm guessing they were working on serving it properly, since its the same "model" in scores and such.

simianwords2• 11 months ago

Thanks - still not clear what they did really. Some inference time hacks?

nickphx• 11 months ago

Yay, more room for stray cats.

alienbaby• 11 months ago

The fracturing of all the models offered across providers is annoying. The number of different models and the fact a given model will have different capabilities from different providers is ridiculous.

chrisweekly• 11 months ago

Peer of this post currently also on HN front page, comparing perf for Claude vs Gemini, w/ 1M tokens: https://news.ycombinator.com/item?id=44878999

DiabloD32• 11 months ago

Neat. I do 1M tokens context locally, and do it entirely with a single GPU and FOSS software, and have access to a wide range of models of equivalent or better quality.

Explain to me, again, how Anthropic's flawed business model works?

codazoda• 11 months ago

Tell us more?

DiabloD32• 11 months ago

Nothing really to say, its just like everyone else's inference setups.

Select a model that produces good results, has anywhere from 256k to 1M context (ex: Qwen3-Coder can do 1M), is under one of the acceptable open weights licenses, and run it in llama.cpp.

llama.cpp can split layers between active and MoE, and only load the active ones into vram, leaving the rest of it available for context.

With Qwen3-Coder-30B-A3B, I can use Unsloth's Q4_K_M, consume a mere 784MB of VRAM with the active layers, then consume 27648MB (kv cache) + 3096MB (context) with the kv cache quantized to iq4_nl. This will fit onto a single card with 32GB of VRAM, or slightly spill over on 24GB.

Since I don't personally need that much, I'm not pouring entire projects into it (I know people do this, and more data does not produce better results), I bump it down to 512k context and fit it in 16.0GB, to avoid spill over on my 24GB card. In the event I do need the context, I am always free to enable it.

I do not see a meaningful performance difference between all on the card and MoE sent to RAM while active is on VRAM, its very much a worthwhile option for home inference.

Edit: For completeness sake, 256k context with this configuration is 8.3GB total VRAM, making _very_ budget good inference absolutely possible.

ffitch• 11 months ago

I wonder how modern models fair on NovelQA and FLenQA (benchmarks that test ability to understand long context beyond needle in a haystack retrieval). The only such test on a reasoning model that I found was done on o3-mini-high (https://arxiv.org/abs/2504.21318), it suggests that reasoning noticeably improves FLenQA performance, but this test only explored context up to 3,000 tokens.

dang2• 11 months ago

Related ongoing thread:

Claude vs. Gemini: Testing on 1M Tokens of Context - https://news.ycombinator.com/item?id=44878999 - Aug 2025 (9 comments)

whalesalad2• 11 months ago

My first thought was "gg no re" can't wait to see how this changes compaction requirements in claude code.

pmxi• 11 months ago

The reason I initially got interested in Claude was because they were the first to offer a 200K token context window. That was massive in 2023. However, they didn't keep up once Gemini offered a 1M token window last year.

I'm glad to see an attempt to return to having a competitive context window.

iamnotagenius• 11 months ago

they all degrade well before 1M tokens.

markb139• 11 months ago

I’ve tried 2 AI tools recently. Neither could produce the correct code to calculate the CPU temperature on a Raspberry Pi RP2040. The code worked, looked ok and even produced reasonable looking results - until I put a finger on the chip and thus raised the temp. The calculated temperature went down. As an aside the free version of chatGPT didn’t know about anything newer than 2023 so couldn’t tell me about the RP2350

anvuong2• 11 months ago

How can you be sure putting the finger on the chip raise the temp? If you feel hot that means heat from the chip is being transferred to your finger, that may decrease the temp, no?

broshtush• 11 months ago

From my understanding putting your finger on an uncooled CPU acts like a passive cooler, thus actually decreasing temperature.

fwip• 11 months ago

I don't think a larger context window would help with that.

fpauser4• 11 months ago

Best comment ;)

ghjv• 11 months ago

wouldn't your finger have acted as a heat sink, lowering the temp? sounds like the program may have worked correctly. could be worth trying again with a hot enough piece of metal instead of your finger

logicchains• 11 months ago

With that pricing I can't imagine why anyone would use Claude Sonnet through the API when Gemini 2.5 Pro is both better and cheaper (especially at long-context understanding).

CuriouslyC• 11 months ago

Claude is a good deal with the $20 subscription giving a fair amount of sonnet use with Code. It's also got a very distinct voice as far as LLMs go, and tends to produce cleaner/clearer writing in general. I wouldn't use the API in an application but the subscription feels like a pretty good deal.

siva7• 11 months ago

Ah, so claude code on subscription will become a crippled down version

joduplessis• 11 months ago

As far as coding goes Claude seems to be the most competent right now, I like it. GPT5 is abysmal - I'm not sure if they're bugs, or what, but the new release takes a good few steps back. Gemini still a hit and miss - and Grok seems to be a poor man's Claude (where code is kind of okay, a bit buggy and somehow similar to Claude).

wahnfrieden8• 11 months ago

Are you evaluating gpt5-thinking on high mode, via API or Codex CLI on Pro tier? Just wondering what specifically you compared since those factors affect its performance and context

brokegrammer2• 11 months ago

Many people are confused about the usefulness of 1M tokens because LLMs often start to get confused after about 100k. But this is big for Claude 4 because it uses automatic RAG when the context becomes large. With optimized retrieval thanks to RAG, we'll be able to make good use of those 1M tokens.

m4r71n• 11 months ago

How does this work under the hood? Does it build an in-memory vector database of the input sources and runs queries on top of that data to supplement the context window?

brokegrammer2• 11 months ago

No idea how it's implemented because it's proprietary. Details here: https://support.anthropic.com/en/articles/11473015-retrieval...

menaerus2• 11 months ago

RAG commonly implies some sort of vector database to be built and which will then be used for response augmentation. If it operates over the repo, I believe it will index your codebase using those vector embeddings.

Balgair• 11 months ago

Wow!

As a fiction writer/noodler this is amazing. I can put not just a whole book in as before, not just a whole series, but the entire corpus of authors in.

I mean, from the pov of biography writers, this is awesome too. Just dump it all in, right?

I'll have to switch using to Sonnet 4 now for workflows and edit my RAG code to be longer windows, a lot longer

irthomasthomas2• 11 months ago

Brain: Hey, you going to sleep? Me: Yes. Brain: That 200,001st token cost you $600,000/M.

qwertox• 11 months ago

desperately need LLMs to maintain extremely effective context

Last time I used Gemini it did something very surprising: instead of providing readable code, it started to generate pseudo-minified code.

Like on CSS class would become one long line of CSS, and one JS function became one long line of JS, with most of the variable names minified, while some remained readable, but short. It did away with all unnecessary spaces.

I was asking myself what is happening here, and my only explanation was that maybe Google started training Gemini on minified code, on making Gemini understand and generate it, in order to maximize the value of every token.

ericol• 11 months ago

"...in API"

That's a VERY relevant clarification. this DOESN'T apply to web or app users.

Basically, if you want a 1M context window you have to specifically pay for it.

sporkland• 11 months ago

Does anyone have data on how much better these 1M token context models produce better results than the more limited windows alongside certain RAG implementations? Or how much better in the face of RAG the 200k vs 1M token models perform on a benchmark?

poniko• 11 months ago

[Claude usage limit reached. Your limit will reset at..] .. eh lunch is a good time to go home anyways..

chmod775• 11 months ago

For some context, only the tweaks files and scripting parts of Cyberpunk 2077 are ~2 million LOC.

not_that_d• 11 months ago

My experience with the current tools so far:

1. It helps to get me going with new languages, frameworks, utilities or full green field stuff. After that I expend a lot of time parsing the code to understand what it wrote that I kind of "trust" it because it is too tedious but "it works".

2. When working with languages or frameworks that I know, I find it makes me unproductive, the amount of time I spend writing a good enough prompt with the correct context is almost the same or more that if I write the stuff myself and to be honest the solution that it gives me works for this specific case but looks like a junior code with pitfalls that are not that obvious unless you have the experience to know it.

I used it with Typescript, Kotlin, Java and C++, for different scenarios, like websites, ESPHome components (ESP32), backend APIs, node scripts etc.

Botton line: usefull for hobby projects, scripts and to prototypes, but for enterprise level code it is not there.

jeremywho2• 11 months ago

My workflow is to use Claude desktop with the filesystem mcp server.

I give claude the full path to a couple of relevant files related to the task at hand, ie where the new code should hook into or where the current problem is.

Then I ask it to solve the task.

Claude will read the files, determine what should be done and it will edit/add relevant files. There's typically a couple of build errors I will paste back in and have it correct.

Current code patterns & style will be maintained in the new code. It's been quite impressive.

This has been with Typescript and C#.

I don't agree that what it has produced for me is hobby-grade only...

taberiand2• 11 months ago

I've been using it the same way. One approach that's worked well for me is to start a project and first ask it to analyse and make a plan with phases for what needs to be done, save that plan into the project, then get it to do each phase in sequence. Once it completes a phase, have it review the code to confirm if the phase is complete. Each phase of work and review is a new chat.

This way helps ensure it works on manageable amounts of code at a time and doesn't overload its context, but also keeps the bigger picture and goal in sight.

nwatson• 11 months ago

One can also integrate with, say, a running PyCharm with the Jetbrains IDE MCP server. Claude Desktop can then interact directly with PyCharm.

hamandcheese• 11 months ago

Any particular reason you prefer that over Claude code?

JyB• 11 months ago

That's exactly how you should do it. You can also plug in an MCP for your CI or mention cli.github.com in your prompt to also make it iterate on CI failures.

Next you use claude code instead and you make several work on their own clone on their own workspace and branches in the background; So you can still iterate yourself on some other topic on your personal clone.

Then you check out its tab from time to time and optionally checkout its branch if you'd rather do some updates yourself. It's so ingrained in my day-to-day flow now it's been super impressive.

risyachka• 11 months ago

Pretty much my experience too.

I usually go to option 2 - just write it by myself as it is same time-wise but keeps skills sharp.

fpauser4• 11 months ago

Not to degenerate is really challenging these days. There are the bubbles that simulate multiple realities to us and try to untrain us logic thinking. And there are the llms that try to convice us that self thinking is unproductive. I wonder when this digitalophily suddenly turns into digitalophobia.

flowerthoughts3• 11 months ago

I predict microservices will get a huge push forward. The question then becomes if we're good enough at saying "Claude, this is too big now, you have to split it in two services" or not.

If LLMs maintain the code, the API boundary definitions/documentation and orchestration, it might be manageable.

fsloth• 11 months ago

Why microservices? Monoliths with code-golfed minimal implementation size (but high quality architecture) implemented in strongly typed language would consume far less tokens (and thus would be cheaper to maintain).

arwhatever• 11 months ago

Won’t this cause [insert LLM] to lose context around the semantics of messages passed between microservices?

You could then put all services in 1 repo, or point LLM at X number of folders containing source for all X services, but then it doesn’t seem like you’ll have gained anything, and at the cost of added network calls and more infra management.

urbandw311er• 11 months ago

Why not just cleanly separated code in a single execution environment? No need to actually run the services in separate execution environments just for the sake of an LLM being able to parse it, that’s crazy! You can just give it the files or folders it needs for the particular services within the project.

Obviously there’s still other reasons to create micro services if you wish, but this does not need to be another reason.

fpauser4• 11 months ago

Same conclusion here. Also good for analyzing existing codebases and to generate documentation for undocumented projects.

j45• 11 months ago

It's quite good at this, I have been tying in Gemini Pro with this too.

johnisgood4• 11 months ago

but for enterprise level code it is not there

It is good for me in Go but I had to tell it what to write and how.

sdesol16• 11 months ago

I've been able to create a very advanced search engine for my chat app that is more than enterprise ready. I've spent a decade thinking about search, but in a different language. Like you, I needed to explain what I knew about writing a search engine in Java for the LLM, to write it in JavaScript using libraries I did not know and it got me 95% of the way there.

It is also incredibly important to note that the 5% that I needed to figure out was the difference between throw away code and something useful. You absolutely need domain knowledge but LLMs are more than enterprise ready in my opinion.

Here is some documentation on how my search solution is used in my app to show that it is not a hobby feature.

https://github.com/gitsense/chat/blob/main/packages/chat/wid...

jiggawatts• 11 months ago

Something I’ve discovered is that it may be worthwhile writing the prompt anyway, even for a framework you’re an expert with. Sometimes the AIs will surprise me with a novel approach, but the real value is that the prompt makes for excellent documentation of the requirements! It’s a much better starting point for doc-comments or PR blurbs than after-the-fact ramblings.

viccis2• 11 months ago

I agree. For me it's a modern version of that good ol "rails new" scaffolding with Ruby on Rails that got you started with a project structure. It makes sense because LLMs are particularly good at tasks that require little more knowledge than just a near perfect knowledge of the documentation of the tooling involved, and creating a well organized scaffold for a greenfield project falls squarely in that area.

For legacy systems, especially ones in which a lot of the things they do are because of requirements from external services (whether that's tech debt or just normal growing complexity in a large connected system), it's less useful.

And for tooling that moves fast and breaks things (looking at you, Databricks), it's basically worthless. People have already brought attention to the fact that it will only be as current as its training data was, and so if a bunch of terminology, features, and syntax have changed since then (ahem, Databricks), you would have to do some kind of prompt engineering with up to date docs for it to have any hope of succeeding.

pvorb• 11 months ago

I'm wondering what exact issue you are referring to with Databricks? I can't remember a time I had to change a line I wrote during the past 2.5 years I've been using it. Or are you talking about non-breaking changes?

alfalfasprout• 11 months ago

The bigger problem I'm seeing is engineers that become over reliant on vibe coding tools are starting to lose context on how systems are designed and work.

As a result, their productivity might go up on simple "ticket like tasks" where it's basically just simple implementation (find the file(s) to edit, modify it, test it) but when they start using it for all their tasks suddenly they don't know how anything works. Or worse, they let the LLM dictate and bad decisions are made.

These same people are also very dogmatic on the use of these tools. They refuse to just code when needed.

Don't get me wrong, this stuff has value. But I just hate seeing how it's made many engineers complacent and accelerated their ability to add to tech debt like never before.

mnky9800n2• 11 months ago

Yea that’s right. It’s kind of annoying how useful it is for hobby projects and it is suddenly useless on anything at work. Haha. I love Claude code for some stuff (like generating a notebook to analyse some data). But it really just disconnects you from the problem you are solving without you going through everything it writes. And I’m really bullish on ai coding tools haha, for example:

https://open.substack.com/pub/mnky9800n/p/coding-agents-prov...

pqs• 11 months ago

I'm not a programmer, but I need to write python and bash programs to do my work. I also have a few websites and other personal projects. Claude Code helps me implement those little projects I've been wanting to do for a very long time, but I couldn't due to the lack of coding experience and time. Now I'm doing them. Also now I can improve my emacs environment, because I can create lisp functions with ease. For me, this is the perfect tool, because now I can do those little projects I couldn't do before, making my life easier.

zingar• 11 months ago

Big +1 to customizing emacs! Used to feel so out of reach, but now I basically rolled my own cursor.

chamomeal2• 11 months ago

LLMs totally kick ass for making bash scripts

MangoCoffee2• 11 months ago

At the end of the day, all tools are made to make their users' lives easier.

I use GitHub Copilot. I recently did a vibe code hobby project for a command line tool that can display my computer's IP, hard drive, hard drive space, CPU, etc. GPT 4.1 did coding and Claude did the bug fixing.

The code it wrote worked, and I even asked it to create a PowerShell script to build the project for release

dekhn• 11 months ago

For context I'm a principal software engineer who has worked in and out of machine learning for decades (along with a bunch of tech infra, high performance scientific computing, and a bunch of hobby projects).

In the few weeks since I've started using Gemini/ChatGPT/Claude, I've

1. had it read my undergrad thesis and the paper it's based on, implementing correct pytorch code for featurization and training, along wiht some aspects of the original paper that I didn't include in my thesis. I had been waiting until retirement until taking on this task.

2. had it write a bunch of different scripts for automating tasks (typically scripting a few cloud APIs) which I then ran, cleaning up a long backlog of activities I had been putting off.

3. had it write a yahtzee game and implement a decent "pick a good move" feature . It took a few tries but then it output a fully functional PyQt5 desktop app that played the game. It beat my top score of all time in the first few plays.

4. tried to convert the yahtzee game to an android app so my son and I could play. This has continually failed on every chat agent I've tried- typically getting stuck with gradle or the android SDK. This matches my own personal experience with android.

5. had it write python and web-based g-code senders that allowed me to replace some tools I didn't like (UGS). Adding real-time vis of the toolpath and objects wasn't that hard either. Took about 10 minutes and it cleaned up a number of issues I saw with my own previous implementations (multithreading). It was stunning how quickly it can create fully capable web applications using javascript and external libraries.

6. had it implement a gcode toolpath generator for basic operations. At first I asked it to write Rust code, which turned out to be an issue (mainly because the opencascade bindings are incomplete), it generated mostly functional code but left it to me to implement the core algorithm. I asked it to switch to C++ and it spit out the correct code the first time. I spent more time getting cmake working on my system than I did writing the prompt and waiting for the code.

7. had it Write a script to extract subtitles from a movie, translate them into my language, and re-mux them back into the video. I was able to watch the movie less than an hour after having the idea- and most of that time was just customizing my prompt to get several refinements.

8. had it write a fully functional chemistry structure variational autoencoder that trains faster and more accurate than any I previously implemented.

9. various other scientific/imaging/photography related codes, like impleemnting multi-camera rectification, so I can view obscured objects head-on from two angled cameras.

With a few caveats (Android projects, Rust-based toolpath generation), I have been absolutely blown away with how effective the tools are (especially used in a agent which has terminal and file read/write capabilities). It's like having a mini-renaissance in my garage, unblocking things that would have taken me a while, or been so frustrating I'd give up.

I've also found that AI summaries in google search are often good enough that I don't click on links to pages (wikipedia, papers, tutorials etc). The more experience I get, the more limitations I see, but many of those limitations are simply due to the extraordinary level of unnecessary complexity required to do nearly anything on a modern computer (see my comments about about Android apps & gradle).

stpedgwdgfhgdd• 11 months ago

For enterprise software development CC is definitely there. 100k Go code paas platform, micro services architecture, mono repo is manageable.

The prompt needs to be good, but in plan mode it will iteratively figure it out.

You need to have automated tests. For enterprise software development that actually goes without saying.

dclowd9901• 11 months ago

It also steps right over easy optimizations. I was doing a query on some github data (tedious work) and rather than preliminarily filter down using the graphql search method, it wanted to comb through all PRs individually. This seems like something it probably should have figured out.

amelius2• 11 months ago

It is very useful for small tasks like fixing network problems, or writing regexp patterns based on a few examples.

MarcelOlsz• 11 months ago

Here's how YOU can save $200/mo!

brulard14• 11 months ago

For me it was like this for like a year (using Cline + Sonnet & Gemini) until Claude Code came out and until I learned how to keep context real clean. The key breakthrough was treating AI as an architect/implementer rather than a code generator.

Most recently I ask first CC to create a design document for what we are going to do. He has instructions to look into the relevant parts of the code and docs to reference them. I review it and few back-and-forths we have defined what we want to do. Next step is to chunk it into stages and even those to smaller steps. All this may take few hours, but after this is well defined, I clear the context. I then let him read the docs and implement one stage. This goes mostly well and if it doesn't I either try to steer him to correct it, or if it's too bad, I improve the docs and start this stage over. After stage is complete, we commit, clear context and proceed to next stage.

This way I spend maybe a day creating a feature that would take me maybe 2-3. And at the end we have a document, unit tests, storybook pages, and features that gets overlooked like accessibility, aria-things, etc.

At the very end I like another model to make a code review.

Even if this didn't make me faster now, I would consider it future-proofing myself as a software engineer as these tools are improving quickly

imiric• 11 months ago

This is a common workflow that most advanced users are familiar with.

Yet even following it to a T, and being really careful with how you manage context, the LLM will still hallucinate, generate non-working code, steer you into wrong directions and dead ends, and just waste your time in most scenarios. There's no magical workflow or workaround for avoiding this. These issues are inherent to the technology, and have been since its inception. The tools have certainly gotten more capable, and the ecosystem has matured greatly in the last couple of years, but these issues remain unsolved. The idea that people who experience them are not using the tools correctly is insulting.

I'm not saying that the current generation of this tech isn't useful. I've found it very useful for the same scenarios GP mentioned. But the above issues prevent me from relying on it for anything more sophisticated than that.

john-tells-all• 11 months ago

I've seen this referred to as Chain of Thought. I've used it with great success a few times.

https://martinfowler.com/articles/2023-chatgpt-xu-hao.html

aatd86• 11 months ago

For me it's the opposite. As long as I ask for small tasks, or error checking, it can help. But I'd rather think of the overall design myself because I tend to figure out corner cases or superlinear complexities much better. I develop better mental models than the NNs. That's somewhat of a relief.

Also the longer the conversation goes, the less effective it gets. (saturated context window?)

ramshanker2• 11 months ago

Same here. A small variation: I explicitly use website to manage what context it gets to see.

drums8787• 11 months ago

My experience is the opposite I guess. I am having a great time using claude to quickly implement little "filler features" that require a good amount of typing and pulling from/editing different sources. Nothing that requires much brainpower beyond remembering the details of some sub system, finding the right files, and typing.

Once the code is written, review, test and done. And on to more fun things.

Maybe what has made it work is that these tasks have all fit comfortably within existing code patterns.

My next step is to break down bigger & more complex changes into claude friendly bites to save me more grunt work.

unlikelytomato• 11 months ago

I wish I shared this experience. There are virtually no filter features for me to work on. When things feel like filler on my team, it's generally a sign of tech debt and we wouldn't want to have it generate all the code it would take. What are some examples of filler features for you?

On the other hand, it does cost me about 8 hours a week debugging issues created by bad autocompletes from my team. The last 6 months have gotten really bad with that. But that is a different issue.

apimade• 11 months ago

Many who say LLMs produce “enterprise-grade” code haven’t worked in mid-tier or traditional companies, where projects are held together by duct tape, requirements are outdated, and testing barely exists. In those environments, enterprise-ready code is rare even without AI.

For developers deeply familiar with a codebase they’ve worked on for years, LLMs can be a game-changer. But in most other cases, they’re best for brainstorming, creating small tests, or prototyping. When mid-level or junior developers lean heavily on them, the output may look useful.. until a third-party review reveals security flaws, performance issues, and built-in legacy debt.

That might be fine for quick fixes or internal tooling, but it’s a poor fit for enterprise.

bityard• 11 months ago

I work in the enterprise, although not as a programmer, but I get to see how the sausage is made. And describing code as "enterprise grade" would not be a compliment in my book. Very analogous to "contractor grade" when describing home furnishings.

Aeolun7• 11 months ago

Umm, Claude Code is a lot better than a lot of enterprise grade code I see. And it actually learns from mistakes with a properly crafted instruction xD

typpilol3• 11 months ago

I've found having a ton of linting tools can help the AI write much better and secure code.

My eslint config is a mess but the code it writes comes out pretty good. Although it makes a few iterations after the lint errors pop for it to rewrite it, the code it writes is way better.

therealpygon• 11 months ago

I mostly agree, with the caveat that I would say it can certainly be useful when used appropriately as an “assistant”. NOT vibe coding blindly and hoping what I end up with is useful. “Implement x specific thing” (e.g. add an edit button to component x), not “implement a whole new epic feature that includes changes to a significant number of files”. Imagine meeting a house builder and saying “I want a house”, then leaving and expecting to come back to exactly the house you dreamed of.

I get why, it’s a test of just how intuitive the model can be at planning and execution which drives innovation more than 1% differences in benchmarks ever will. I encourage that innovation in the hobby arena or when dogfooding your AI engineer. But as a replacement developer in an enterprise where an uncaught mistake could cost millions? No way. I wouldn’t even want to be the manager of the AI engineering team, when they come looking for the only real person to blame for the mistake not being caught.

For additional checks/tasks as a completely extra set of eyes, building internal tools, and for scripts? Sure. It’s incredibly useful with all sorts of non- application development tasks. I’ve not written a batch or bash script in forever…you just don’t really need to much anymore. The linear flow of most batch/bash/scripts (like you mentioned) couldn’t be a more suitable domain.

Also, with a basic prompt, it can be an incredibly useful rubber duck. For example, I’ll say something like “how do you think I should solve x problem”(with tools for the codebase and such, of course), and then over time having rejected and been adversarial to every suggestion, I end up working through the problem and have a more concrete mental design. Think “over-eager junior know-it-all that tries to be right constantly” without the person attached and you get a better idea of what kind of LLM output you can expect including following false leads to test your ideas. For me it’s less about wanting a plan from the LLM, and more about talking through the problems I think my plan could solve better, when more things are considered outside the LLMs direct knowledge or access.

“We can’t do that, changing X would break Y external process because Z. Summarize that concern into a paragraph to be added to the knowledge base. Then, what other options would you suggest?”

tonyhart7• 11 months ago

it depends on model but sonnet is more than capable for enterprise code

when you stuck at claude doing dumb shit, you didnt give the model enough context to know better the system

after following spec driven development, works with LLM in large code base make it so much easier than without it like its heaven and hell differences

but also it increase in token cost exponentially, so there's that

hoppp2• 11 months ago

I used it with Tyopescript and Go, SQL, Rust

Using it with rust is just horrible imho. Lots and lots of errors, I cant wait to stop this rust project already. But the project itself is quite complex

Go on the other hand is super productive, mainly because the language is already very simple. I can move 2x fast

Typescript is fine, I use it for react components and it will do animations Im lazy to do...

SQL and postgresql is fine, I can do it without it also, I just dont like to write stored functions cuz of the boilerplatey syntax, a little speed up saves me from carpal tunnel

epolanski8• 11 months ago

I really find your experience strikingly different than mine, I'll share you my flow:

- step A: ask AI to write a featureA-requirements.md file at the root of the project, I give it a general description for the task, then have it ask me as many questions as possible to refine user stories and requirements. It generally comes up with a dozen or more of questions, of which multiples I would've not thought about and found out much later. Time: between 5 and 40 minutes. It's very detailed.

- step B: after we refine the requirements (functional and non functional) we write together a todo plan as featureA-todo.md. I refine the plan again, this is generally shorter than the requirements and I'm generally done in less than 10 minutes.

- step C: implementation phase. Again the AI does most of the job, I correct it at each edit and point flaws. Are there cases where I would've done that faster? Maybe. I can still jump in the editor and do the changes I want. This step in general includes comprehensive tests for all the requirements and edge cases we have found in step A, both functional, integration and E2Es. This really varies but it is generally highly tied to the quality of phase A and B. It can be as little as few minutes (especially true when we indeed come up with the most effective plan) and as much as few hours.

- step D: documentation and PR description. With all of this context (in requirements and todos) at this point updating any relevant documentation and writing the PR description is a very short experiment.

In all of that: I have textual files with precise coding style guidelines, comprehensive readmes to give precise context, etc that get referenced in the context.

Bottom line: you might be doing something profoundly wrong, because in my case, all of this planning, requirements gathering, testing, documenting etc is pushing me to deliver a much higher quality engineering work.

mcintyre19942• 11 months ago

You’d probably like Kiro, it seems to be built specifically for this sort of spec-driven development.

TZubiri2• 11 months ago

Remember kids, just because you CAN doesn't mean you SHOULD

mrcwinn2• 11 months ago

This tells me they've gotten very good at caching and modeling the impact of caching.

fpauser4• 11 months ago

O observed that claude produces a lot of bloat. Wonder how such llm generated projects age.

cadamsdotcom• 11 months ago

I’m glad to see the only company chasing margins - which they get by having a great product and a meticulous brand - finding even more ways to get margin. That’s good business.

howinator• 11 months ago

I could be wrong, but I think this pricing is the first to admit that cost scales quadratically with number of tokens. It’s the first time I’ve seen nonlinear pricing from an LLM provider which implicitly mirrors the inference scaling laws I think we're all aware of.

jpau• 11 months ago

Google[1] also has a "long context" pricing structure. OpenAI may be considering offering similar since they do not offer their priority processing SLAs[2] for context >128K.

[1] https://cloud.google.com/vertex-ai/generative-ai/pricing

[2] https://openai.com/api-priority-processing/

energy1232• 11 months ago

Is this marginal pricing or if you go from 200,000 to 200,001 tokens your total costs double?

reverseblade2• 11 months ago

Does this cover subscription?

anonym292• 11 months ago

API only for now, but at the very bottom of the post: "We're also exploring how to bring long context to other Claude products."

So, not yet, but maybe someday?

_joel• 11 months ago

Fantastic, use up your quota even more quickly. :)

phyzix5761• 11 months ago

What I've found with LLMs is they're basically a better version of Google Search. If I need a quick "How do I do..." or if I need to find a quick answer to something its way more useful than Google and the fact that I can ask follow up questions is amazing. But for any serious deep work it has a long way to go.

mr_moon• 11 months ago

I feel exactly the same way. why skim and sift 15 different stackoverflow posts when an LLM can pick out exactly the info I need?

I don't need to spin up an entire feature in a few seconds. I need help understanding where something is broken; what are some opinions o best practice; or finding out what a poorly written snippet is doing.

context still v important for this though and I appreciate cranking that capacity. "read 15000 stackoverflow posts for me please"

anvuong2• 11 months ago

The action of sifting through through poop to find gold actually positively develops my critical thinking skill. I, too, went through a phase of just asking LLM for a specific concept instead of Googling it and weave through dozens of wiki pages or niche mailing list discussions. It did improve my productivity but I feel like it dulls my brain. So recently I have to tone that down and force myself to go back to the old way. Maybe too much of a good thing is bad.

Whatarethese• 11 months ago

This is my primary use of AI. Looking for a new mountain bike and using AI to list and compare parts of the bike and which is best for my use case scenario. Works pretty well so far.

throawaywpg• 11 months ago

Google always planned search to be just a stopgap

meander_water• 11 months ago

I like to spend a lot of time in "Ask" mode in Cursor. I guess the equivalent in Claude code is "plan" mode.

Where I have minimal knowledge about the framework or language, I ask a lot of questions about how the implementation would work, what the tradeoffs are etc. This is to minimize any misunderstanding between me and the tool. Then I ask it to write the implementation plan, and execute it one by one.

Cursor lets you have multiple tabs open so I'll have a Ask mode and Agent mode running in parallel.

This is a lot slower, and if it was a language/framework I'm familiar with I'm more likely to execute the plan myself.

itissid• 11 months ago

My experience with Claude code beyond building anything bigger than a webpage, a small API, a tutorial on CSS etc has been pretty bad. I think context length is a manageable problem, but not the main one. I used it to write a 50K LoC python code base with 300 unit tests and it went ok for the first few weeks and then it failed. This is after there is a CLAUDE.md file for every single module that needs it as well as detailed agents for testing, design, coding and review.

I won't going into a case by case list of its failures, The core of the issue is misaligned incentives, which I want to get into:

1. The incentives for coding agent, in general and claude, are writing LOTS of code. None of them — O — are good at the planning and verification.

2. The involvement of the human, ironically, in a haphazard way in the agent's process. And this has to do with how the problem of coding for these agents is defined. Human developers are like snow flakes when it comes to opinions on software design, there is no way to apply each's preference(except paper machet and superglue SO, Reddit threads and books) to the design of the system in any meaningful way and that makes a simple system way too complex or it makes a complex problem simplistic.

  - There is no way to evolve the plan to accept new preferences except text in CLAUDE.md file in git that you will have to read through and edit.

  - There is no way to know the near term effect of code choices now on 1 week from now. 

  - So much code is written that asking a person to review it in case you are at the envelope and pushing the limit feels morally wrong and an insane ask. How many of your Code reviews are instead replaced by 15-30 min design meetings to instead solicit feedback on design of the PR — because it so complex — and just push the PR into dev? WTF am I even doing I wonder.

  - It does not know how far to explore for better rewards and does not know it better from local rewards, Resulting in commented out tests and deleting arbitrary code, to make its plan "work".

In short code is a commodity for CEOs of Coding agent companies and CXOs of your company to use(sales force has everyone coding, but that just raises the floor and its a good thing, it does NOT lower the bar and make people 10x devs). All of them have bought into this idea that 10x is somehow producing 10x code. Your time reviewing and unmangling and mainitaining the code is not the commodity. It never ever was.

lpa222• 11 months ago

One of the most helpful usages of CC so far is when I simply ask:

"Are there any bugs in the current diff"

It analyzes the changes very thoroughly, often finds very subtle bugs that would cost hours of time/deployments down the line, and points out a bunch of things to think through for correctness.

swyx• 11 months ago

maybe want to reify that as a claude code hook!

bertil• 11 months ago

That matches my experience with non-coding tasks: it’s not very creative, but it’s a comprehensive critical reader.

neucoas• 11 months ago

I am trying this tomorrow

lpa222• 11 months ago

Let me know how it goes. It’s a game changer

KTibow• 11 months ago

I'm surprised that works even without telling it to think/think hard/think harder/ultrathink.

GuB-422• 11 months ago

I added this to my toolbox in addition to traditional linters.

My experience is that it is about 10% harmful, 80% useless and 10% helpful. Which is actually great, the 10% is worth it, but it is far from a hands off experience.

By harmful I mean something like suggesting a wrong fix to code that works, it usually happens when I am doing something unusual or counter intuitive, for example having a function "decrease_x" that (correctly) adds 1 to x. It may hint for better documentation, but you have to be careful not to go on autopilot and just do what it says.

By useless I mean something like "you didn't check for null" even though the variable can't be null or is passed to a function that handles the "null" case gracefully. In general, it tends to be overly defensive and following the recommendations would lead to bloated code.

By helpful I mean finding a real bug. Most of them minor, but for some, I am glad I did that check.

LLMs complement traditional linters well, but they don't replace them.

csomar• 11 months ago

it usually happens when I am doing something unusual or counter intuitive,

That’s usually your signal that your code needs refactoring.

elcritch3• 11 months ago

Recently I realized you can add Github Copilot as a reviewer to a PR. It's surprisingly handy and found a few annoying typos and one legit bug mostly from me forgetting to update another field.

aurareturn• 11 months ago

I do the same with Github Copilot after every change.

I work with a high stakes app and breaking changes cause a ton of customer headaches. LLMs have been excellent at catching potential little bugs.

hahn-kev• 11 months ago

We use Code Rabbit for this in our open source project.

i_have_an_idea3• 11 months ago

While this is cool, can anything be done about the speed of inference?

At least for my use, 200K context is fine, but I’d like to see a lot faster task completion. I feel like more people would be OK with the smaller context if the agent acts quickly (vs waiting 2-3 mins per prompt).

jeffhuys2• 11 months ago

There’s work being done in this field - I saw a demo using the same method as stable diffusion does, but then for text. Was extremely fast (3 pages of text in like a second). It’ll come.

wahnfrieden8• 11 months ago

Meanwhile the key is to become proficient at using worktrees to parallelize agents instead of working serially with them

i_have_an_idea3• 11 months ago

Sounds nice, in theory, but in practice I want to iterate on one, perhaps, two tasks at a time, and keep a good understanding of what the agent is doing, so that I can prevent it from going off the rails, making bad decisions and then building on them even further.

Worktrees and parallel agents do nothing to help me with that. It's just additional cognitive load.

maxnevermind• 11 months ago

Does very large context significantly increase a response time? Are there any benchmarks/leader-boards estimating different models in that regard?

hoppp2• 11 months ago

So I can upload 1M tokens per prompt but pay $3 per 1M input tokens?

Its really expensive to use.

Aeolun7• 11 months ago

Only the first time. After that it’s 0.3 per 1M input tokens (cached).

psyclobe• 11 months ago

Isn’t Opus better? Whenever I run out of Opus tokens and get kicked down to Sonnet it’s quite a shock sometimes.

But man I’m at the perfect stage in my career for these tools. I know a lot, I understand a lot, I have a lot of great ideas-but I’m getting kinda tired of hammering out code all day long. Now with Claude I am just busting ass executing in all these ideas and tests and fixes-never going back!

Aeolun7• 11 months ago

Haha, I think I recognize that. I’m just worried my actual skills will athrophy while I use Claude Code like I’m a manager on steroids.

stavros• 11 months ago

By definition the thing that atrophies is the thing you never need to use.

as367• 11 months ago

That is an unfortunate logo.

wiseowise• 11 months ago

This is something that I wish I would unremember.

tomsanbear• 11 months ago

I just want a better way to invalidate old context... It's great that I can fit more context, but the main challenge is claude getting sidetracked with 10 invalid grep calls, pytest dumping a 10k token stack trace etc.... And yes the ability to go back in time via esc+esc is great but I want claude to read the error stack learn from it and purge from its context or at least let me lobotomize ot selectively... Learning and discarding the raw output from tool calls feels like the missing piece here still.

typpilol3• 11 months ago

Vscode recently rolled out checkpoints where you can go back to a previous state of the conversation. But it's still not enough.

We honestly need to be able to see exactly what's in the context and be able to manually edit it.

aledalgrande• 11 months ago

I hope that they are going to put something in Claude Code to display if you're entering the expensive window. Sometime I just keep the conversation going. I wouldn't want that to burn my Max credits 2x faster.

terminalshort• 11 months ago

Yeah, that 1 MM tokens is a $15 (IIRC) API call. That's gonna add up quick! My favorite hypothetical AI failure scenario is that LLM agents eventually achieve human level general intelligence, but have to burn so many tokens to do it that they actually become more expensive than a human.

k92942• 11 months ago

I believe Claude Code uses cache aggressively, so this 1kk tokens will be 90% discounted or do I miss something?

socrateslee• 11 months ago

It's like double "double the dose"

forgingahead• 11 months ago

Wish it was on the web app as well!

williamtrask• 11 months ago

Claude is down.

EDIT: for the moment... it supports 0 tokens of context xD

nojs3• 11 months ago

Currently the quality seems to degrade long before the context limit is reached, as the context becomes “polluted”.

Should we expect the higher limit to also increase the practical context size proportionally?

m13rar• 11 months ago

This is amazing. shout out to anthropic for doing this. I would like to have a CLAUDE Model which is not nerfed with ethics and values to please the users and write overtly large plans to impress the user.

elcritch3• 11 months ago

I'm finding GPT5 to be more succinct and on par with Claude Code so far. They're really toned down the obsequiousness.

truelson• 11 months ago

We do know Parkinson’s Law ( https://en.m.wikipedia.org/wiki/Parkinson%27s_law ) will apply to all this, right?

simon_rider• 11 months ago

feels like we just traded "not enough context" for "too much noise." The million-token window is cool for marketing, but until retrieval and summarization get way better, it’s like dumping the entire repo on a junior dev’s desk and saying "figure it out." They’ll spend half their time paging through irrelevant crap, and the other half hallucinating connections. Bigger context is only a net win if the model can filter, prioritize, and actually reason over it

whalesalad2• 11 months ago

I can't tell you the number of times I had almost reached utopia only to hit compaction limits. Post-compaction I am usually dead in the water and the spiraling or repetition begins. Claude has a hard time compacting/remembering/flaggign a-ha moments from the session. Stuff that is important in the context of the task, but not appropriate for CLAUDE.md for instance. I have been thinking for months that if the context window was 2-3x larger, I would be unstoppable. So happy for this change, and excited to test it this week.

MagicMoonlight• 11 months ago

It’s a stupid metric because nothing in the real world has half a million words of context. So all they’re doing is feeding it imagined slop, or sticking together random files.

zaptrem• 11 months ago

It's useful for hours-long long-context debugging sessions in Claude Code, etc.

cintusshied• 11 months ago

For folks using LLMs for big coding projects, what's your go-to workflow for deciding which parts of the codebase to feed the model?

aitchnyu• 11 months ago

Aider automatically makes an outline of your whole codebase, which takes fraction of the tokens of the real files.

https://aider.chat/docs/repomap.html

mrcwinn2• 11 months ago

I wish they’d fix other things faster. Still can’t upload an Excel file in the iOS app, even with analyst mode enabled. The voice mode feels like it’s 10 years behind OpenAI (no realtime, for example). And Opus 4.1 still occasionally goes absolutely mental and provides much worse analysis than o3 or GPT5-thinking.

Rooting for Anthropic. Competition in this space is good.

I watched an interview with Dario recently where he said he wasted a “product guy” and it really shows.

cognix_dev• 11 months ago

To be honest, I am not particularly interested in whether the next model is better than the previous one. Rather than being superior, it is important that it maintains context and has enough memory capacity to not interfere with work. I believe that is what makes the new model competitive.

CodeCompost• 11 months ago

In Visual Studio as well?

omlelelom_kimox• 11 months ago

throwmeaway222• 11 months ago

Why do I get the feeling that HN devs on here want to just feed it their entire folder, source, binaries everything and have it make changes in seconds.

Roark66• 11 months ago

I noticed the quality of answer degrades horribly beyond few thousands of tokens. Maybe 10k. Is anyone actually successfully using these 100k+ token contexts for anything?

nprateem• 11 months ago

Anyone else found Claude has become hugely more stupid recently?

It used to always pitch answers at the right level, but recently it just seems to have left its common sense at the door. Gemini just gives much better answers for non-technical questions now.

amelius2• 11 months ago

1M?

640K ought to be enough for anybody ... right?

doppelgunner• 11 months ago

Great! Now we can use AI to read and think like a specific "book".

hassadyb• 11 months ago

i personally use it in my codding tasks such ana amazing and powerful llm

muzani• 11 months ago

Of course Bolt is the customer spotlight. These vibe coding tools chuck the entire codebase and charge for tokens used. By 10k lines of code or so, these apps were not able to fit.

ndkap• 11 months ago

Does anybody know which technology most of these companies that support 1M tokens use? Or is it all hidden?

t43562• 11 months ago

I think this highlights some problems with software development in general. i.e. the code isn't enough - you need to have domain knowledge too and a lot of knowledge about how and why the company needs things done in some way or another. You might imagine that dumping the contents of your wiki and all your chat channels into some sort of context might do it but that would miss the 100s of verbal conversations between people in the company. It would also fall foul of the way everything tends to work in any way you can imagine except what the wiki says.

Even if you transcribed all the voice chats and meetings and added it in, it challenges a human to work out what is going on. No-context human developers are pretty useless too.

nothercastle• 11 months ago

How much will it cost when you get near the 1m context mark? Its got to be in thousands per query

Claude Sonnet 4 now supports 1M tokens of context

Related Stories