sdesol

Claude Code has largely won individual developer mindshare and has been on top ever since it came out.

Claude Code's success is not due to the agent but because the model is considered the best for programming and is very heavily subsidized, compared to pay as you go API prices. Consumers and Enterprise are not really locked in and will go where it makes the most sense.

I think they have almost no loyalty by actual developers.

you can use /tree to go back before the compaction

This is really a killer feature in my opinion. I'm currently working on pi brains extension that is designed to solve the compaction issue I have, which is, I don't know what the AI knows after compaction.

https://gitsense.com/screenshots/pi-inspect-overview.png

The screenshot shot above contains what I call a compaction capsule which contains all the messages/events at the point of compaction. The "Files" section in the right side panel shows me how many files were read/modified since compaction.

After compacting, I can easily tell what Pi use to know and use `/tree` to go back in time.

I still wouldn't compact more than 5 times before starting a new session since compaction messages persists in Pi.

I am curious as to how this protect works. I am currently working on a pi brain extensions and one of the commands that I will support is:

/brains inspect

that will let you inspect a session and one of the features is the ability to review a compacted message like so:

https://gitsense.com/screenshots/inspect-pi-session-compacte...

https://gitsense.com/screenshots/inspect-pi-session-compacte...

https://gitsense.com/screenshots/inspect-pi-session-compacte...

With /compact in Pi, it creates a message that you can easily review and I am curious as to how '/protect' works.

One of the features that I am working on is to make it easy for agents to retrieve the exact message/event before compaction and I am curious if /protect is a deterministic process or if it is just instructions.

My extension for pi https://github.com/gitsense/pi-brains solves the too many skills problem and it can be adapted to work with any coding agent that supports hooks like Claude and Codex.

You can find a simple example at https://github.com/gitsense/gsc-rules-demos which shows how skills can be injected when needed. The example is:

"read the file at data/accounting/q1.ledger and explain what this ledger tracks"

If you know what the use needs to read or edit, you can inject knowledge/skills for the agent.

There are some additional things I need to implement first but I do have a hands on repo that you actually try at

https://github.com/gitsense/gsc-rules-demos

One of the rules shows how you can inject information when you know an agent wants to read a certain file. So if you have skills related to a certain file/directory, you can inject the information when you need to.

So using the OPs visualization tool, if you find the agent wandering you could create a new rule to guide the agent and compare it with and with guidance.

A lot of people want a use case.

I think the issue is, on the surface this is very much a "watch this non-deterministic actvity" but I can see the value in it. This is self promotional but I am working on a "Brain" (https://github.com/gitsense/pi-brains) for the Pi (https://github.com/earendil-works/pi) coding agent and I can actually see this being quite useful to quickly tell if you did make an agent smarter or not.

There is obviously different ways you can do this, but I can see being able to quickly visualize changes based on additional knowledge/constraints can be quite useful.

I may try moving it all to Pi, but I wonder if I should learn how to better configure the things there.

Honestly if the difference was marginal, I would move to Pi. I just tried layzpi and I had the agent write/compile a hello world in c and it required 20k in tokens. My minimal setup required 5.3k in tokens.

People will say the cost is minimal since this is cached but 15k is a lot since that needs to be reasoned by the LLM. I haven't looked at Claude Code but I read somewhere the system prompt is like 20k so I can see how Claude Code might have seem smarter as what you are working was probably addressed in the system prompt.

For me the bloat is not worth it since I am more interested in the LLM being able to reason better.

What was lacking? This is self promotional but I am working on

https://github.com/gitsense/pi-brains

which is designed around the Pi philosophy of less is better by focusing on ondemand context/guidance. I won't bloat the context unless the LLM needs to do something I know it will need better guidance with. I have a demo repo for this at https://github.com/gitsense/gsc-rules-demos

One of the examples is, if I know the agent is reading a specific file, I will inject additional context. So if the agent never needs to do something in a certain file or directory, I don't need to pollute the context with "what it may need to know".

I think that would be implied by "how did you arrive at the code". I think the conversation will revove around how did you "guide" things.

One of the most annoying things I've seen AI do is accidentally pull in old requirements and start building off of them.

This is actually the reason for me wanting to create what I call "Brain checkpoints". After chatting with the agent and letting it do its thing for discovery, I needed an easy way to understand what it knows at any given point. What files it has read, what tool calls were made and so forth.

Checkpoints work extremely well in Pi since you can create a branch, and have the agent generate a checkpoint based on what has changed since the last checkpoint and this information will not pollute the main conversation but you benefit from the token caching.

I think it makes no sense to share the conversation but I do think developers will be required to provide lessons, notes and checkpoints in the conversation.

What I think will happen is you will be required to run the conversation through a company required prompt that will include things like, does the conversation contain "think harder" without providing any guidance for how to correct things.

Basically, did the developer treat the conversation like a literal slot machine.

Nobody wonders out loud how a senior developer arrives at the result he does without an AI; they care that his deliverable is high quality and meets whatever standard/requirement exists.

I think the issue that many may not want to think about is, how do we promote/reward/fire developers in the AI assisted programming age. As part of code reviews, I think it will be second nature to ask "how did you arrive at the code".

Has anyone tried teaching the agents to use jj or gitbutler-cli to do their ideas of lightweight branches?

This is very trivial if your coding agents supports hooks like Pi. I have demo repo that goes over how rules work at https://github.com/gitsense/gsc-rules-demos

The basic idea is, you always ensure you inject the instructions for how to use jj or any other scm. LLMs are pattern matchers and they can fully map behaviour, but due to heavy training on git, they can forget.

With hooks, you can easy detect when 'git' is used and basically tell the agent "As stated earlier you must use jj". So it will try again using `jj` and you block the agent when ever it tries to use git.

It is a wasted turn but the reality is, it won't cost must since you can leverage caching and you only waste tokens on the invalid command and not the output generated from git.

This is what I am doing with my pi-brains extension for pi

https://github.com/gitsense/pi-brains

I will make another update by the end of this week that contains what I call "brain checkpoints" that will make it easier for developers to debug and understand AI reasoning.

The idea is after a task has been finished, you would commit lessons, notes, and "brain checkpoints" that are designed to live with the code.

now they can generate one in a few minutes and send it out for review.

I think we will very soon move to a prove to me you've read it protocol and/or introduce speed bumps to slow things down.

Would usuability (for what herdr is optimized for) not be the bigger use case for users?

I never heard of this project but in 5 minutes I got the features that I wanted without having to introduce muscle memory. I just wanted a dead simple way to point and click to select different terminals. The only thing I neeed to remember now is "ctrl-b + v" to create a new pane.

It is not about me liking it or not. The word threaten comes with implications. This means they acted in hostile manner based on your words.

got essentially threatened by leadership for not using AI.

This sounds made up or your workplace is rather odd to say the least. Maybe english isn't your first language and "threatened" is not the correct word?

I can't imagine what the point of autogenerating agent context could be

I have a fork of the OpenAI Codex repo at https://github.com/gitsense/smart-codex that shows why you may want to autogenerate agent context.

I makes navigating a 4000+ file repo extremely context efficient. It is important to note that the goal is too keep the context as clean as possible and not necessary speed.

Honestly this is where I would have fable generate a checklist and you just monitor opus to ensure it is going through the checklist. I think ignore is often the result of a context that is not focused enough.

Why aren't the teams using shared checks? Are the codes in different repos?

I even have "don't overrule me without asking" in my global AGENTS.md, and it simply doesn't do that.

You really need to look into hooks based on your coding agent. This is very much a solved problem as I demonstrate with

https://github.com/gitsense/pi-brains

I have a test repo

https://github.com/gitsense/gsc-rules-demos

that shows how you can block and warn and do other things.

You obviously can't have a "Don't make a mistake" rule though.

Better a round trip than bad/incorrect results. Also, the cache should kick in so the cost will be minimal.

they still need tool-trial-and-error dance after injecting them?

It honestly depends on the model. For my pi-brains extension for pi

https://github.com/gitsense/pi-brains

I've found after the first hook injection they get it, but there are occasions it can forget, but since everything is driven by hooks, you can inject as often as needed.

The issue with skills is, they are a one time thing, so you really can't use skills to correct haviorial issues.

The article doesn't go into how they managed the AI context when implementing things but I would not be surprised if it was done in a methodical way, 80% - 90% of the test could have passed.

This is a hard problem, but one worth solving, I think, since it means less tokens and better AI reasoning. I believe LLMs are good enough that, if given the right context, it can very much solve almost all tasks.

If this works, it means we can probably get by with smaller models (since it doesn't need to know everything). LLMs are pattern matchers, and if you can provide them with the right shape (context), they should produce the expected output.

For my solution to work, you need business buy-in, which I don't think will be a problem. Enterprise wants to know how tokens are being spent, so I can see them wanting structured analysis during code reviews.

What may also not be obvious is that the information is ultimately designed to live with your code. Lessons and notes are designed to be mapped to files, so if you want to know why a piece of code is implemented in a certain way, you can have the LLM filter by files to help find the needle in the haystack.

It is a hard problem, but the only missing piece is discipline, which I believe business leaders will not have an issue with enforcing since we are ultimately talking about eliminating/significantly reducing the bus factor in our code.

If you look at https://github.com/gitsense/smart-ripgrep, you can get a better sense of how context can be injected when it is needed.

So I want to say there's still a lot of value in context engineering though it seems to diminish with each model release

I can't see how it would diminish unless you are literally working on public domain stuff. Unless stuffing context becomes cost effective and will not affect AI reasoning (this will be much harder), I don't see why context engineering is here to stay until we have close to AGI.

The user flow I am trying to get adopted for sessions is to turn them into notes and lessons when you have finished and it should be part of the code review process.

By propery categorizing lessons and notes, it should make it easy to scrub and keep up to date.

I also suggest mapping lessons and notes to files when possible to make discovery and cleanup easier.

You don't need another layer

I do think we need another layer, but it should be a routing layer. I am finalizing my pi-brains extension for Pi (https://github.com/earendil-works/pi) which does this:

https://github.com/gitsense/pi-brains

Right now "humans" need to define the routing rules for how to access information, but I will support what I call "knowledge agents" that can monitor conversations to inject context when needed.

For AI agents, it means well-designed CLI tools that help the agent orient itself in a task and pull exactly the 'context-for-the-job' it needs right then.

This is exactly what I am trying to solve and I have what I call smart repositories that demonstrates this at

https://github.com/gitsense/smart-ripgrep

https://github.com/gitsense/smart-codex

The issue I am finding is, getting the agent to pull what it needs, even when the data is there is still challenging since LLMs are trained on blind discovery where the pattern is:

grep -> read -> grep -> read ...

What is working for me now is thanks to Pi (pi.dev). I am working on a pi-brains extension that makes it dead simple to control the lifecycle for an agent so if I detect that it uses `rg` without `gsc rg`, I can block the agent and inject a steering message that says always search with context.

I can also see if they try to "read" without first looking at the files metadata and so forth.

I'm finalizing things right now, but I think pi with my brains extension should allow domain experts to better guide agents so they can find what they need, when they need it.