agentdev001

Gotcha, I feel like model or provider-specific installs would be a nice QoL improvement in that case. Presumably, part of this issue (beyond the ethos of minimalism) is the aim of shipping shipping an agnostic toolset. For myself, im openai-first, and of course that pushes me to favor their hosted tools (in this case web search), and their native CPT/RL'd stuff (Ie apply-patch).

Though, imo, the fact that pi maintains its "we only include the bare minimum!" statement is part of the draw for me. Especially considering that im in an enterprise env; being able to internally share custom implementations of out-of-the-box Codex/CC stuff is really nice.

I do wonder how they'd go about shipping a default web search tool. Big problem there is the lethal trifecta. Shipping something that arbitrarily allows untrusted content to be retrieved non-deterministically I'm sure is a long conversation on Pi's end. Pushing it off to the user to decide is easy.

I'd like to understand what features you're referring to that are missing from base-install Pi CLI.

Obligatory yes, but only if you're subscription-based and not pay-per-token as enterprise users are.

Am I wrong to be somewhat peeved by the use of "RAG" in these contexts? I always read things like this, and wonder if instead the author should be saying "Semantic Retrieval" or something something Vector, etc. Retrieval augmented generation captures tool-use, and; semantic search of course is really just a tool under the hood.

To make an anology, in my mind, this is akin to saying "fuel air mixture system" when referring to direct fuel injection specifically, when of course, a carburetor also lives in that category.

Nice work! Excited to try $YOUR_HARNESS out!

Reading your comment reminded me; I actually did something quite similar at $MY_BETTER_STARTUP! My approach is slightly different, however, employing what I like to call State-Horizon-Aware-Rercursive-Threaded-Graph-Position-Topology.

With a 400% increase in words, $MY_WAY_BETTER_HARNESS looks to be about four times as performant. SHARTGPT isn’t just a harness engineer's playground — it's a the cyber jungle gym that frees them from $MY_HARNESS.

If you want, I can even include a sentence or two that will really tell those potential investors why they should shower you with money instead of the other commenter! Just say the word!

(Looking for lead investors for our angel syndicate btw! DM me if interested)

Sounds like user error to me. Codex gives an llm a tool to allow it to use shell in the context of the host and user in which it is running. If a resource is sensitive, and accessible in that context, then the user is doing something wrong. Would you change your practices if you treated your coding agent as an untrusted human ssh'd under the identity you use for it?

In any case. There are solutions in the comments on the issue, as well as this hn thread.

Ah, I wasn't aware things regressed there. Yea certainly workarounds n soft fork sorts of things definitely would work- but thats a bummer than things have changed.

From watching Pr's and issues- seems like openai at least wants to come across as if theyre supporting non-oai models :/

You can use the Codex harness with non-openai providers if you want.

I keep butting into the question of; why opencode, when you've got codex available? Codex is open source as well, and i can't seem to picture a situation where one would want Opencode over Codex.

As far as I can tell, they tick the same boxes- but one has the support of a big boy model provider.

The "Very good" I'm referring to is far better than only 99%. I can't offer solid stats off the top sadly, so you'll have to just take my word for it ;)

I'll take the opportunity to note that if you're running solid evals, you'll have data to back the efficacy of your system. If you are seeing a hallucination rate of 1%, then you certainly should be working on your harness/toolset/context/prompting etc.

Saying "1% hallucination rate..." is akin to saying "30,000mi lifespan for [modern japanese make engine]". Something is wrong.

I find papers/articles which discuss solutions that rely heavily on a model in the middle unreadable, if the models used are not discussed.

The data you need to get into context for a small model, vs a big boy frontier model, vs a fine tuned open weight big boy- are all very different. I can understand what they're doing here, and most of the 'why', but- not all of the why.

I have gone through this process and evaluated the results. Maybe you're referring to their comment as written, but going through what OC described + handholding leads to very good results in my experience.

Nvidia Openshell solves most of the hard problems I've run into while building stuff in this space.

Observability is, for my purposes, solved by a given framework supporting OpenTelemetry.

Guardrails is where I've gotten the most value of openshell being a neat package. Agent workload scope is written as policy in openshell, and capability is backed by openshell handling all execution.

Monitoring/deployment/versioning is helped as well, depending on how agents/runners are slotted into the system. Deployment namely is quite well supported- openshell has kube/helm bits that are experimental atm, but seem like a logical approach imho.

Evals and a/b testing isnt something ive explored in depth, considering that agents with composable tool sets + frontier models are beyond my expectations already.