sally_glance

Quick reminder that it took 6 years for Snowden to step up with dozens or even hundreds of other involved employees not talking before.

The law doesn't actually work by allowing any possible interpretation to hold equal weight the way that many programmers think it does

Is that so? Recent rulings in the US specifically gave me the impression that when backed by sufficient legal representation and goodwill on the judging side indeed any possible interpretation will suffice.

I think that's what makes law making complicated - you either err on the side of leaving too much room for interpretation or not enough.

Very true. As with all standards, there will always be people who disagree. We still mostly follow them either because we're forced to or because the effort required to establish another standard doesn't outweigh the benefits.

Personally I've always been a proponent of project specific standards, but after many years of discussions about more or less individual preference I've come to think that maybe settling on something global isn't the worst idea. Not that I think it must be this one in particular, but it's not the worst start either.

Well actually there is a reasonably objective standard defining software quality criteria on the source code level (ISO 5055). They also define 29 criteria for maintainability: https://www.it-cisq.org/coding-rules/

Homelander approves.

Well yeah humanity in the sense of everyone except maybe Uyghurs, Tibetans, Mongols, ... I mean open models are really cool, but I have a hard time believing China is doing it purely for the greater good. The commoditizing your complement story sounds much more plausible.

Well, until you established monopoly you need to build trust. Open Source is one way of doing just that. One of the better ways I would say even...

It does not use -p, but it does try to impersonate Claude when talking to the Anthropic API. Will they detect the difference in usage patterns and ban anyone who exploits them? Who knows.

Does that also merge view/vote metrics? I mean I could probably look it up in the source, but I'm lazy...

I've had my private servers running Arch (managed by Ansible) for the last 5 years but have recently been looking into Talos for the same reasons. Setting up a single node k8s using Talos was actually pretty straight forward. In the end I decided against it just because I couldn't justify the continuous load caused by k8s components when my stuff is only sporadically in use... Instead I now have them on NixOS and I really enjoy the declarative approach (although the language annoys me).

I've seen it used in production by larger orgs. The scale where you plan for around 6 months of migration, customization and integration of your legacy zoo with 7 different user account DBs. On one hand, all of these projects were successful and now run it in production. On the other, they all really needed the 6 months to whip it into shape.

Edit: Meaning I would use it if you need to get up and running quickly, but it's a solid foundation to build on long-term.

For me the largest value-add is the unified API. Being able to instantly start trialling a new model with zero code changes is well worth 5%. The other part is not having to deal with billing for multiple platforms.

1. For me pruning is a bit less about cost than performance. Recent research suggests lower context size is nearly always better, and many harnesses implement a sliding window for tool output pruning. Also not every provider supports caching, and if they do it might have expired (especially on restored sessions).

2. That's a good hint, I'm currently only trying with tighter turn and token limits for subagents and an error summary on exceeding them. Not sure how else (besides steering and prompt engineering) to ensure the subagent doesn't go wild...

I have a hunch model proficiency for a given CLI tool very much correlates with how many StackOverflow answers and blog entries providing examples for it there are...

Great job and congrats! Working on my own harness has been one of my favorite side projects in the past couple of weeks, of course I never finish anything... But I'm very interested in your experience with the following:

1. Context management - specifically pruning old tool call responses, truncation of tool output and automatic compaction. Those have worked pretty great for me, benefits of reducing context greatly seem to outweigh gains from "remembering" everything. I always leave short summaries though.

2. "Subagents" - my latest attempts revolve around not exposing any tools for the main agent at all, except for a run_agent tool where the subagent has access to the classic search/execute/fetch tools. My theory is that if subagents return concise summaries this would automatically keep the parent agent context clean for much longer. Still experimenting though, writing prompts for subagents may also be too far outside of the current training sets.

Can't speak for OP but I tried providing ast-grep in the execution context of an execute_bash tool, but even with pretty aggressive steering most models just don't seem to use it a lot. More expensive/SOTA models or higher reasoning increases the chances but lowers speed and raises cost. Maybe due to training bias for exploration tasks?

Is there a complete list of the tools somewhere? I'm interested in how you chose to expose the AST specifically. In my own harness attempts I wanted to keep the number of tools absolutely minimal and briefly experimented with including an AST lib to use via an execute_python tool (plus some examples in the system prompt). Results were mixed though, with most models preferring ripgrep.

Not sure this is a great idea. The model only internalized what it was trained on and writing prompts/context for itself isn't part of that. I try to keep my context as clean as possible, mostly today's models seem smart/aligned enough to be steered by a couple of keywords.

Helix?

Maybe a silly bet where the head of sales had 1-2 glasses of wine too much... "I bet they will still pay us 20 bucks/mo without CC! Don't believe me? I'm going to prove it!"

Maybe the brain is more akin to a network of networks and the actual reasoning part is not all that large? There are lots of areas dedicated exclusively to processing input and controlling subsystems. I can imagine a future where large artificial networks work in a similar way, with multiple smaller ones connected to each other.

How are Chatbot UIs different from search engines? Just look how that turned out... Yeah we have Kagi and DDG, but quality, completeness of results (for most topics) and cost still drives most people to Google.

Switching is maybe feasible for those who have the resources, but the majority will be stuck with large providers. They establish quasi-monopolies, then monetize (with ads). It's the sad cycle of commerce.

This is the hard part - especially with larger initiatives, it takes quite a bit of work to evaluate what the current combination of harness + LLM is good at. Running experiments yourself is cumbersome and expensive, public benchmarks are flawed. I wish providers would release at least a set of blessed example trajectories alongside new models.

As it is, we're stuck with "yeah it seems this works well for bootstrapping a Next.js UI"...

It's wild that everyone seems to have forgotten that Ticketmaster acquired TradeDesk and actively marketed to scalpers [1] just a couple of years ago. Seems they shut down the platform last year, maybe the "ticket bank" [2] idea worked better... Pretty clear to me that they will use any chance to monetize their monopoly.

[1] https://www.cbc.ca/news/business/competition-bureau-ticketma...

[2] https://www.musicbusinessworldwide.com/judge-signals-hell-le...

I think the problem is the union argument type - intuitively we read "array of strings OR numbers", but actually it means "array of strings AND numbers". Probably generics would be more appropriate here, with the type param constrained to string or number. Then tsc would also complain about pushing a number without checking the item type before.

Yeah, and just like all social media platforms adopted short form video sooner or later they are going to give in to what consumers pay for (in attention or money). Right now it's anyone's guess what that might be in the context of software development.

How do you know their definition isn't only "received extortion letters" and "exfiltrate data" is fine as long as it didn't lead to the former?

I guess we'll just have to wait and see how things turn out. Currently it seems we have examples of where it seems like the technology allows some amount of innovation (AlphaGo, software, math proofs) and examples where they seem surprisingly stupid (recipes?).

Btw, it looks like there is a growing body of research evaluating exactly this. I found this nice overview with even some benchmarks specifically for scientific innovation: https://github.com/HKUST-KnowComp/Awesome-LLM-Scientific-Dis...

In software, they can and do perform experiments (make a change then observe the log output). I don't think they possess a "world model" or that it's worth spending too much thought on... My reasoning is more along the lines that our brains are also just [very advanced] inference machines. We also hallucinate and mis-identify images (there are image/video classification tasks where humans have lower scores).

For me the most glaring difference to how humans work is the lack of online learning. If that prevents them from being able to innovate, I'm not so sure.

Experimentation leads to experience, so I feel like this was included by the parent comment. And in the case of writing software, agents are able to experiment today. They run tests, check log output, search DBs... Sure, they can't have apples fall on their heads like Newton had but they can totally observe the apple falling on someones head in a video.