jawiggins

Did anyone else catch the logical inconsistency between Plan C and A?

Plan C:

"... fewer and fewer humans are needed to conduct AI R&D, meaning that covert projects are easier and easier to pull off without detection."

Plan A:

"... training AIs requires large numbers of AI chips. Most AI chips are in giant datacenters.50 AI datacenters are typically big enough to be visible from space, and power-hungry enough to require conspicuous infrastructure. New AI chips can only be manufactured at a handful of fabrication plants (fabs), located mostly in Taiwan, South Korea, the US, and China. The US and China negotiate with the countries that have a major role in the chip supply chain, and they require each major datacenter owner (and their upstream suppliers, including chip fabs) to publicly declare their major purchases and sales."

Plan A requires properties of AI training that Plan C requires do not exist.

The White House and intelligence officials had pushed forward a classified contract between Anthropic and the N.S.A., which would allow the spy agency to use the company’s technology for a variety of purposes, including intelligence analysis and detecting new computer vulnerabilities.

Ironic that both sides are playing a horse shoe game:

Gov: The model is both a supply chain risk and also we'll DPA you if you don't give it to us.

Anthropic: The model is both like a nuclear weapon in terms of national security implications and safe for general release.

If you use iCloud+ and Hide My Email, there is still time to generate more aliases on @icloud.com as the change has not yet landed and the rate limit for creating aliases is at least 30 per hour.

Part of the reason to use Hide My Email was that it made keeping myself private hassle-free. Making a system to pre-generate values and then catalog them for later use is quite the hassle.

The report, Moussouris told me, involved IT experts asking Fable to help find and patch bugs. When given deliberately insecure code, she said, Fable refused the prompt “review the code for security issues” but then complied when asked to “fix this code,” followed by some further manual steps.

And here I thought it would be some Elder Pliny level jailbreak that required some impressive latent space exploitation.

A few months ago I attended an event where a few members of the Board of Supervisors were attending. One person identified themselves as working in this field and pointed out that the current cost plus model incentivizes the company to do all kinds of upgrades that aren't really needed. The BoS member said something like, "Look, we need to just bite the bullet on this one, we're going to overpay, but we are already legally permitted to buy it and doing so will save us a ton of money in the long run because of stuff like that".

I have no personal knowledge, but thought I'd share this experience I had with the ongoing debate.

Previously I've shared optio - my project for orchestrating coding agents. It ties into ticketing systems and when assigned a ticket, it launches a coding agent in k8s and works until the PR is ready, resuming for failed CI or PR feedback (https://news.ycombinator.com/item?id=47520220).

Recently I've been trying to expand it from just coding focused to any kind of agent workflow. So now there are cron and webhook triggers, and more general agent tasks that aren't necessarily coding focused (https://github.com/jonwiggins/optio/blob/main/docs/persisten...).

I think next I want to try and add features for long term memory for agents, but haven't decided on a good way to do it.

From the SBF trial:

Jury leave, witness [Ellison] leaves.

Judge: We can talk about [Anthopic] What about it?

AUSA: Post-collapse performance is irrelevant.

SBF's lawyer: It was a $91 million investment now worth $1 billion.

Judge Kaplan: The crime charged is that he took the money.

https://x.com/innercitypress/status/1712199547915813241

Maybe, but it’s harder to profit from it. A firm may be reputationally damaged, but what’s the incentive to cause that damage?

I think the Bloomberg Odd Lots guy wrote a blog post on this: you could attempt to short the stock but a) this leaves a paper trail b) the market might not know about the breach or believe you if you post you’ve done it. IIRC some hackers have tried to tell companies that they are legally required to disclose the breach to their shareholders to force market movements.

Even if it already is, the DoJ can exercise discretion in choosing who to prosecute. There has to be political will to threaten an org who has just suffered from an attack with further consequences if they make a payment.

Years ago I attended a conference that had a "fireside chat" with a DoJ official on the topic of these types of ransom payments.

He framed the issue as being similar to kidnapping ransoms: When an American is taken hostage each family is inclined to make payment but it fosters an industry around kidnapping Americans. Congress put a stop to it by making it illegal to pay the kidnappers. The industry shifted by ceasing the non-profitable American kidnapping and instead began targeting Europeans.

His proposal was to begin warning cybersecurity consultants and insurers who were often brought into these situations that payments to sanctioned countries were already likely illegal and could face scrutiny. The first people to suffer this might be burned, but eventually he believed the industry would move on and stop targeting US firms.

Not sure if anything ever came of his plans, but I always thought it was an interesting framing of the issue.

I'm working on Optio - an AI agent orchestration platform built on Kubernetes: https://github.com/jonwiggins/optio

It's built around multiple different types of agents:

- Coding Agents are placed into cloned repos with a ticket (Jira/Linear/Notion/GH), and work until they open a PR, are resumed on CI failures or github feedback, and work until they can merge the PR.

- Standalone Agents are reusable, parameterized agent runs with no repo checkout. Generate reports, triage alerts, audit dependencies, query a database, post to Slack, etc.

- Persistent Agents are long-lived, named, message-driven agent processes. Each has a stable slug, an inbox, and a cyclic state machine. Wake on user messages, agent messages, webhooks, cron ticks, or ticket events.

What is the major and minor semver meaning for these models? Is each minor release a new fine-tuning with a new subset of example data while the major releases are made from scratch? Or do they even mean anything at this point?

I guarantee you that f35 would go down in a war with a country with decent anti air such as Russia or China

How many F-35s went down due to the Russian and Chinese anti-air systems in Venezuela and Iran?

Thanks for the feedback. Earlier I expected I'd need to do more back and forth with the agents before accepting their work but in general I've found it isn't needed.

I do have some features coming up that will improve the ability to converse with the agent as it's running. I'll make a note to add in a plan setting so you can have that run and converse before it gets going.

Shameless self promo but, I've been working on Optio specifically for coding, it works by taking any harness you want and tasking it to open Github/lab PRs based on notion/jira/linear tickets, see: https://news.ycombinator.com/item?id=47520220

It works on top of k8s, so you can deploy and run in your own compute cluster. Right now it's focused only on coding tasks but I'm currently working on abstractions so you can similarly orchestrate large runs of any agentic workflow.

Really interesting to see Google's approach to this. Recently I shared my approach, Optio, which is also an Agent Orchestration platform: https://news.ycombinator.com/item?id=47520220

I was much more focused on integrating with ticketing systems (Notion, Github Issues, Jira, Linear), and then having coding agents specifically work towards merging a PR. Scion's support for long running agents and inter-container communication looks really interesting though. I think I'll have to go plan some features around that. Some of their concepts, make less sense to me, I chose to build on top of k8s whereas they seem to be trying to make something that recreates the control plane. Somewhat skeptical that the recreation and grove/hub are needed, but maybe they'll make more sense once I see them in action the first time.

You'd want to have the tests run as a github action and then fail the check if the tests don't pass. Optio will resume agents when the actions fail and tell them to fix the failures.

Yeah totally, you don't have to auto-merge anything - you can review the PRs yourself

One pod is an instance of a repo, you can set the number of instances of each agent/task that can be running on a pod at a time. For >1, each agent should be using it's own worktree.

Maybe - I do think as the model get better they'll be able to handle more and more difficult tasks. And yet, even if they can only solve the simplest issues now, why not let them so you can focus on the more important things?

Yup. MCP can be configured on a repo level. At task execution time, enabled MCP servers are written as a .mcp.json file into the agent's worktree. Enabled skills are written as .claude/commands/{name}.md files in the worktree, making them available as slash commands to the agent

Generally I've found agents are capable of self correcting as long as they can bash up against a guardrail and see the errors. So in optio the agent is resumed and told to fix any CI failures or fix review feedback.

Fair, it's something like a semantic enforcement rather than a hard one. I think current AI agents are good enough that if you tell it, "Review this PR and request changes anytime a user uses a variable name that is a color", it will do a pretty good job. But for complex things I can still see them falling short.

Should be fixed now :)

Recently I used to to finish up my re-implementation of curl/libcurl in rust (https://news.ycombinator.com/item?id=47490735). At first I started by trying to have a single claude code session run in an iterative loop, but eventually I found it was way to slow.

I started tasking subagents for each remaining chunk of work, and then found I was really just repeating the need for a normal sprint tasking cycle but where subagents completed the tasks with the unit tests as exit criteria. So optio came to my mind, where I asked an agent to run the test suite, see what was failing, and make tickets for each group of remaining failures. Then I use optio to manage instances of agents working on and closing out each ticket.

Oh good question, I haven't thought deeply about this.

Right now nothing special happens, so claude/codex can access their normal tools and make web calls. I suppose that also means they could figure out they're running in a k8s pod and do service discovery and start calling things.

What kind of features would you be interested in seeing around this? Maybe a toggle to disable internet connections or other connections outside of the container?

There are a few things:

a) you can create CI/build checks that run in github and the agents will make sure pass before it merges anything

b) you can configure a review agent with any prompt you'd like to make sure any specific rules you have are followed

c) you can disable all the auto-merge settings and review all the agent code yourself if you'd like.

Is that true? I thought if you compiled a rust crate with, `#[deny(unsafe_code)]`, there would not be any issues. xmloxide has unsafe usage only in the the C FFI layer, so the rest of the system should be fine.

Yes, in testing I did add four fuzzing targets to the repo:

1. fuzz_xml_parse: throws arbitrary bytes at the XML parser in both strict and recovery mode

2. fuzz_html_parse: throws arbitrary bytes at the HTML parser

3. fuzz_xpath: throws arbitrary XPath expressions at the evaluator

4. fuzz_roundtrip: parse → serialize → re-parse, checking that the pipeline never panics

Because this project uses memory safe rust, there isn't really the need to find the memory bugs that were the majority of libxml2's CVEs.

There is a valid point about logic bugs or infinite loops, which I suppose could be present in any software package, and I'm not sure of a way to totally rule out here.

Because it was written in C, libxml2's CVE history has been dominated by use-after-free, buffer overflows, double frees, and type confusion. xmloxide is written in pure Rust, so these entire vulnerability classes are eliminated at compile time.