MadsRC

Amazing news!

The one thing I wish they’d support is multiple zones and an RBAC system to grant certain users access to specific zones. If they’d offer that they’d be a serious contender to replace Cloudflare and AWS/GCP DNS

Not really, and neither does the doc it links to. It explains the difference between a personal agent (that acts as you) and a multiplayer agent (that has its own identity).

But they walk all this back by saying that for PRs in GitHub it uses the upstream Claude for GitHub app - that is one installation of a GH app, which means one identity, one list of repos it can access. In the audit logs it will be impossible to see if it was Claude Tag from Channel Y or Claude Tag from channel X.

Arguably that’s a limitation of the abysmal state of machine identities in GH.

Agent identity and attribution continues to be important, but current systems makes it oh so difficult.

I’d be curious how they’ve solved the attribution/provenance/identity problem here. Are instances of Claude Tag, across channels, sharing the same identity? Can I grant one instance access to a range of AWS roles and another instance access to other roles?

During an incident, how do I know which Claude Tag called AWS?

I would love to see a forge that consists of composable components, self-host able with federated identities and maybe a few centralized indexers

The approval part is really interesting - No problems with timeouts or operators not being around to approve?

Ah yes, the API will accept requests that doesn’t include the client attestation (or the fingerprint from src/utils/fingerprint.ts. At least it did a couple of weeks back.

They are most likely using these as post-fact indicators and have automation they kicks in after a threshold is reached.

Now that the indicators have leaked, they will most likely be rotated.

What signing?

Are you referencing the use of Claude subscription authentication (oauth) from non-Claude Code clients?

That’s already possible, nothing prevents you from doing it.

They are detecting it on their backend by profiling your API calls, not by guarding with some secret crypto stuff.

At least that’s how things worked last week xD

We were not. I reached out to the team at BerriAI to offer my assistance as a security professional, given that they requested help from security experts.

Dropped you a mail from mads.havmand@nansen.ai

Very interesting!

I’ve got an internal tool that we use. It doesn’t do the deterministic classifier, but purely offloads to an LLM. Certain models achieve a 100% coverage with adversarial input which is very cool.

I’m gonna have a look at that deterministic engine of yours, that could potentially speed things up!

Well, the head of reliability did leave a month or two ago zD

Last I heard of it this was proposed as a directive as opposed to regulation, meaning every single member state would have to interpret it and create their own national implementation. Just like with GDPR.

So 27 individual implementations of this, as opposed to the current 27 different implementations of how to incorporate and assign equity?

Seems… silly?

I’m all for making it more attractive to create startups in the EU… But I don’t think a directive is the right way

This is cool - Whenever I have a new idea for a thing I spend too much time writing boilerplate IAM and backend stuff, taking away time that could be spend on actual business logic. Thought about packaging the boilerplate stuff up before, never gotten around to it. Glad you did!

A thing to consider would be to make it easier (or perhaps bake it in) to separate out parts of the app into a separate origin. Something that would be good for pretty much any SaaS app would be to separate the IAM out (could still embed it with an iframe) - this allows you to keep a fairly tight security policy for the IAM stuff and a more lax one for the rest of the app. Kinda how Google separates out accounts.google.com.

This looks cool, but I’m sad you’ve chosen a name that already associated with another security tool :(

Now? Chainalysis has always worked for governments…

It was basically spawned out of the government needing help with investigating crypto - I think it was Mt. Gox…

Shameless plug, I pushed a small CLI for detecting unpinned dependencies and automatically fix them the other day: https://codeberg.org/madsrc/gh-action-pin

Works great with commit hooks :P

Also working on a feature to recursively scan remote dependencies for lack of pins, although that doesn’t allow for fixing, only detection.

Very much alpha, but it works.

This looks great!

With the self-host option, it’s not really clear through the docs if one is able to override the base url of the different model providers?

I’m running my own OpenAI, Anthropic, Vertex and Bedrock compatible API, can I have it use that instead?

When this was released I thought that perhaps we could mitigate it by having the tooling only load “rules” if they were signed.

But thinking on it a bit more, from the LLMs perspective there’s no difference between the rule files and the source files. The hidden instructions might as well be in the source files… Using code signing on the rule files would be security theater.

As mentioned by another comms ter, the solution could be to find a way to separate the command and data channels. The LLM only operates on a single channel, that being input of tokens.

Data exfil detection is a game of whack-a-mole. There is an endless variation of ways I can get data of your machine or out of your network.

Your time is much better spend detecting or preventing compromise.

This is quite interesting.

What does a seat entail? You talk about self serve (I love it!), but would the users that self-serve take up a seat? Or are seats just for the folks creating the modules?

I find the JetBrains integration spotty - I have to run ‘eval “$(mise activate)”’ in every terminal session for it to pick up the env. I have vague recollections of also having issues with running executables from the IDE as it is missing the env details.

The SDK discovery works great though :D

Not the parent, but I use it for projects (including Python projects) to run tests, code generation etc. when I can’t use mise.

It’s actually surprisingly efficient if you batch writes at the expense of some added latency. The WarSyream team found that batching into chunks of either 4MB of data or 250ms was optimal.

Downside is the 250ms latency. But then again, a fair amount of workloads can deal with 250ms of latency.

This is why systems such as WarpStream regularly runs compaction jobs to more efficiently store objects and cut down on API calls.

Totally unrelated to the thread, just a comment on something I noticed: You seem to be posting from 2 different accounts: soatok and some_furry. Dunno if that’s on purpose, but in the off chance that it is not: Now you know.

Love you blog, it’s nice to read something written by an actual human being nowadays. I keep several of you articles bookmarked for reference.

Does anyone know why companies don’t release under AGPL for everyone and then under a proprietary commercial license to themselves? Essentially dual-license it.

AGPL would dissuade Google and AWS from using it, and the commercial license would allow the licensee (themselves) to commercialize it?

Yup, same in Denmark. It’s called “fritstilling” - basically they pay you severance equally to the amount of months you should have gotten advanced notice (3-many months).

But there has to be a very good reason. Such as theft, or actual security worries.

What happens when “db-server-4x-16g.cluster1.fqdn” stops hosting a database?

I’m all for DNS instead of IP’s, but we need to stop encoding too much information into names…

That’s because it totally was generated by a large language model.

This seems very close to Langchain's "summary" memory functionality (which seems to have existed since March 2023?) - granted I haven't read the paper just yet