sluongng

how would this help smaller labs? would it put more burdens on them when trying to compete with trillion-dollar companies or would it help?

Yeah I vibe coded https://github.com/sluongng/capyfun during a hackathon recently to add a generative transformation layer on top of the traditional imperative transformations.

There are plenty of cool advancements in reducing inference cold start when I was meeting with folks in person at FOSDEM this year. However, I still struggle to understand: why would folks care about this?

Major AI Labs all have secured their own compute in the form of hardware, data center, and power generation. That means their resource pool is fixed, and they can do all sorts of tricks to pre-load, pre-allocate, etc... to improve on inference latency.

Cold start is usually a solution for "cloud" environment when your pool is flexible, and you only pay for what you use. Its effectiveness lowered in bare-metal settings as folks do not care about scaling up and down as much.

So my question is: who is this for? AWS and GCP running Anthropic models?

You can run bisect with first-parent

My teammate has a great time reimplementing Ninja (slop-free) in Go here https://github.com/buildbuddy-io/reninja to make it even faster with Remote Build Execution.

Yeah, I don't disagree with your assessment at all. I think the H2A ratio is still a good metric for the AI adoption rate of an organization. At a higher H2A ratio, you will also start to hear people measuring things using token volumes, which I think is also a similar metric (because most models nowadays run on a relatively fixed Tokens/second speed).

All of this is not a direct signal to a productivity boost. I think at higher volumes, you will need to start to account for the "yield" rate of the token volumes above: what are the volumes of tokens that get to the final production deployment? At which stage is it a constraint on the yield? Is it the models, or is it the harness, or something else (i.e. Code Review, CI/CD, Security Scans etc...)? And then it becomes an optimization problem to reduce the Cost of Goods Sold while improving/maintaining Revenues. The "productivity" will then be dissolved into multiple separate but more tangible metrics.

I do. The reason why the current generation of agents are good at coding is because the labs have sufficient time and computes to generate synthetic chain-of-thoughts data, feed those data through RL before use them to train the LLMs. These distillation takes time, time which starts from the release of the previous generation of models.

So we are just now getting agents which can reliably loop themselves for medium size tasks. This generation opens a new door towards agent-managing-agents chain of thoughts data. I think we would only get multi-agents with high reliability sometimes by the mid to end of 2026, assuming no major geopolitical disruption.

Yeah the 8 agents limit aligns well with my conversations with folks in the leading labs

https://open.substack.com/pub/sluongng/p/stages-of-coding-ag...

I think we need much different toolings to go beyond 1 human - 10 agents ratio. And much much different tooling to achieve a higher ratio than that

Most of the time, the CI resources in a python monorepo is not spent on packaging. It’s spent on running the tests.

I would love to read more about how the author is tackling the testing problem in their setup.

https://sluongng.substack.com/i/186718212/test-is-king I wrote about this less than a month ago. Things are moving pretty fast in this direction.

Oh this is really neat for the Bazel community, as depending on tree-sitter to build a gazelle language extension, with Gazelle written in Go, requires you to use CGO.

Now perhaps we can get rid of the CGO dependency and make it pure Go instead. I have pinged some folks to take a look at it.

Not yet in Linux?

I suspect they just use no_std whenever its applicable

https://github.com/facebook/buck2/commit/4a1ccdd36e0de0b69ee...

https://github.com/facebook/buck2/commit/bee72b29bc9b67b59ba...

Turn out if you have strong control over the compiler and linker instrumentations, there are a lot of ways to optimize binary size

Zstd dictionary compression is essentially how Meta's Mercurial fork (Sapling VCS) stores blobs https://sapling-scm.com/docs/dev/internals/zstdelta. The source code is available in GitHub if folks want to study the tradeoffs vs git delta-compressed packfiles.

I think theoratically, Git delta-compression is still a lot more optimized for smaller repos. But for bigger repos where sharding storaged is required, path-based delta dictionary compression does much better. Git recently (in the last 1 year) got something called "path-walk" which is fairly similar though.

The evolving schema is much more attractive than a bunch of plain text HTTP headers when you want to communicate additional metadata with the file download/upload.

For example, there are common metadata such as the digest (hash) of the blob, the compression algorithm, the base compression dictionary, whether Reed-Solomon is applicable or not, etc...

And like others have pointed out, having existing grpc infrastructure in place definitely helps using it a lot easier.

But yeah, it's a tradeoff.

https://github.com/googleapis/googleapis/blob/master/google/... is a more complete version of this. It supports resumable uploads, and the download can start from an offset within a file, allowing you to download only part of the file instead of the whole.

Another version of this is to use grpc to communicate the "metadata" of a download file, and then "side" load the file using a side channel with http (or some other light-weight copy methods). Gitlab uses this to transfer Git packfiles and serve git fetch requests iirc https://gitlab.com/gitlab-org/gitaly/-/blob/master/doc/sidec...

The underlying problem is that most container images are not cache efficient. Compressed tarballs arent and that’s what most of container images are. And Bazel relies heavily on caching to stay fast.

Most of the hyper scaler actually do not store container images as tarballs at scale. They usually flatten the layers and either cache the entire file system merkle tree, or breaking it down to even smaller blocks to cache them efficiently. See Alibaba Firefly Nydus, AWS Firecracker, etc… There is also various different forms of snapshotters that can lazily materialize the layers like estargz, soci, nix, etc… but none of them are widely adopted.

They use Google's web indexing technology adapted for trigrams, which was mostly developed to support their massive internal monorepo

Do you have a source for this? I would love to read more about it.

In the doc of the Zoekt repo, it says

What does cs.bazel.build run on?

Currently, it runs on a single Google Cloud VM with 16 vCPUs, 60G RAM and an attached physical SSD.

https://github.com/sourcegraph/zoekt/blob/main/doc/faq.md#wh...

so at least they were using Zoekt up until a certain point in the past.

Nice. This is one of the main reasons why I picked CachyOS recently. Now I can fallback to Ubuntu if CachyOS gets me stuck somewhere.

The most concerning part about modern CI to me is how most of it is running on GitHub Actions, and how GitHub itself has been deprioritizing GitHub Actions maintenance and improvements over AI features.

Seriously, take a look at their pinned repo: https://github.com/actions/starter-workflows

Thank you for your interest in this GitHub repo, however, right now we are not taking contributions.

We continue to focus our resources on strategic areas that help our customers be successful while making developers' lives easier. While GitHub Actions remains a key part of this vision, we are allocating resources towards other areas of Actions and are not taking contributions to this repository at this time.

I don't view it as a bug. It's a personality trait of the model that made "user steering" much easier, thus helping the model to handle a wider range of tasks.

I also think that there will be no "perfect" personality out there. There will always be folks who view some traits as annoying icks. So, some level of RL-based personality customization down the line will be a must.

Why not https://man7.org/linux/man-pages/man2/flock.2.html?

More on Netflix's Remote Workstation setup for artists https://aws.amazon.com/solutions/case-studies/netflix-workst...

I think this is a common practice by now.

Here is a talk from Netflix about cloud workspace for their artists https://aws.amazon.com/solutions/case-studies/netflix-workst...

Hmm it’s weird that this submission and comments are being shown to me as “hours ago” while they are all 2 days old

For performance, Go using CGO is going to be closer to Python than Go.

This is not true. In a lot of libraries, unless there are asm implementations on the pure Go side, the CGO implementation often outperforms. Zstd used to be one of the most notable example.

CGO is slow and often still painful to build cross platform.

This is true. I found that using Bazel to manage the entire build graph made CGO a lot easier to deal with. By adopting Bazel, you formalize the cost of operating a cc toolchain and sysroot up front instead of hiding it inside nested layers of environment variables and CI container images. Bazel also made your build faster and cross-platform CGO easier.

Go is no longer able to be built into a single static binary.

The "C" portion of CGO can be made static as well. It often results in a much bigger binary than a dynamically-linked binary though. In setups where you control the runtime environment (i.e. web servers), I don't see a clear benefit in shipping duplicate bytes inside a static binary. Even for consumer-facing use cases (i.e. devtools), a static binary can be too big and unwieldy versus just a set of pre-built binaries targeting specific platforms via Github Release.

Let me try to take the other side:

`ctx.Value` is an `any -> any` kv store that does not come with any documentation, type checking for which key and value should be available. It's quick and dirty, but in a large code base, it can be quite tricky to check if you are passing too many values down the chain, or too little, and handle the failure cases.

What if you just use a custom struct with all the fields you may need to be defined inside? Then at least all the field types are properly defined and documented. You can also use multiple custom "context" structs in different call paths, or even compose them if there are overlapping fields.

Isn’t Rekor runs on top of Trillian?

Totally noob question: what is "redis-proxy" in this doc? Envoy with Redis protocol codec?

Could you please elaborate on how repack/gc is handled with a libgit2 backend? I know that Alibaba has done something similar in the past based on libgit2, but I have yet to see another implementation in the wild like this.

Very cool project. I hope Cloudflare workers can support more protocols like SSH and GRPC. It's one of the reasons why I prefer Fly.io over Cloudflare worker for special servers like this.