HN user
I do something similar, embed large compressed data blocks as Lua string literals in some project, and I've just given up and used
lua_string_literal = string.format("%q", compressor(data))
I don't know if that's available in Cobalt, but for standard Lua I like the compromise, it's not /that/ wasteful.
To push the trail metaphor further, every summer I visit a particular national park. One of my favorite trails there was the main route through the forest, which also served as a secondary path for rangers transporting equipment by car. At several points, the trail crossed rivers, and the bridges always seemed overengineered, with unusually high weight limits. Especially since the rest of the trail was barely passable by vehicle.
A few years later, the park began offering cottages along the route and started paving critical sections, particularly steep areas where trucks (presumably transporting building materials for the cottages) struggled. Had those bridges not already been built to support these trucks, the park would have needed to close the trail for upgrades. Or make a second trail with the new requirements in mind.
The trail expanded faster than it was paved, but each year you could go further as the trail grew longer, and faster as key sections were improved.
Yeah and using combobulate feels so natural, I just wish it was easier to add support for other languages. In addition to the navigation, and simple edits (e.g. M-k kills node) you have Searching/editing/etc. using tree-sitter queries directly within emacs, possibly replacing other uses of external tree-sitter based tools.
It makes editing supported languages almost as pleasant as editing Lisp.
With how well the integration already work in existing editors, I'm curious how the UX can be improved further by an editor made specifically with AST editing in mind.
I was very surprised to see the results for common lisp. As I scrolled down I just figured that the language was not included until I saw it down there. I would have guessed SBCL to be much faster. I checked it out locally and got: Rust 9ms, D: 16ms, and CL: 80ms.
Looking at the implementation, only adding type annotations, there was a ~10% improvement. Then the tag-map using vectors as values which is more appropriate than lists (imo) gave a 40% improvement over the initial version. By additionally cutting a few allocations, the total time is halved. I'm guessing other languages will have similar easy improvements.
Exactly yes, that's what I was going to comment. You sometimes need to debug at every layers. All abstractions end up leaking in some way. It's often worth it, but it does not save us from the extra cognitive load and from learning the layers underneath.
I'm not necessarily against the approach shown here, reducing tokens for more efficient LLM generation; but if this catches on, humans will read and write it, will write debuggers and tooling for it, etc. It will definitely not be a perfectly hidden layer underneath.
But why not, for programming models, just select tokens that map concisely existing programming languages ? Would that not be as effective ?