langs

the current bottlenecks for agents are actually bottlenecks of the LLMs, rather than the agents themselves.

That’s a good analogy, but I think the reason we currently need the "nurse" role is the need to interact with the physical world. Most software products don't require this step, so the demand for “nurses” or "doctors" will likely decrease. If robotics technology continues to advance, the number of nurses in real world will also decrease in the foreseeable future.

How does your solution differ from other KV Cache approaches, such as LMCache?

nope. any rewrite will be an AI rewrite soon.

I vide-coded some, but the core is hand-craft arch/algorithm, that's the most valuable part I want to protect.