Smartfunc: Turn Docstrings into LLM-Functions

by alexmolas • 1 year ago

70 25 1 year ago

shaism• 1 year ago

Very cool. I implemented something similar for personal use before.

At that time, LLMs weren't as proficient in coding as they are today. Nowadays, the decorator approach might even go further and not just wrap LLM calls but also write Python code based on the description in the Docstring.

This would incentivize writing unambiguous DocStrings, and guarantee (if the LLMs don't hallucinate) consistency between code and documentation.

It would bring us closer to the world that Jensen Huang described, i.e., natural language becoming a programming language.

psunavy032• 1 year ago

People have been talking about natural language becoming a programming language for way longer than even Jensen Huang has been talking about it. Once upon a time, they tried to adapt natural language into a programming language, and they came up with this thing called COBOL. Same idea: "then the managers can code, and we won't need to hire so many expensive devs!"

And now the COBOL devs are retiring after a whole career . . .

pizza• 1 year ago

But isn't it actually more like, COBOL lets you talk in COBOL-ese (which is kinda stilted), whereas LLMs let you talk in LLM-ese (which gets a lot closer to actual language)? And then since the skill cap on language is basically infinite, that this becomes a question of how good you are at saying what you want - to the extent it intersects with what the LLM can do.

lukev2• 1 year ago

This is the way LLM-enhanced coding should (and I believe will) go.

Treating the LLM like a compiler is a much more scalable, extensible and composable mental model than treating it like a junior dev.

simonw4• 1 year ago

smartfunc doesn't really treat the LLM as a compiler - it's not generating Python code to fill out the function, it's converting that function into one that calls the LLM every time you call the function passing in its docstring as a prompt.

A version that DID work like a compiler would be super interesting - it could replace the function body with generated Python code on your first call and then reuse that in the future, maybe even caching state on disk rather than in-memory.

hedgehog• 1 year ago

I use something similar to this decorator (more or less a thin wrapper around instructor) and have looked a little bit at the codegen + cache route. It gets more interesting with the addition of tool calls, but I've found JSON outputs create quality degradation and reliability issues. My next experiment on that thread is to either use guidance (https://github.com/guidance-ai/guidance) or reimplement some of their heuristics to try to get tool calling without 100% reliance on JSON.

toxik• 1 year ago

Isn’t that basically just Copilot but way more cumbersome to use?

photonthug• 1 year ago

Treating it as a compiler is obviously the way right? Setting aside overhead if you’re using local models.. Either the code gen is not deterministic in which case you risk random breakage or it is deterministic and you decided to delete it anyway and punt on ever changing / optimizing it except for in natural language? Why would anyone prefer either case? Code folding works fine if you just don’t want to look at it ever.

I can see this eventually going in the direction of "bidirectional synchronization" of NL representation and code representation (similar to how jupytext allows you work with notebooks in browser or markdown in editor). But a single representation that's completely NL and deliberately throwing away a code representation sounds like it would be the opposite of productivity..

huevosabio• 1 year ago

Yes, that would be indeed very interesting.

I would like to try something like this in Rust: - you use a macro to stub out the body of functions (so you just write the signature) - the build step fills in the code and caches it - on failures the, the build step is allowed to change the function bodies generated by LLMs until it satisfies the test / compile steps - you can then convert the satisfying LLM-generated function bodies into a hard code (or leave it within the domain of "changeable by the llm")

It sandboxes what the LLM can actually alter, and makes the generation happen in an environment where you can check right away if it was done correctly. Being Rust, you get a lot of more verifications. And, crucially, keeps you in the driver's seat.

lukev2• 1 year ago

Ah, cool, didn't read close enough.

Yeah, I do think that LLMs acting as compilers for super high-level specs (the new "code") is a much better approach than chatting with a bot to try to get the right code written. LLM-derived code should not be "peer" to human-written code IMO; it should exist at some subordinate level.

The fact that they're non-deterministic makes it a bit different from a traditional compiler but as you say, caching a "known good" artifact could work.

hombre_fatal• 1 year ago

https://github.com/eeue56/neuro-lingo

You can even pin the last result:

    pinned function main() {
      // Print "Hello World" to the console
    }

vrighter• 1 year ago

a compiler has one requirement that llms cannot provide. It has to be robust.

simonw4• 1 year ago

I really like how this integrates with the schema feature I added to the underlying LLM Python library a few weeks ago: https://simonwillison.net/2025/Feb/28/llm-schemas/#using-sch...

noddybear• 1 year ago

Cool! Looks a lot like Tanuki: https://github.com/Tanuki/tanuki.py

nate_nowack2• 1 year ago

yea its a popular DX at this point: https://blog.alternatebuild.dev/marvin-3x/

miki123211• 1 year ago

There's also promptic which wraps litelm, which supports many, many, many more model providers, and it doesn't even need plugins.

Llm is a cool cli tool, but IMO litellm is a better Python library.

simonw4• 1 year ago

I think LLM's plugin architecture is a better bet for supporting model providers than the way LiteLLM does it.

The problem with LiteLLM's approach is that every model provider needs to be added to the core library - in https://github.com/BerriAI/litellm/tree/main/litellm/llms - and then shipped as a new release.

LLM uses plugins because then there's no need to sync new providers with the core tool. When a new Gemini feature comes out I ship a new release of https://github.com/simonw/llm-gemini - no need for a release of core.

I can wake up one morning and LLM grew support for a bunch of new models overnight because someone else released a plugin.

I'm not saying "LLM is better than LiteLLM" here - LiteLLM is a great library with a whole lot more contributors than LLM, and it's also been fully focused on being a great Python library - LLM has also had more effort invested in the CLI aspect than the Python library aspect so far.

I am confident that a plugin system is a better way to solve this problem generally though.

asadm• 1 year ago

I was working on a similar thing but for JS.

Imagine this: It would be cool when these functions essentially boiled down to a distilled tiny model just for that functionality instead of an api call to foundation one.

dheera• 1 year ago

I often do the reverse -- have LLMs insert docstrings into large, poorly commented codebases that are hard to understand.

Pasting a piece of code into an LLM with the prompt "comment the shit out of this" works quite well.

simonw4• 1 year ago

Matheus Pedroni released a really clever plugin for doing that with LLM the other day: https://mathpn.com/posts/llm-docsmith/

You run it like this:

  llm install llm-docsmith
  llm docsmith ./scripts/main.py

And it uses a Python concrete syntax tree (with https://pypi.org/project/libcst/) to apply changes to just the docstrings without risk of editing any other code.

nonethewiser• 1 year ago

Funny. I frequently give the LLM the function and ask it to make the doc string.

TBH I find doc strings very tedious to write. I can see how this would be a great specification for an LLM but I dont know that its actually better than a plain text description of the function since LLMs can handle those just fine and they are easier to write.

senko• 1 year ago

Many libraries with the same approach suffer the same flaw: can't easily use the same function with different LLMs at runtime (ie. after importing the module where it is defined).

I initially used the same approach in my library, but changed it to explicitly pass the llm object around and in actual production code it's easier/more flexible to use.

Examples (2nd one also with docstring-based llm query and structured answer): https://github.com/senko/think?tab=readme-ov-file#examples

_1tan• 1 year ago

Is there something like this but for Java?

Smartfunc: Turn Docstrings into LLM-Functions

Related Stories