Show HN: Fine-grained stylistic control of LLMs using model arithmetic

https://github.com/eth-sri/language-model-arithmetic

by OcelotBane • 3 years ago

85 8 3 years ago

We developed a new framework that enables flexible control of generated text in language models. By combining several models and/or system prompts in one mathematical formula, it lets you tweak your style and combine model outputs with ease. A handy tool for those working with LLMs, looking for more fine-grained control of stylistic output. More details in our paper: https://arxiv.org/abs/2311.14479. Feedback and potential applications are welcome.

Der_Einzige• 3 years ago

This is EXCELLENT work! Exactly the sorts of stuff I was wanting to see more of when I complained about the lack of LLM related prompt tools:

https://gist.github.com/Hellisotherpeople/45c619ee22aac6865c...

Work like this is more important than much of what VC backed startups are producing. Work like yours may continue to languish (unjustly) for a b it before investors get a clue and realize that tools like this, automatic1111, and other tools which give lots of control to the users on how they use their models are actually the most lucrative investments.

spacecadet• 3 years ago

As someone who is working on the same problem, 1, kudos to the team! 2, appreciate your comment.

carterschonwald• 3 years ago

Could I do a similar thing by doing convex sums of models for token sampling ?

OcelotBane• 3 years ago

Convex sums of different models are also supported, as long as the models share the same vocabulary/tokenizers. As soon as models have different tokenizers, we cannot sum the outputs of the models unless we find some way to combine the vocabularies of both tokenizers. Applications of these convex sums could be (1) model ensembling and (2) Contrastive Decoding (https://arxiv.org/abs/2309.09117) which uses a formula very similar to TopPTopK(M_large, top_p=0.9) - 0.5 * M_small.

Roark66• 3 years ago

I really like the README for this code on github. I wish more papers code explained how to replicate the results.

rosecross• 3 years ago

My first thought was this could be great for alignment, e.g., M_aligned = M - M_toxic.

Turns out, that’s exactly what they’ve explored in the paper.

Great stuff.

Oranguru• 3 years ago

Thank you for your work. I have been having trouble achieving the desired level of formality in the generated text. When I ask for slightly formal content, the result tends to be too formal. However, when I ask the model to reduce the formality or use a semi-formal tone, the text becomes too informal. This will allow me to exercise more control over the style of the model's output and stop constantly battling with it.

jay-barronville• 3 years ago

https://ar5iv.labs.arxiv.org/html/2311.14479 ← Here’s the link to the web-based version of the paper, for anyone who—like me—prefer to read arXiv papers via the web instead of downloading PDF documents.

Show HN: Fine-grained stylistic control of LLMs using model arithmetic

Related Stories