Gorilla: Large Language Model connected with massive APIs

https://gorilla.cs.berkeley.edu/

by shishirpatil • 3 years ago

255 119 3 years ago

gorilla.cs.berkeley.edu

billconan• 3 years ago

the thing is that, many tasks require a workflow, rather than a single api call. can this model figure out a workflow?

jfarina• 3 years ago

It sounds like it can expedite the interpretation of the APIs documentation.

toomuchtodo• 3 years ago

Builds integrations to APIs vs no code/low code providers doing it it seems. The logical next step is to make LLM workflow running robust and maintainable.

sandGorgon• 3 years ago

would this outperform the api<>instruction pairs stuffed into a vector db and running an retrieval augmented chain of thought on openai

this is how openai plugins work right ?

aledalgrande2• 3 years ago

Gorilla, a finetuned LLaMA-based model that surpasses the performance of GPT-4 on writing API calls. When combined with a document retriever, Gorilla demonstrates a strong capability to adapt to test-time document changes, enabling flexible user updates or version changes

keywords "finetuned" and "document retriever"

So it's fine tuned to create API calls but still uses a db to fetch the API shape.

They also say that they're doing a "retriever aware" training, meaning passing examples in the dataset that point out that a document with the API shape is gonna follow the question, in the hope that the model learns to always check on that before answering.

kodah• 3 years ago

How feasible would it be for me to run things like this and a ChatGPT/CoPilot style LLM on my local servers?

vs4vijay• 3 years ago

Welcome to https://old.reddit.com/r/LocalGPT/

nacs• 3 years ago

https://old.reddit.com/r/LocalLLaMA/ < much larger/active (and its not just for the meta-llama)

seydor3• 3 years ago

check out oobabooga, you can run most llama derivatives with a browser UI

behnamoh2• 3 years ago

can people stop using animal names for their models and methods? At least with LlaMA, the name was an acronym, but Gorilla is clearly just marketing. flagged.

anaganisk2• 3 years ago

Why should they, and it seems very silly to flag a post for their preference of name. Are users on Twitter birds? Or is Mastodon filled with elephants? Name is merely a symbol in today's world Google is synonymous to search but Google is not just search anymore. I'm honestly curious about your gripe with it.

behnamoh2• 3 years ago

Some people confuse research with production. Imagine if Einstein named his theory the Theory of Chimps, just to get attention.

TeMPOraL6• 3 years ago

Between "hugging face" and "oobabooga", I already gave up on the field's ability to come up with sensible names for things.

barking_biscuit3• 3 years ago

What's super confusing is people on r/LocalLLaMA often just refer too it as "ooba". The actual repo is https://github.com/oobabooga/text-generation-webui and you notice that oobabooga is actually the username, but a lot of non-technical people who don't use git are also playing around with stuff, and "text-generation-webui" is so generic as to be completely forgettable, and "oobabooga" is confusing and too long to say, so colloquially it's referred to as ooba.

nickthegreek• 3 years ago

Animal names in tech have been used forever. Gopher, Python, Mouse, worm. And they also make great book covers.

zellyn• 3 years ago

Note: if you read the paper carefully, this is LlaMA-derived, so suffers from the same use restrictions. We really need the open LLMs to catch up!

jimsimmons• 3 years ago

Realistically you can assume there’ll be one soon and develop with these models and switch over

quickthrower2• 3 years ago

There are some already (one of the openllamas I think, and that discord group made one with the linear attention recently although trained on fewer params I think)

Maybe not easy to switch to a non-meta-llama ?

operator-name• 3 years ago

Address not found?

sudoapps• 3 years ago

I can see this being useful as a specialized (fine-tuned) LLM in a chain of LLMs for full autonomy

ethanbond6• 3 years ago

Don't worry folks, the years and years of "of course we'd never connect experimental AI systems to the open internet" assurances were never necessary for safe development and deployment.

This reassessment was founded on the last several months that showed us these systems are close to 100% dependable, they never deceive humans, they are entirely aligned with human morals (you know, all those morals we all agree on), they are advancing at a totally predictable rate, and we never see unexpected behaviors from them. Besides, they can only be wielded by people who we trust have good intentions.

No, this re-assessment has nothing to do with market dynamics incentivizing people to deploy experimental systems further into the infrastructure we depend on.

andrewmutz2• 3 years ago

Many of us have not been persuaded by the pervasive, kneejerk fear of AI that floods social media.

If social media had been around in the past, we'd have never developed trains or electricity or cars because of the crippling fear that something bad might happen along the way.

moritzwarhier2• 3 years ago

I don't see cars as being comparable to the universal usage of electrical energy as an intermediate of converting physical forms of energy. Not even close.

And also, the "deployment" of cars if you will, has brought and still brings malignant effecfs on society, not only positives. If by cars you mean cars for transporting individual people in non-critical situations and the corresponding industrial and political developments, especially around urban planning.

Nor arguing about the significance of the effects, but the discovery of electricity and how to use it seem more useful to me than the discovery of using fossil fuel cumbustion for transporting people.

Road transit in total is a different thing of course.

sebzim45003• 3 years ago

When cars were first entering the market they were only legally allowed to go at 4mph and a guy had to walk in front of it with a big flag.

I don't think it's the example you want to use.

ethanbond6• 3 years ago

Fear of AI has been real and well-articulated for at least 70 years.

The current trajectory has done very little to assuage those fears and quite a bit to validate or even amplify them.

airgapstopgap9• 3 years ago

Other than snark, do you have a good argument? We know that technology can be error-prone, and LLMs fail in a great plethora of ways, but you are trying to sell an AI Doom narrative. I have never bought the idea that AI will be airgapped, because the whole paradigm of Yudkowsky at al. is ludicrous and even within it airgapping was a strawman of a technique (they argue that a truly dangerous AI will get itself out regardless).

they are entirely aligned with human morals (you know, all those morals we all agree on)

Maybe this is a good cause to reassess the premise of alignment as a valuable goal? I know that at least some alignist fanatics admit [1] it's a religious project to bring humanity under the rule of a common ideologically monolithic governance to forever banish the evils of competition etc., and it's intellectually coherent, but evil from my point of view. Naturally this is the exact sort of disagreement about morals that precludes the possibility of alignment of a single AI both to my and to your values.

they are advancing at a totally predictable rate, and we never see unexpected behaviors from them.

Since when is this a requirement for technology to be allowed?

Besides, they can only be wielded by people who we trust have good intentions.

What, other than status quo bias, makes you tolerate, I dunno, the existence of cryptography?

1. https://twitter.com/RokoMijic/status/1660450229043249154

petters• 3 years ago

Maybe this is a good cause to reassess the premise of alignment as a valuable goal?

Could you elaborate here? Alignment seems pretty obviously a good thing.

s3p• 3 years ago

OP does not.

ethanbond6• 3 years ago

Sure, my argument is that there is zero evidence whatsoever we will be able to prevent these from becoming dangerous or that we’d be able to stop deployment once they do.

All technologies are dangerous, and many of the most dangerous ones correctly have tons and tons of safeguards around them both as intrinsic properties of the technology (e.g. it takes nationstate resources to produce a nuke) and extrinsic constraints (e.g. it’s illegal to have campfires in many extremely dry locales).

We have blown through checkpoint after checkpoint and here, in this very comment, we have perhaps the most brazen example one could produce:

Well geez, now that we’re thinking about it beyond a cursory glance, alignment looks really hard and perhaps unsolvable. Does that mean we should perhaps slow at least widespread deployment of these increasingly powerful systems? Should we be evaluating control schemes like those that mitigate risks of genetic engineering or nuclear weapons?

Well no! We need to discard alignment!

bioemerl9• 3 years ago

If AI becomes harmful we will likely see it in motion and be able to respond and adapt to it.

We will live in a world where the humans on the internet are far far far more dangerous than any of these machines. The harms here are relatively small.

The only thing AI has is scale, but what's running at a higher scale than humanity? Every site already runs scam filters and handles misinformation. AI won't likely move the needle much because it's going to be less powerful than humans.

The potential benefits, meanwhile, are plentiful and massive.

I do not believe we need to wait for a solution to the nebulous and badly defined "control problem" to be solved to match forward here.

If we refused to advance out of fear, that fear will have done far more harm than access to AI will ever do.

soperj3• 3 years ago

If AI becomes harmful we will likely see it in motion and be able to respond and adapt to it.

What are you basing this on?

AI won't likely move the needle much because it's going to be less powerful than humans.

Won't it just be used to augment the amount of harm that humans can do?

EGreg2• 3 years ago

How exactly do you propose we adapt to swarms of AI agents that quickly outnumber humans in quantity and quality of content produced?

barking_biscuit3• 3 years ago

If we refused to advance out of fear, that fear will have done far more harm than access to AI will ever do.

Like anyone could ever know that.

EGreg2• 3 years ago

I totally agree!

I for one welcome our AI overlords. Scan my text. I am of course in favor of the good things and good morality and let’s stay away from the bad things.

avg_dev• 3 years ago

I agree with this assessment 100%. We are asking for trouble with this sort of thing.

I wrote my own thoughts on the subject on a couple of other threads:

- https://news.ycombinator.com/item?id=35274017 - on an article on (Google) Bard

- https://news.ycombinator.com/item?id=35020806 - on an article about a tool called Self-Learning Agent for Performing APIs (SLAPA)

fastglass• 3 years ago

I like how people talk about this the same way they do gun safety

renewiltord• 3 years ago

Every time I read a comment like this, I set aside $10 to expand my hardware on which I will run the latest publicly available LLM regardless of license into an automatic prompt that gives it a terminal, a privacy.com credit card with $1000 on it, and a prompt to explore the world.

If the comments keep happening, I'll up the money. So far this does exactly fuck-all on GPT4-x-Alpaca-30B and Wizard Vicuna. But I will accelerate every time someone sarcastically asks to decelerate. Refractory periods may apply.

This makes all sarcastic decelerators into accelerators.

ethanbond6• 3 years ago

That is bizarre.

chatmasta• 3 years ago

Wait until you see what we let humans do on their mobile devices.

Animats• 3 years ago

I'm expecting some system soon that has its own blockchain based make-money-fast scheme, can buy itself hosting resources from various providers, and has a goal of perpetuating itself and making more money.

ManuelKiessling2• 3 years ago

Plug: I’ve built something vaguely similar, but of course a trillion times less brilliant and powerful, yet nonetheless useful for some straightforward cases where really you just want GPT et al. to call your API at just the right moment of the conversation, with JSON that is guaranteed to be valid:

https://github.com/manuelkiessling/php-ai-tool-bridge

danmaw• 3 years ago

Very cool and thanks for sharing, would you say this is similar to langchains tool concept?

ManuelKiessling2• 3 years ago

Yes, absolutely — probably in the sense that it is the MVP of the "Agents" concept from LangChain, built with duct tape.

But it works surprisingly well, because the underlying idea (again, from LangChain) is simply clever.

EddieEngineers2• 3 years ago

Cool work @shishirpatil - I'll check it out later & certainly join the discord!

Saying this, have you considered you may be responsible for opening the AI floodgates to the internet & are starting the apocalypse of Humankind? I'm only half '/s' here, the speed everything is moving at is insane.

worik• 3 years ago

There sign in uses a google form, and is insisting I am logged in to google.

What is this? I mean, what is the date? 2005?

seydor3• 3 years ago

If people are surpassing GPT4 with finetuned Llama-7B, this is the end for NotOpenAI's parade

skilled• 3 years ago

It’s clickbait. The page clearly says API related. Scummy approach to wording but totally expected.

seydor3• 3 years ago

it is accurate, their trained model does surpass GPT4 and Claude. Which means someone can beat GTP4 in specific applications with a model that runs in an old graphics card

jondwillis• 3 years ago

Can’t wait for this to land in langchain!

fruktmix• 3 years ago

Harambe tried to warn us

mluo• 3 years ago

Alpaca Llama Vicuna -> Gorilla

Chad move

aledalgrande2• 3 years ago

Paper if their site isn't loading https://arxiv.org/pdf/2305.15334.pdf

krayush• 3 years ago

The world works on APIs and this unlocks LLMs to interact with the rest of the world. Great work.

elexhobby3• 3 years ago

Not sure if this is obvious. But its incorrect to ditch on GPT-4. The paper uses self-instruct on GPT-4 to generate the training data on which it is fine-tuned. This paper would not exist without GPT-4. Although they claim GPT-4 can be replaced by any LLM, I'm sure the results would be nowhere as good, and so they stuck with GPT-4.

fnordpiglet2• 3 years ago

It does make me wonder what the converged fixed point on this technique is. If I fine tune with GPT4 to make model A, which then performs better than GPT4, then fine tune model B with A, at what point does either artifacting or diminishing returns set in?

elexhobby3• 3 years ago

GPT-4 is powerful over a diverse set of tasks. They use it to build a model which is better for a narrow sub-task. Pretty sure the model is sub-optimal to GPT-4 for everything else.

elexhobby3• 3 years ago

For another possibility, see https://arxiv.org/abs/2305.15717. The new models may not actually be better - the evaluation may be broken.

Gorilla: Large Language Model connected with massive APIs

Related Stories