HuggingChat – ChatGPT alternative with open source models

http://huggingface.co/chat?model=mistralai/Mixtral-8x7B-Instruct-v0.1

by victormustar • 3 years ago

61 6 3 years ago

giuscri• 3 years ago

Where do I find info on how web search in LLMs work and how they’re trained to do that?

It's called retrieval augmented generation (RAG) and there's no extra training. The data (e.g. web search result) is given as input to the LLM.

If you search for "retrieval augmented generation" you'll find papers, tutorials, videos etc about it.

d4rkp4ttern• 3 years ago

Web search is not a capability of a “bare” LLM, but in an LLM-based system it can be done by giving the LLM access to a “web search tool”, I.e essentially you instruct it to output a specific structured text (typically json but doesn’t have to be) indicating its “intent” to search, and your wrapper intercepts/detects this structured response, does the actual search and returns the results (e.g snippets from top k results) into the context of the LLM amd have it use these to respond to your question.

A similar thing can be done with external documents - your wrapper retrieves docs/fragments relevant to the query, puts them in the context and lets the LLM use them to answer the query. This is called Retrieval Augmented Generation (RAG).

The above is a highly simplified description. In the Langroid library (a multi-agent framework from ex-CMU/UW-Madison researchers) we have these and more. For example here’s a script that combines web search and RAG:

https://github.com/langroid/langroid/blob/main/examples/docq...

joshstrange• 3 years ago

I can’t wait for mistralai/Mixtral-8x7B-Instruct support in llama.cpp. This models does much better in a few tests compared to other open models. GPT4 still reigns supreme but this gets a lot closer IMHO.

ZoomZoomZoom• 3 years ago

It's been pushed yesterday. Works just fine.

crakenzak• 3 years ago

I've been using this almost exclusively for `mistralai/Mixtral-8x7B-Instruct-v0.1`. Great service and hope it grows in popularity!

HuggingChat – ChatGPT alternative with open source models

Related Stories