Haystack 1.0 – open-source NLP framework to build NLProc back end applications

by antti909 • 5 years ago

109 34 5 years ago

JPKab2• 5 years ago

I've been looking at this for a while now.

My company has a need to accurately identify, with high precision, records in elasticsearch, but with a bit more of a semantic match that existing elasticsearch plugins don't support. Ideally the best of huggingface on top of elasticsearch.

Has anyone on here tried this out? Curious what your experiences are.

tholor2• 5 years ago

Semantic document search is one of the core use cases we see in the community (besides Question Answering) and Haystack was pretty much started because we saw that you need much more than just models. It's so much pain to integrate models properly with document storage (e.g. elasticsearch), route requests effectively in larger pipelines or track user feedback in production. Have you tried using DPR or sentence transformers for your case?

Disclaimer: I am one of the maintainers of Haystack:)

matheist• 5 years ago

I had a client who was using sentence transformers with elasticsearch already. My colleague suggested switching to haystack to enable a larger number of model architectures. Switching over to haystack was pretty straightforward because we just used it as a wrapper around sentence transformers, but I do remember some inconvenience around all the other dependencies that haystack pulled in.

Haystack does a lot more besides just wrapping sentence transformers, and we weren't using the rest of it, so it was just a lot of extra dependencies sitting around taking up disk space and memory (I think we had to go up to a larger instance size). I remember feeling a bit frustrated that the dependencies weren't split up into "core" and "optional" in a more fine-grained way, but maybe most users don't mind and so it doesn't make sense for them to prioritize that?

[edit: looks like there's an open issue related to this: https://github.com/deepset-ai/haystack/issues/1070]

[edit 2: 'JPKab happy to share more about using huggingface and elasticsearch. email is in my profile]

antti9099• 5 years ago

Noted, we've been discussing dependencies internally indeed :) Thanks for the highlight above!!

eriklarsonr• 5 years ago

Yeah I’m using their FAISS document store and QA pipeline to run semantic search over a set of YouTube transcripts. Was easier to set up than Jina AI in my specific use case, and the search results are actually useful. Only real constraint for me is GPU access, creating the embeddings to store in the FAISS index sans a GPU takes an unreasonable amount of time.

antti9099• 5 years ago

Glad it worked for you - thanks for sharing!

antti9099• 5 years ago

Happy to help too - feel free to ask in the community channels as well :)

visarga• 5 years ago

Tried the demo but it could not answer correctly any of my questions, maybe it didn't have the answers in its index.

tholor2• 5 years ago

The demo corpus there just contains documents about countries and capital cities. So you could try asking questions like "What's the climate of Beijing?" or "How many people live in the capital of the US?".

timomo2• 5 years ago

btw the demo can be found at https://haystack-demo.deepset.ai/

anentropic3• 5 years ago

Haystack looks great, but the demo maybe highlights some difficulties with this kind of task.

"What is the population of Italy?" ...gives the population of Rome as first answer at 78.32 relevance :)

I get similar result for some other countries.

"What is the population of Cambridge?" ...to be fair, this is an ambiguous place name as there are several around the world. However the answer it gives is quite far removed from any of them: "In 1788, Kingston had a population of 25,000", Relevance: 93.14

dexter89_kp3• 5 years ago

Interesting. My current side project is exploring semantic search using sentence transformers. Will definitely check this out.

antti9099• 5 years ago

Happy to help! - we've got Slack and everything :)

sorenbs• 5 years ago

This is really cool!

Can Haystack be used to index structured data, or just text?

Is it required to use elastic as the backend, or can you use a simpler file-based or in-memory backend?

antti9099• 5 years ago

Re structured data - in theory, yes :) We have to work a bit more in that direction. Here's the first step - querying table data, which could be really helpful for reports, financial data, etc. In regards to the storage backend - it's currently Elasticsearch, OpenSearch, SQL+FAISS/Milvus/Weaviate (when using dense vectors/dense passage retrieval). There is also an in-memory datastore using python primitives for fast prototyping.

(Also, latest features highlights here https://www.deepset.ai/blog/new-features-in-haystack-v1.0)

artembugara2• 5 years ago

Interesting. Could anyone think of a use case with news data? We index over 1 million news articles daily.

Maybe, there's a way to have something for a specific industry?

https://newscatcherapi.com/

antti9099• 5 years ago

Whoa, this is cool :) I could think of a ton of marketing (or maybe even 'devrel') applications for it.

whalesalad• 5 years ago

This is perhaps just an innocent question for the community - but it's simultaneously an ingenious approach to inbound marketing.

artembugara2• 5 years ago

That's both and I feel no shame for it.

Der_Einzige• 5 years ago

Is there any path forward to make Haystack do word-level extractive summarization? e.g. like this: https://github.com/Hellisotherpeople/CX_DB8

or like this: https://huggingface.co/spaces/Hellisotherpeople/Unsupervised...

I am trying to find anything better than these two for this task. I feel like Haystack could be an option - but I am not sure.

MilosRusic• 5 years ago

Haystack supports abstractive summarization and you have access to all models here: https://huggingface.co/models?pipeline_tag=summarization&sor.... you can build them into a pipeline e.g. with a retriever to filter out irrelevant docs (e.g. those not containing a certain word)

Probably you are interested in extractive summarization for explainability reasons? To overcome this, the summarizer module will show you the passages that were used for creating the abstractive summarization. Hope this is a potential solution for your case!

dragosbulugean• 5 years ago

does it work on a typical ES index, or do you have to re-index in a certain way for it to work?

antti9099• 5 years ago

Depends! When using 'DPR' (dense passage retrieval, potentially for more accurate results), a re-indexing is required.

amelius2• 5 years ago

How do they benchmark the quality of their search results?

Jansen312• 5 years ago

If the author is here, please change the name. Haystack is already used in building industry for many years. Search project haystack.

JPKab2• 5 years ago

Not the author, but not a reasonable request to make. It's not like the concept of "finding a needle in a haystack" is novel. There are dozens of projects with the name "haystack".

epwr• 5 years ago

It's a little more complicated. Project Haystack is big. Like the main way building IoT data is tagged around the English speaking world. And it's a semantic vocabulary & ontology, so there's a lot of overlapping search results.

amelius2• 5 years ago

Perhaps name it "needle" then. It also would be more appropriate, since that's the part you're interested in.

anentropic3• 5 years ago

There is also https://django-haystack.readthedocs.io/en/master/ a search library for Python+Django... much closer to the same space

Haystack 1.0 – open-source NLP framework to build NLProc back end applications

Related Stories