ocolegro

Hey HN,

We've just released R2R V3 with a completely RESTful API that covers everything you need for production RAG applications. The biggest change is our Git-like knowledge graph architecture, but we've also unified all the core objects you need to build real applications.

Check our [API](https://r2r-docs.sciphi.ai/api-and-sdks/introduction) or join our [Discord](https://discord.gg/p6KqD2kjtB) if you want to dive deeper. We'd love feedback from folks building in production!

We've been using hatchet for cloud deployments and have really enjoyed the reliable execution / observability, congrats on the launch.

new docs are out if anyone was still wanting that, thanks.

Thanks I agree that would be a more streamlined introduction.

The quickstart clearly has too much content in retrospect, and the feedback here makes it clear we should simplify.

thanks, this is really solid feedback - we will make a more inclusive docker image to make the setup easier/faster.

Think of R2R as an SDK with an out of the box admin dashboard / playground that you can plug into.

yes, I think so.

Awesome - interested to hear your thoughts / feelings after you get a chance to try it out.

its a optional dep used for kgs

I'm just seeing this now.

The key advantages can be extracted from the response above to Kluless -

R2R is built around RESTful API and is dockerized, so devs can get started on app development immediately.

The system was designed so that devs can typically scale data ingestion up to provider bottlenecks w/out extra work.

We have implemented user-level permissions and high level document management alongside the vector db, which most devs need to build in a production setting, along with the API and data ingestion scaling.

Lastly, we also log every search and RAG completion that flows through the system. This is really important to find weaknesses and tune the system over time. Most devs end up needing an observability solution for their RAG.

All of these connect to an open source developer dashboard that allows you to see uploaded files, test different configs, etc.

These basic features mean that devs can spend more time on iterating / customizing their application specific features like custom data ingestion, hybrid search and advanced RAG.

No we don't have any explicit code graph tools. Sourcegraph might be a good starting point for you, their SCIP indices are pretty nice

great question, I can talk about how we do the more challenging "List all YC founders that worked at Google and now have an AI startup."

For this we have a target dataset (the YC co directory) that we have around 100 questions over. We have found that when feeding an entire company listing in along with a single question we can get an accurate single answer (needle in haystack problem).

So to build our evaluation dataset we feed each question with each sample into the cheapest LLM we can find that reliably handles the job. We then aggregate the results.

This is not perfect but it allows us to have a way to benchmark our knowledge graph construction and querying strategy so that we can tune the system ourselves.

See the guide here - https://r2r-docs.sciphi.ai/cookbooks/local-rag

we have instructions for getting setup and running w/ ollama. It should be pretty smooth.

I can't answer for the kindly poster above (ty), but from our experience techniques like HyDE are great when you are getting a lot of comparative questions.

For instance, if a user asks "How does A compare to B" then the query expansion element of HyDE is incredibly useful. The actual value of translating queries into answers for embedding is a bit unclear, since most embedding models we are using have been ft'ed to map queries onto answers.

Is this an example that could benefit from something like knowledge graph construction or structured entity extraction?

I'm just curious because we have theorized and seen in practice that extraction is a way to answer questions which require connected information across disparate chunks, like you can see in the simple cookbook here [https://r2r-docs.sciphi.ai/cookbooks/knowledge-graph].

Or do you think this is something that can just be solved with more advanced multimodal ingestion?

Currently R2R has out of the box logic for the following:

csv, docx, html, json, md, pdf, pptx, txt, xlsx, gif, jpg, png, svg, mp3, mp4.

There are a lot of good questions around ingestion today, so we will likely figure out how to intelligently expand this.

For mp3s we use whisper to transcribe, for videos we transcribe with whisper and sample frames to "describe" with a multimodal model. For images we again transcribe to a thorough text description - https://r2r-docs.sciphi.ai/cookbooks/multimodal

We have been testing multi-modal embedding models and open source models to do the description generation. If anyone has suggestions on SOTA techniques that work well at scale we would love to chat and work to implement these. Long run we'd like the system to be able to handle multi-modal data locally.

Have you tried it out yet, how does it compare with gpt-4o?

No not yet, I've had difficulty getting these different providers to work together on integrations. If you have any suggestions we are all ears.

In the meantime we've built our own dashboard which shows ingested documents, and has a customizeable chat interface - https://github.com/SciPhi-AI/R2R-Dashboard.

It's still a bit rough though.

Yes, there is a guide to running R2R with local models here - https://r2r-docs.sciphi.ai/cookbooks/local-rag

We agree that ingestion and extraction are a big part of the problem for building high quality RAG.

We've talked to a lot of different developers about these problems and haven't found a general consensus on what features are needed, so we are still evaluating advanced approaches.

For now our implementation is more general and designed to work across a variety of documents. R2R was designed to be very easy to override with your own custom parsing logic for these reasons.

Lastly, we have been focusing a lot of our effort on knowledge graphs since they provide an alternative way to enhance RAG systems. We are training our own model for triples extraction that will combine with the automatic knowledge graph construction in R2R. We are planning to release this in the coming weeks and are currently looking for beta testers [we made a signup form, here - https://forms.gle/g9x3pLpqx2kCPmcg6 for anyone interested]

Take a look here - https://r2r-docs.sciphi.ai/quickstart#ingest-data and here https://r2r-docs.sciphi.ai/cookbooks/client-server#ingest-do...

Since multiple people have requested we are pushing a quick change to make this emphasized in the docs.

You are right that the quickstart is pretty large, we will think about how we can trim that and show only the essentials.

To do what you are requesting is pretty easy, you can just launch the server and use the client directly. The code would look like this:

```python

from r2r import R2RClient

base_url = "http://localhost:8000" # or other

client = R2RClient(base_url)

# load my_file_paths

...

response = client.ingest_files(file_paths=my_file_paths)

# optionally set metadata, document ids, etc.. [https://r2r-docs.sciphi.ai/api-reference/endpoint/ingest_fil...]

```

All the pipelines are async, so for ingestion we have typically seen that R2R can saturate the vector db or embedding provider. We don't yet have backpressure so it is up to the client to rate limit.

Ingestion is pretty straightforward, you can call R2R directly or use the client-server interface to pass the html files in directly to the ingest_files endpoint (https://r2r-docs.sciphi.ai/api-reference/endpoint/ingest_fil...).

The data parsers are all fairly simple and easy to customize. Right now we use bs4 for handling HTML but have been considering other approaches.

What specific features around ingestion have you found lacking?

Sure, happy to.

R2R is built around RESTful API and is dockerized, so devs can get started on app development immediately.

The system was designed so that devs can typically scale data ingestion up to provider bottlenecks w/out extra work.

We have implemented user-level permissions and high level document management alongside the vector db, which most devs need to build in a production setting, along with the API and data ingestion scaling.

Lastly, we also log every search and RAG completion that flows through the system. This is really important to find weaknesses and tune the system over time. Most devs end up needing an observability solution for their RAG.

All of these connect to an open source developer dashboard that allows you to see uploaded files, test different configs, etc.

These basic features mean that devs can spend more time on iterating / customizing their application specific features like custom data ingestion, hybrid search and advanced RAG.

So when is the crunchyroll integration rolling out?

Yes, the framework is designed to directly deploy a FastAPI application with an associated Python client.

You can see the client here - [https://github.com/SciPhi-AI/R2R/blob/main/r2r/client/base.p...].

Yes, this is on the shortlist.

Do you have any preferred frameworks?

awesome! Please take a look and let me know what you think.

the only time we've found pgvector to be preferable to qdrant is when the per-category embeddings are very few. E.g. in cases where filtering is expected to reduce the dataset greatly for each query (>99.9%).

This is when the relational nature of postgres really shines - you can even do in-mem calculations for distance.

This is AMAZING feedback and it is on brand with what I've heard from a number of builders. Thanks for sharing your experiences here.

The infra challenges are real - it has been what I have been struggling the most with in providing high quality support for early users. Most want to be able to reliably firehose 10-100s of GBs of data through a brittle multistep pipeline. This was something I struggled with when building AgentSearch [https://huggingface.co/datasets/SciPhi/AgentSearch-V1] with LOCAL data - so introducing the networking component only makes things that much harder.

I think we have a lot of work to do to robustly solve this problem, but I'm confident that there is an opportunity to build a framework that results in net positives for the developer.

FWIW, Your feedback would be invaluable as the project continues to grow.

This is an open source solution that is meant to offer similar capability / ease of use, but with transparency & flexibility for the developer.