rawsh

You can actually get solid performance with pretrained chat models: https://raw.sh/posts/chess_puzzles

On lichess puzzles gpt4o with the compiled prompt is around 70%, I think the 270M transformer is around 95%

Bit confused what the value add is over a framework like DSPy. This still requires you to create an eval dataset with ground truth, basically the only hard part of using DSPy. Easily getting the optimized prompt and having some metrics out of the box is not worth nearly $1k/mo IMO

Side note: I’ve had a lot of luck combining automatic prompt optimization with finetuning. There is definitely some synergy https://raw.sh/posts/chess_puzzles

I built a web version with WASM at https://pdfgrep.com a few years ago in case it’s helpful to anyone

https://github.com/VikParuchuri/marker is solid, but slow and needs gpu(s) to be practical

Is it possible to use this for hybrid search in combination with pg_embedding? My understanding is that hybrid search currently requires syncing with Postgres

Documents actually never get uploaded! PDF text extraction happens on the client using a web worker and MuPDF compiled to WASM.

1. PDF parsed and chunked on the client

2. Sparse vectors are regenerated for the entire document corpus and the existing vectors are updated

3. Dense vectors are generated for the new text and upserted along with the new sparse values

The original documents stay on your device.

Nope, it’s a serious project; I mostly made it for personal use during my last semester of college. I rewrote it a few times and packaged it up because I think it’s genuinely useful. Langchain gets you 80% of the way there but you run into issues with it very quickly.

DankGPT is able to draw context from a library of documents (textbook, papers, class slides) to explain any topic and answer complicated reasoning problems.

It’s very similar to ChatPDF, but you can include multiple documents and it has much better context selection. This leads to better answers in practice (less “the source does not contain information on…” and hallucinations)

https://dankgpt.com