dvorka

Enjoying the work on MyTraL - sovereign athlete / personal training log: https://mytral.fitness/

I went on sabbatical to fulfill my dream project - consolidating years of training logs. I'm enjoying the technical challenges involved - digitizing paper hand written logs / visual models, navigating the maze of athletic metrics with their crazy trademarked names and multidimensional models. Having fun building AI coaches: agents ranging in character from Al Pacino in Any Given Sunday to the coach from my teenage years, utilizing ICL model-based predictions, ... and more.

The best part is the rush of memories while ingesting my own history - photos and recordings I completely forgot, as well as navigating data shared by friends.

This month has Strava & LeCol everesting challenge - signed up and added support that suits my needs to MyTraL. Good times.

I needed exactly this for years excellent work!

Working on MyTraL - sovereign athlete / personal training log: https://mytral.fitness/

I went on sabbatical to fulfill my dream project - consolidating 30 years of training logs that span everything from paper and Excel spreadsheets to various fitness services and devices I used. I'm enjoying the technical challenges involved - digitizing paper hand written logs using OCR / visual generative models, navigating the maze of athletic metrics with their crazy trademarked names and SOTA multidimensional models. Having incredible fun building AI coaches: agents ranging in character from Al Pacino in Any Given Sunday to the coach from my teenage years, utilizing ICL / PFN model-based predictions, ... and more.

The best part is the rush of memories while ingesting my own history - photos and recordings I completely forgot, as well as navigating data shared by friends - records they didn't see in years because the original applications they used no longer exist or won't run on their current HW.

Any suggestion what to use as embeddings model runtime and semantic search in C++?

It depends on vendor really - I have Lenovo T480 and I replaced keyboard earlier this year (there are various options like w/ or w/o backlit + layout (I'm Czech), I have 2 batteries - one for "normal" use and extended one (in size and capacity) for traveling, changing multiple SSDs and RAM is possible (not soldered)... it's not framework, but easily fixable and Linux friendly HW.

This is so true! But the adventure doesn't end there. I have 2 billing accounts from the past when I was building projects on AppEngine. Annual exercise to keep them alive (even if no action is needed in the end) is of similar complexity. Why do I need these accounts? Because I want to use Google services for which I don't pay.

"In the good old days, it was a good practice to run a new protocol proposal through some standards bodies like W3C or OASIS, which was mostly a useful exercise. Is the world somewhere else already, or would it be a waste of time?"

Rear is a really interesting project with admirable goals. I believe this is just the beginning, but you have already done a great job!

I have been working on my note-taking application (https://github.com/dvorka/mindforger) for some time and wanted to go in the same direction. However, I gave up (for now). I used ggerganov/llama.cpp to host LLM models locally on a CPU-only machine with 32GB RAM, and used them for both RAG and note-taking use cases (like https://www.mindforger.com/index-200.html#llm). However, it did not work well for me - the performance was poor (high hardware utilization, long response times, failures, and crashes) and the actual responses were rarely useful (off-topic and impractical responses, hallucinations). I tried llama-2 7B with 4b quantization and a couple of similar models. Although I'm not happy about it, I switched to an online commercial LLM because it performs really well in terms of response quality, speed, and affordability. I frequently use the integrated LLM in my note-taking app as it can be used for many things.

Anyway, Reor "only" uses the locally hosted LLM in the generation phase of the RAG, which is a nicely constraint use case. I believe that a really lightweight LLM - I'm thinking about a tiny base model fine-tuned for summarization - could be the way to go (fast, non-hallucinating). I'm really curious to know if you have any suggestions or if you will have any in the future!

As for the vector DB, considering the resource-related problems I mentioned earlier, I was thinking about something similar to facebookresearch/faiss, which, unlike LanceDB, is not a fully-fledged vector DB. Have you made any experiments with similarity search projects or vector DBs? I would be interested in the trade-offs similar to small/large/hosted LLMs.

Overall, I think that both RAG with my personal notes as a corpus and a locally hosted generic purpose LLM for the use cases I mentioned above can take personal note-taking apps to a new level. This is the way! ;)

Good luck with your project!

@bewuethr exactly! BTW there is even easier way how to configure HSTR:

    hstr --show-configuration >> ~/.inputrc && . ~/.inputrc