Ask HN: 16 yo Nephew, in E. Africa, wants to train an LLM with on disk Wikipedia

https://news.ycombinator.com/item?id=40151667

by a_w • 2 years ago

14 17 2 years ago

news.ycombinator.com

Hello HN!

My 16 year old nephew lives in an East African nation where there is practically no internet access.

Last week he asked me for advise as to how to go about training an open source LLM using an on disk Wikipedia (~80 GB).

Any suggestions? Thanks!

icsa4• 2 years ago

Use a model already trained on Wiki[edia using llamafile.

You can download llamafile and several models, put them on a USB drive or hard drive, them send the drive to him via DHL.

a_w8• 2 years ago

That is a great suggestion, thank you!

I think he wants to tinker, and learn more about how they work. What I neglected to mention is that he's already learned to program (developing Android apps, and he's also learned Python). He is a very bright and curious kid.

icsa4• 2 years ago

Have him check out:

LLM training in simple, raw C/CUDA

----------------------------------

https://github.com/karpathy/llm.c

It is only 1,000 lines of easy to read C code. There is also Python reference code.

icsa4• 2 years ago

Btw, I support some Kenyan high school students and am looking at supplying a few schools with llamafile+models on flash drives for their computer science curricula.

joegibbs• 2 years ago

What kind of GPUs does he have?

a_w8• 2 years ago

I believe he has a laptop with an Intel i5 with integrated graphics.

runjake• 2 years ago

In addition to the other great suggestions, point him to Karpathy's YouTube channel[1]. Karpathy has an approachable communication style.

Here's his "1 hour intro to LLMs" video: https://www.youtube.com/watch?v=zjkBMFhNj_g

1. https://www.youtube.com/c/AndrejKarpathy

a_w8• 2 years ago

Thanks!

I will try to download it and send it to him.

FrenchDevRemote• 2 years ago

Not an expert, but maybe using RAG/embeddings on the on-disk wikipedia would be better than finetuning on wikipedia?

Most decent LLMs probably were already trained on wikipedia, that doesn't stop them from hallucinating when asked questions about it.

a_w8• 2 years ago

Thanks for the suggestion! I will look into this.

more_corn• 2 years ago

^ This is the way

throwaway11460• 2 years ago

Would it be possible to ship him a Starlink terminal? Internet access could do wonders for a young interested guy like that... And he could share that connectivity with people around him too.

a_w8• 2 years ago

I have been thinking about that, but I haven't gotten around to researching its availability in the country yet.

I will do some research over the weekend. Thanks for mentioning it!

a_w8• 2 years ago

Unfortunately, it doesn't look like Starlink is an option.

https://news.ycombinator.com/item?id=40246021

Ask HN: 16 yo Nephew, in E. Africa, wants to train an LLM with on disk Wikipedia

Related Stories