AMICABoard

Video: https://x.com/VulcanIgnis/status/2061198364889526711

Nice work bro!

I did something similar ie llama2.c in a PDF in a PNG before:

https://news.ycombinator.com/item?id=42721805

https://x.com/VulcanIgnis/status/1879649889178837025

The non polyglot ie PDF only versions could run both in chrome and firefox.

I'll have a look at your github, you seem to have done it more elegantly.

lol :)

You can, it's a meme, I stole it from Karpathy :)

It had to be done :) If not us, who else would?

Damn Good stuff, bot the doom and this linux of your! Kudos.

I had done LLM in PDF sometime Nov last year but it wasn't released. Since the doom in pdf uptick, I had released a demo here:

https://x.com/VulcanIgnis/status/1879649889178837025

Actually you don't need an old version of emscripten, hit me up for the Makefile and pdf templates or wait till offical release. My build script will make the pdf work both on chrome and firefox, adobe support is pending.

One super upvote!I used used this cray ack to some demo code. I really admire kej715 for adding a backend to ack just from reading specification and limited access to hardware as there are no loving cray 1's anymore. I had to try it! Did't build at first so I even contributed a tiny fix to make it work on my sys.

This is also the guy who wrote a DtCyber that emulates many vintage super computers and stuff. Pretty cool! Thank you man!

Nice progressive port by Nathan with networking and clustering

- OS / Kickstart: 1.x (1985) + - CPU: Motorola 68EC020 (1984) + - RAM: Less than 1.5 MB

adf on request, still in alpha

This is also a PNG

See: https://x.com/VulcanIgnis/status/1879649889178837025

L2E Llama2.c in a PDF in a Shroedinger PNG

Visit: https://github.com/trholding/llama2.c

Download the header image from my repo, which is a Polyglot PNG (both a PNG and a PDF). Rename the extension from .png to .pdf and open it in Firefox to run llama2.c inside the PDF! Note that this doesn't work in Chrome.

The PDF contains a version of Karpathy's llama2.c running the tiny 260k model. For more details, check: https://twitter.com/VulcanIgnis/status/1879649889178837025.Y... can find the image here: https://github.com/trholding/llama2.c/blob/master/assets/l2e....

Pure PDF versions of the smaller and smol models are compatible with both Chrome and Firefox, but Adobe Acrobat is not yet supported. I created the PDF part back in November, planning to turn it into a self-regenerating comic demo and add Acrobat support. If Adobe doesn't update their JS engine or improve documentation, I might fork my own reader with WASM for better performance and AI capabilities in PDFs.

Emscripten was used to compile this to something between ASM.JS and JS. Enjoy experimenting with this, though remember it's a tiny 260k model, not super intelligent.

I am passionate about PDFs and frustrated with Adobe's JS engine, which isn't true JS 1.3. This project represents running LLMs inside documents, turning documents into LLM OS. If Adobe doesn't fix JS in PDF, I'm motivated to enhance PDF capabilities myself. This is not the thermodynamic god, just pee/acc where we accelerate peedf :)

Okay its a tiny 260k model that is running in it. I had made the pdf part back in in Nov, I thought I'd share since stuff inside PDF is a trend. I'll upload the pure PDF larger and small models which work both on chrome and firefox pdf readers.

I am pretty angry at Adobe as their JS Engine is not JS 1.3 as claimed. It is something in between... So we invented LLM inside Documents. And Documents becoming LLM OS. So Adobe, if you are not going to fix JS in PDF, I'll fork my own reader with WASM and all the bells and whistles!

This is not the thermodynamic god, just pee/acc where we accelerate peedf :)

Happy to answer questions!

Making Steve Job's vision come true ~40 years later: https://www.youtube.com/watch?v=2qLuerYx2IA

Thank you :) The open source community is great, the core is a confluence of the awesome work of others. I'm just a orchestrator. I wish I had more time to make it into something like llama.cpp and an OS around it.

Hmmm interesting take.

Likely, cos other langs too. The fault may be my incomplete implementation.

My bad, I directly linked to the C file instead of the project here:

It is a program that given a model file, tokenizer file and a prompt, it continues to generate text.

To get it to work, you need to clone and build this: https://github.com/trholding/llama2.c

So the steps are like this:

First you'll need to obtain approval from Meta to download llama3 models on hugging face.

Go to https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct, fill the form and then go to https://huggingface.co/settings/gated-repos see acceptance status. Once accepted, do the following to download model, export and run.

huggingface-cli download meta-llama/Meta-Llama-3.1-8B-Instruct --include "original/*" --local-dir Meta-Llama-3.1-8B-Instruct

git clone https://github.com/trholding/llama2.c.git

cd llama2.c/

# Export Quantized 8bit

python3 export.py ../llama3.1_8b_instruct_q8.bin --version 2 --meta-llama ../Meta-Llama-3.1-8B-Instruct/original/

# Fastest Quantized Inference build

make runq_cc_openmp

# Test Llama 3.1 inference, it should generate sensible text

./run ../llama3.1_8b_instruct_q8.bin -z tokenizer_l3.bin -l 3 -i " My cat"

Will do! Thanks :)

:)

Hmm yeah, it started as fork of karpathy's llama2.c and some experiments. So it is an abomination I agree.

Thanks, I'll read it up. Interesting.

Actually "My cat is funny" was the prompt it continued that. I got to fix some stuff to reflect meta's implementation and also fix the chat mode, then it would be usable. Will take a few days to do that.

Interesting, I heard something like that, but now I must read about it.

Lol :)

To be honest your pr and these notes are super helpful cos, otherwise I'd have been too lazy to read up the original implementation, but I can't merge it. Will make the fix soonish and credit it to you. I'll tell you a secret, nobody is good at C except for a few wizards like woz, Linus, jart, Knuth, Bellard and the like. I am also a total looser at C. I just love C. And also the base of this is mostly Karpathy's work and other awesome folks who are way better at everything than me.

Llama.cpp is the king, this is just a lowly wanna be peasant. But some day it will reach there.

I think the non English part is mostly hit and miss in this primitive version, probably cos the implementation is not correct. I got to read up a lot and fix it.

Maybe we should make "Brain Damage Factor" a official term to denote how much types of quantizations degrade output compared to unquantized.:)

Ja Amiga 500! My first computer. Still in love with her...:)