How to use Alpaca-LoRA to fine-tune a model like ChatGPT

https://replicate.com/blog/fine-tune-alpaca-with-lora

by bfirsh • 3 years ago

173 48 3 years ago

replicate.com

camdenlock• 3 years ago

The weights for LLaMA have not yet been released publicly. To apply for access, fill out this Meta Research form.

Cute. ;)

k_eshav• 3 years ago

iirc someone posted the weights to a torrent. you can look it up. :)

isoprophlex2• 3 years ago

I grabbed it shortly after the leak and left it on for a while. Never seeded that much data on a single torrent. This is some screaming hot data.

isoprophlex2• 3 years ago

Low-rank adaptation (LoRA) ... has some advantages over previous methods:

- It is faster and uses less memory, which means it can run on consumer hardware.

- The output is much smaller (megabytes, not gigabytes).

- You can combine multiple fine-tuned models together at runtime.

This is great news for my dream of building a fine-tuned interactive messenger, that can deliver a message on my behalf by training it on my personality & the information I want to convey.

Now just add text to speech and a talking head, as discussed in that other submission about cloning yourself with AI... https://news.ycombinator.com/item?id=35280418

cal5k• 3 years ago

What if you died but your chatbot didn't know?

BoorishBears2• 3 years ago

This is an idea I've been mulling over.

Maybe software running on your PC to capture everything you type, a voice transcriber that filters out your voice specifically and records that, and you've got a dataset that covers a lot of who you are.

Fine tune a model on that and boom, you're "immortal", and as LLMs get better and better, the fidelity of "you" gets better and better.

zer00eyz• 3 years ago

Something like this is a theme in the book Idoru by William Gibson. What happens to the data of a dead person is one theme. Marriage to an AI another.

TehCorwiz• 3 years ago

I'm reminded of this scene from the "I, Robot" movie: https://www.youtube.com/watch?v=ZKxr0wyIic4

elorant• 3 years ago

And then you hook it up to hundreds of dating apps and it just does the boring job of making introductory chat presenting you at the end with only the women who are interested in a real date.

speedgoose2• 3 years ago

The problem of this strategy is that your dates may realise that you are not talking as well as a good dating LLM once they meet you.

nl3• 3 years ago

And then someone uses it to create lots of fake people to populate the app so real people buy more subscriptions.

gravitronic• 3 years ago

My friend was doing this with mIRC scripts on our city's DALnet room in 2001. His bot hit up random users, asked a/s/l, and if they were female would text to speech "BabyGurl18 is female".

So we'd be watching a movie, he'd have his computer speakers turned up in his room, his bot would announce and he'd get up and leave, already in conversation

stavros• 3 years ago

When you consider what will happen when everyone on dating apps is an LLM, you see the issue: in the end, everyone is "interested" in everyone who swiped right.

rishsriv• 3 years ago

This looks fantastic. Will try replacing our current fine-tuned FLAN-UL2 model with this.

I wonder how the devtooling around this will evolve. Seems like a matter of days until someone creates a GUI wrapper around this, and obviates the need to use programmer time for fine-tuning

anymoonus• 3 years ago

I'm curious, what are the differences between T5, Flan-T5, and Flan-UL2 for fine-tuning? Does the instruction tuning matter at all, once you're fine-tuning?

tysam_and• 3 years ago

LoRA has actually been around for a little while! I first saw it when it became popular in fine-tuning models quantized down to about 8 bits or so. I'm sure it's doing stuff in the 4bit range now! :D

I believe it's a core toolbox piece of tech required to really push the limits of LLMs either in original training or in inference. Similar sort of to how batch norm was for convolutional neural networks. I look forward to seeing how this will be applied in the future.

braingenious• 3 years ago

I love these idea of LoRAs for LLMs.

Has anybody made a llama/alpaca erebus model? I read about them in the oobabooga docs and a locally-run language model fine tuned on literotica could be the funniest thing I’ve ever seen.

credit_guy2• 3 years ago

I guess this LoRA is the missing piece.

NVIDIA stated recently that GPT bots will become one million times more powerful in ten years. Many people doubted that.

With LoRA, I see a much higher improvement. These guys claim a 10000 times reduction in parameter size. A different way to look at it, is that with the current hardware you can train a model that has 10000 times more parameters. If you add a 100x improvement in hardware in 10 years (not at all unrealistic), that's the million. But we will have significant improvements in training methods too.

flangola73• 3 years ago

Where do you find 10,000 more data?

nathanasmith• 3 years ago

Audio, video, and LLMs enabled for real world interaction will pave part of the way for sure.

credit_guy2• 3 years ago

You don't need 10,000 more data to train a model with 10,000 more parameters. You can use the same data and the model will perform better. Much better. The problem is that currently it is nearly impossible to run and train the model with more parameters.

ChatGPT has nearly 200 billion parameters. GPT-4 we don't know, there are rumors that it has 100 trillion parameters, but they are probably unfounded. In any case, we've seen how much more powerful GPT-4 is. Imagine a GPT-5 with 1 quadrillion parameters.

And then imagine that after you've trained it to some reasonable level, you "downsample" the parameters, using the SVD approach described in LoRA, and get a GPT-5 with the same 200 billion parameters as ChatGPT, but with many, many times more power, even than GPT-4.

slicktux• 3 years ago

Anyone else click on this thinking it was about the wireless protocol?

y3sar• 3 years ago

Both are fascinating tech

nico2• 3 years ago

Can a model be fine-tuned “online”?

If cost wasn’t an issue, could I fine-tune a model in real time, while also using it for inference?

joshka• 3 years ago

Stats from https://replicate.com/blog/replicate-alpaca suggest 90 minutes on 4 x A100/80GB. This is 3.5 hours on A100/40GB - I'm guessing LoRA is probably parallizable.

all22• 3 years ago

Stats from TFA say 3 hours to fine tune on an A100 processor.

all22• 3 years ago

I wonder if one could "fine tune" specific layer values... That might be faster than updating every weight in every layer.

nl3• 3 years ago

Yes and no.

The training process is modifying the network weights. These are usually written to copies of the file instead of overwriting it (because what if the loss is actually worse after an epoch of training?)

But there's nothing stopping inference from occurring on a model that is being trained.

eachro• 3 years ago

How does LoRA save more than 50% of the memory usage? I see that the weight updates have much lower memory footprint by virtue if being low rank. But you still need the dense weights for the forward pass dont you?

leereeves• 3 years ago

I'm not an expert, but I believe it only saves memory in the final model, after training is done, by merging the low rank LoRA wrapper matrices with the original weight matrices.

For example, if an original layer has N inputs and outputs (an NxN weight matrix) LoRa adds a 16xN matrix before it and an Nx16 matrix after it, trains only those new matrices, and finally multiplies all three matrices to get a single 16x16 matrix.

rcarmo• 3 years ago

So they use cog before installing it? Apparently this wasn’t proofread.

Also, is it just me or there are currently more ways to run LLMs on a CPU than on a GPU springing up on GitHub? I have hacked my own, but my chat UI is awful, so what is the nicest, pre-packaged CUDA-friendly way to run this now?

syntaxing• 3 years ago

The easiest way to run alpaca Lora locally is with this little known fork [1] that uses Docker. You’ll be up and running in about 20 min with pretty much any modern consumer Nvidia GPU.

[1] https://github.com/chris-alexiuk/alpaca-lora

techn00• 3 years ago

It feels like I'm living in a cartoon with all these terms: > In this blog post, we’ll show you how to use LoRA to fine-tune LLaMA using Alpaca training data.

mnreef• 3 years ago

Hi All, I have a noob question. I have been reading about Alpaca and Alpaca Lora. I have a use case in which I want to fine tune/train Alpaca Lora on a large corpus of books which are in the txt format. I know for Alpaca, the data was in "Instruction : Prompt" format. however, my text is huge and is not in that format. It's simply a library of books and journal articles. I want to be able to ask a question and the model answers based on the books I trained it on. I also want to be able to ask general questions for example which books discussed topic x or y.

I have tried OpenAI's API to create embeddings, but I want to use Alpaca.

I really appreciate your help.

How to use Alpaca-LoRA to fine-tune a model like ChatGPT

Related Stories