On demand vs. Reserved instances for LLM fine-tuning

Hi everyone,

I am looking at different options to get access to GPUs to train an LLM on an enterprise use case for a customer. They are on a specific Cloud provider with associated credits, so I have no ability to go outside, like Runpod.

We are exchanging on whether we should go for On-demand or Reserved instances. This client is a Fortune 500, so they could reserve it but it might not be the best choice.

It seems to me that with LoRA / QLoRA, and so on, we might be able to fine-tune a Llama 2 with one or two GPUs on-demand, but I am unsure yet.

So our main evaluation criteria are:

- Price

- Availability of GPUs, aka we don't want to waste too much time to get started

In your experience, have you had difficulties to get access to GPUs at a good price? How many of you had to go with reserved instances, and if so what made you choose this option?

I would love to have your feedback!