Long Live the 'GPU Poor' – Open-Source AI Grants

https://a16z.com/2023/08/30/supporting-the-open-source-ai-community/

by rajko_rad • 3 years ago

326 110 3 years ago

Very happy to see oobabooga on the list. Everyone's contributing in different ways, and ooba serves as a fantastic portal to try out various models which I appreciate. Its readme even says it wants to be like the automatic 1111 of text generation. For beginners like me who are curious about this new world, ooba is a great entry point.

rajko_rad17• 3 years ago

We agree!!

appenz2• 3 years ago

We 100% agree!

satvikpendem• 3 years ago

Well, at least this time a16z is giving money to actually useful technologies instead of bloviating about the world changing potential of cryptocurrencies. I imagine they've written down most of their cryptocurrency investments at this point.

That being said, these are some great grants for local LLM builders, local computation for such powerful tools is needed, and will continue to be needed, as industry incumbents like OpenAI continue to not release their models.

espadrine• 3 years ago

I had become honestly cynical of their actions in the past, as they felt hype-driven and externalities-blind. Yet I don’t see how this grant could become rapacious. Hopefully it is an attempt at cleaning the brand which can work as making amends.

zitterbewegung• 3 years ago

Don't VCs basically chase everything because they only rely on 10% or less to succeed but, that 10% needs to have at least 10x or more profit and then they do another round?

rajko_rad17• 3 years ago

Hi HN, we are super excited to announce this new initiative!!

Please note, this program was designed to support individuals, teams, and hackers who are not pursuing commercial companies. Nevertheless, these projects push the state of the art in open source AI and help provide us with a more robust and comprehensive understanding of the technology as it is developed.

We are really proud to be contributing in this small fashion and grateful to the first cohort and all others contributing in this space!

latchkey• 3 years ago

I'm curious, where are you getting the GPUs?

rajko_rad17• 3 years ago

in most cases the teams already had their own source or platform of choice!

qwertox• 3 years ago

Thank you!

rajko_rad17• 3 years ago

Our pleasure :)

thewataccount2• 3 years ago

Hello, I think this is a great project!

If an individual is looking to contribute to the field with different training data ideas - would they need to first establish themselves and get your attention, or would there be a way to submit a proposal?

For myself compute is rather expensive, I could likely afford a few test runs for proofs of concept but beyond that it would be difficult. I've got a single 4090 so I can't run llama70b faster then 1it/s.

rajko_rad17• 3 years ago

Thank you so much for your work and enthusiasm here!! Unfortunately, that’s roughly the best way to approach it and how most of these have worked so far, demonstrations on smaller models (e.g. 7/13B), with slightly smaller datasets, catching the eye of the community, etc. in general it’s not a bad approach to research too, prove out concepts at smaller scale before scaling up!

Michelangelo11• 3 years ago

This is great, but I'm not seeing anything about how to apply, or really anything about the application and selection process at all. Is this by design?

zekrioca2• 3 years ago

It seems so.

tikkun4• 3 years ago

My guess is that they'll prefer to get nominations.

Having once done a grant application process - I think their approach is smart.

If they opened grant applications, they'd get thousands of (sadly mostly very low quality) applications. It's extremely hard to vet.

It's a much safer bet to wait until projects are already popular in the community, already being relied on by people, and where those devs would love to spend more time on their open source projects but they're limited by money.

Yes, the downside of this is that some people could make things happen if they were given money first. But, if I was them, I'd still prefer to do what they're doing and pick projects that are already successful and give grants to those.

(This isn't a comment about your project specifically, for all I know your project might be a popular open source LLM project that lots of people already rely on, in which case they probably already know about your project and will probably reach out to you for the second round, is my guess)

Havoc• 3 years ago

Amazing. Those are really solid 1st picks. A huge chunk of the organic experimentation relies directly on them.

rajko_rad17• 3 years ago

thanks so much - we agree!

minimaxir• 3 years ago

It's interesting to note that there's particular emphasis on "removing censorship from LLM outputs" given a16z's investments outside the AI space in some ideological social media companies.

skilled• 3 years ago

We believe artificial intelligence has the power to save the world—and that a thriving open source ecosystem is essential to building this future.

Thank you for understanding the fundamental flaws of the World and human consciousness, and thank you for spending multiple lifetimes as Buddhist monks (not to mention lamas) to understand the intricacies of the human mind from a scientific perspective. This is exactly what we need right now and this initiative looks to be the answer to all of our problems.

waihtis• 3 years ago

hail e/acc and the thermodynamic god

svnt5• 3 years ago

“We are disguising this round of seed funding as a grant, accomplishing all of the below:

1) brand-washing for people who don’t do the math

2) avoiding angering our LPs and/or violating our fund theses and

3) getting in on the ground floor of things that might follow OpenAI through the non-profit-to-VC-darling door

Thanks for reading our press release, feel free to try to apply using your personal network (somehow) because we don’t care as we’ve already landed our positions.”

notatoad• 3 years ago

unless i'm missing something, they're not taking equity. that's what makes it a grant, and not seed funding.

what math should i be doing here?

svnt5• 3 years ago

Probably the grants have clauses that convert them to debt or equity in case of a transition of the company. [0]

The bigger math is significance_of_contribution = $size_ai_grant_program/$size_ai_vc_program (or maybe $size_ai_grant_program - $size_ai_vc_program)

[ 0 https://link.springer.com/chapter/10.1007/978-3-030-72428-3_... ]

evanwolf2• 3 years ago

This is insurance for their already huge investments (and future rounds) in AI businesses. It's a pittance relative to everything else they have at risk. It's prudent since failure of small projects like these can delay work on profitable corporate efforts.

svnt5• 3 years ago

This is a good point I hadn’t considered.

wslh2• 3 years ago

We work in source code security auditing with semi-automated tools using AI and other techniques.

Is this approach covered in the grants?

rajko_rad17• 3 years ago

if the work is OSS/public and you aren't trying to start a VC-backed company, it could be! We usually want to see some early work already published, community reception and validation, but feel free to DM Matt or myself and we can take a look!

wslh2• 3 years ago

if the work is OSS/public and you aren't trying to start a VC-backed company, it could be!

Yes, it is a company that often applies and receives grants. For example, we are working on adding AI to security tools such as https://github.com/CoinFabrik/scout

redox993• 3 years ago

I want to congratulate you based on your current grant recipients. They are all very impactful individuals/teams. It shows that you clearly did your research and put the money in the right places.

rajko_rad17• 3 years ago

thank you!! Indeed, we have been interacting in space very closely and doing our best to keep up and contribute where possible

malux85• 3 years ago

Is there a way we can apply? I am building https://atomictessellator.com, and since I am 100% technical I am not sure about commercialising or open sourcing it, I definitly would like to open source some of the components (e.g. turnkey dockerized scientific computing environments, the molecular force viewer, and also open up a hub for the environments and models) - when I was in Stanford meeting with some comp chemists a few months ago they got very excited for this.

I negotiated with my day job to work 2AM to 10AM so that I can work on this passion project in the afternoons, some grant funding would really help me focus on this more. There's quite a bit of interesting stuff not on the website in catalyst discovery and molecular dynamics. Contact me if you'd like to chat / demo.

In my ideal world I would work on this full time and totally open source it.

intalentive• 3 years ago

TheBloke has had this sponsorship in his byline for a little while now.

UncleOxidant• 3 years ago

"We believe artificial intelligence has the power to save the world"

I don't agree with the side that says AI is about to destroy everything, it seems very hyperbolic... but neither do I agree with this sentiment that it's going to save the world.

hgsgm• 3 years ago

I don't even understand. A16Z's crypto already saved the world. Why is the world in trouble again?

geodel• 3 years ago

It's to make it even more savier. Think if someone could combine LLAMA with blockchain and create Lockchain. This is kind of saviness we all need.

andrewprock• 3 years ago

Agree. It's not at all obvious the world is in need of saving.

That said, software of all kinds (including machine learning) does make the world more interesting and exciting, so happy to see more investment.

jxf• 3 years ago

Q: What are the grant amounts?

mirekrusin• 3 years ago

They should call this initiative Open AI.

capableweb3• 3 years ago

LibreAI for extra fun :)

Kon-Peki• 3 years ago

I'm glad you guys are sharing some money around, but I really have a hard time thinking of a person with "Dual 4090s" as "GPU Poor". LOL

capableweb3• 3 years ago

It's all relative. Compared to the huge companies spending thousands per month/week/day on GPU usage, having two 4090s (which you could acquire for like ~$3000) could be considered on the lower end for sure.

jacquesm2• 3 years ago

Depending on where you live what you pay for power may well be a bigger factor than what you've paid for the hardware.

qntty• 3 years ago

I interpret "GPU poor" the same way as "house poor", i.e. you're poor (lack cash) because you've use it to purchase GPUs, not that you don't have enough GPUs.

KaoruAoiShiho• 3 years ago

No, GPU poor comes from this definition: https://www.businessinsider.com/gpu-rich-vs-gpu-poor-tech-co...

Less than 10k H100s makes you GPU poor. The author is presumably using it as a proxy for "possible to achieve AGI".

ForHackernews• 3 years ago

It's like how you're house poor if you only have one brownstone in Brooklyn and a pied-à-terre in Palo Alto.

htrp• 3 years ago

ehartford and thebloke are definitely great picks advancing SOTA in underserved fields.

Curious why there are no image ai projects receiving grants here?

appenz2• 3 years ago

No deeper reason. I think there just a lot in LLMs happening right now which skewed it towards them. We would love to do something in the SD ecosystem.

rajko_rad17• 3 years ago

Great callout @htrp... @appenz is our expert here btw !

retrocryptid• 3 years ago

I'm still not sure what I'm supposed to be using AI for. I already know how to code, tried for a decade to make ACR (auto content recognition) work with CNNs, and as best I can tell, chatGPT can write a news article for you if you write it first and then hire someone else to edit it.

artninja1988• 3 years ago

Thank you for doing this. Although I'm sure it's not purely for altruistic reasons, open source ai is important:)

thinktown• 3 years ago

What was actually being offered here? Money? How much? This seems like such vague VC-marketing speak.

bodecker4• 3 years ago

Congrats to the folks involved.

Even knowing this is partly motivated by branding/marketing, it's great to see a16z getting more aligned with solving real pain points (vs crypto and churning out shallow media in recent years). Hope they can keep it up and hopefully more "thought leaders"/VCs follow suit. Best of luck.

jacquesm2• 3 years ago

No, they're still saving the world, only now not through crypto but through AI...

https://a16z.com/2023/06/06/ai-will-save-the-world/

skeeter2020• 3 years ago

this just seems like a cost-effective attempt to buy credibility and stay relevant, but I'm a crusty, old curmudgeon so what do I know.

bodecker4• 3 years ago

Of course - there will always be investors/influencers pushing narratives to hype their bets. IMO everyone is better off when the hype is grounded in solving real pain points, which the AI projects I've seen seem to be closer to than most of the crypto projects I've seen.

Certainly there are plenty of grifters in AI too (as with any gold rush) and many AI efforts will fizzle out. But it seems there is more real value being created here than in crypto, which is the main thing I'm excited about and hope to see more of

calderwoodra2• 3 years ago

I agree this initiative helps solve real pain points, and I'm not trying to defend "Crypto" with my question here but isn't solving a "real pain" kinda subjective? For example, many people/companies have the real pain/need to improve ad click-through-rates and reduce crypto/gas fees but maybe these aren't problems you care about.

samstave3• 3 years ago

Solving pain point == A problem we can profit from. Always has been.

Plus A16Z has some questionable motives in general based on previous comments by them...

bodecker4• 3 years ago

Agreed, "real pain" is a subjective/vague term and even crypto is a broad term. My intended use here is to describe solutions that are more grounded/integrated into reality vs. more grounded in a fantasy of how the world works. It is hard to describe well though and if you keep unrolling most perceptions of reality are subjective

ricardo81• 3 years ago

Leftfield thought, imagining a tax (vaguely like carbon credits) on energy/cpu cycles/modelling (just my vague idea). With the proceeds going to other projects to diversify the outcomes.

Always seems to me our economy is being less based on money and more on energy and productivity, basically being rounded down to physics as a healthy market economy should.

itissid2• 3 years ago

Noob Q: If I have 25000$ can I buy some GPUs and support the effort for supporting GPU Poor? Like is there a DIYer's guide to do this?

I don't want to donate the cash but instead set up a handful of GPUs(Somehow?) and let people pay at cost to use them.

itissid2• 3 years ago

Assume location is East Coast(NYC).

capableweb3• 3 years ago

vast.ai can help you offer this and has bunch of docs on how you can start hosting: https://vast.ai/docs/hosting/hosting

You won't be able to limit it to a particular set of clients who want to use it, but you're basically supporting the "GPU Poor" by doing this, as I think it's mostly smaller companies, individuals and researchers using vast.ai, rather than huge companies.

redox993• 3 years ago

25K will either get you money for renting servers, or allow you to jerry rig yourself something like a 8x4090 machine.

The servers that most people would like to run (8xA100, 8xH100) are over 200k, and even if you had the money they're probably very hard to get right now.

naillo2• 3 years ago

8x4090 machines should be a lot cheaper than that. 1599*8=12,792. I.e. closer to 16x4090 machines. Sure labor is worth something but he has it in cash, and also there's barely a lot of people (yet) paying high prices for research (non ML ops) labor hours for this yet so the salary wouldn't be that high anyway.

Just a pet peeve when ppl under exaggerate how many gpus/flops a dollar can buy (if you're slighly smart with your money).

rajko_rad17• 3 years ago

honestly would recommend using spot/rented instances on that budget vs. buying! A100s run around ~$2/hour, the hard part is getting a large allocation... training workloads are quite spikey anyways, usually running lots of smaller experiments before actually doing a larger single run...

brailsafe• 3 years ago

No intention to attribute malice, but wouldn't this be more like buying a condo and renting it out to pay your mortgage rather than donating resources?

rajko_rad17• 3 years ago

we didn't buy a condo haha, are letting people choose their own housing :)

RcouF1uZ4gsC• 3 years ago

Eric Hartford: fine-tuning uncensored LLMs

Makes me very happy that this is being undertaken.

LLMs will be a very valuable tool. Having censorship built into these tools makes me very nervous. Imagine if Word refused to allow you to type any “bad word” in the name of safety and ethics.

naillo2• 3 years ago

A16z is getting 'mad respect' from me for taking this line of the 'ai wars' and generally their attitude on all this. Really cool that they're going straight into 2010 era startup energy on this new wave. Idk I just find them really cool and inspirational.

outside1234• 3 years ago

If nothing else, it is good to see them moving off of the Bitcoin grift

zekrioca2• 3 years ago

I'm not sure why an application link wasn't provided. How is one suppose to reach them to apply?

seism• 3 years ago

Hi Robin Hood. I wonder about the "continue their work without the pressure to generate financial returns" part. I don't see an expectation to work on non-commercial, or socially conscious, applications, just to commit to open source (what level of commitment also being unclear). It would actually be great if support programs along these lines were being offered by governments and public service institutions, with some serious governance and transparency around the process.

Surely there are also some interesting econometrics on the "GPU poverty line" - someone has literature to share?

evanwolf2• 3 years ago

Yeah, this is moneyed interests looking out for their own investments instead of government acting on behalf of the public good.

GaggiX• 3 years ago

This reminds when a random chinese user finetuned the leaked NovelAI Stable Diffusion model and released it online, the Anything V3 model, tech company like Tencent started using it in their products.

tikkun4• 3 years ago

What are other open source LLM projects that people rely on? As a way to come up with ideas for future grant recipients.

This is a good initiative, and an excellent initial batch. A great use of management fees!

I'm struggling to think of additional ones because I feel like this covers everyone I would've suggested (particularly TheBloke, vLLM and oobabooga).

Edit, some ideas for future grant recipients would be:

* MLC LLM

* ExLlama

* An open source fork of text-generation-inference

* AutoGPTQ

rajko_rad17• 3 years ago

Awesome suggestions, thank you!

syntaxing• 3 years ago

Whoa this is honestly awesome, a lot of tools I see on the list are ones I’ve been using so it’s great to see they’re getting financial support. Really hoping Ollama joins the list soon. I have some fun ideas I’ve been wanting to test out but probably requires a couple hours on 8X A100s + EvolveInstruct style data that cost upward of $1K+. Gonna start preparing some code to see if I can join grant #2!

rajko_rad17• 3 years ago

Love it!! Good luck with the mvps!

And thank you so much for the input - we are meeting the folks at ollama!!

stuaxo• 3 years ago

This is great, but "dual 4090s" bring this stuff down to the Ryzen APU level.

jph00• 3 years ago

I'm one of the recipients of the AI grants, to support my work at fast.ai. I'm extremely grateful to a16z for their support. Here's some additional details based on questions I see in the comments:

My grant covered the purchase of a Scalar server from Lambda Labs, which allowed me to configure a system with 8 previous-gen A6000 GPUs, partly also thanks to NVIDIA who has recently given me 3 of those cards, and Lambda Labs who offered to provide everything at cost.

a16z didn't ask for any equity or any kind of quid pro quo, other than to let folks know that they provided this grant. They couldn't have been more helpful through the process - I didn't have to fill out any forms (other than sign a single one page agreement), the contract was clear and totally fair (even explicitly saying that a16z wasn't going to receive any kind of rights to anything), and they wired me the money for buying the server promptly.

Mascobot• 3 years ago

Thanks for all the work Jeremy! There's a large community of folks that have been learning from you, the Fast.ai community and your work. Super excited for this grant!

rajko_rad17• 3 years ago

Honored to have you in the inaugural cohort @jph00!!

jasrys• 3 years ago

Couldn't go to a better home. Thanks for making this field approachable.

jdshaffer• 3 years ago

It makes me very happy to read about things like this -- support where it's needed and appreciated AND absolutely no strings attached. Wow. Wish there were more cases like this.

Best wishes in your work, jph00!

0xcde4c3db• 3 years ago

previous-gen A6000 GPUs

Just in case anyone hasn't surveyed the landscape lately, the distinction being made here is between "RTX A6000" (Ampere architecture; GeForce 30-series equivalent) and "RTX 6000 Ada Generation" (Ada architecture; GeForce 40-series equivalent) products. Nvidia is apparently convinced that it can get away with being almost as terrible at naming technology generations as the USB Implementers Forum.

MichaelZuo• 3 years ago

a16z didn't ask for any equity or any kind of quid pro quo, other than to let folks know that they provided this grant. ... the contract was clear and totally fair (even explicitly saying that a16z wasn't going to receive any kind of rights to anything)...

Thanks for sharing, that is generous, and a pretty solid sign of being honest businessmen.

Reviving1514• 3 years ago

Hi Jeremy,

I'm sure you get this a lot, but thank you for teaching me ML - I am a hardware engineer/manufacturing person who learned to code in the early 2010s and it's been immensely helpful in my career. Learning ML feels like V2 of that - picking up a new skill that is going to end up being useful in so many places. You made that journey much easier and more accessible and I very much appreciate it.

(No need to respond at all - just wanted to pass along the gratitude!)

throwaway_89282• 3 years ago

Thanks for providing context Jeremy. Fast AI has done an admirable job in communicating its commitment to an ethical pursuit of AI advancement. Do you feel such a commitment aligns with the bellicose rhetoric found in a16z's vision? Here is Marc Andreessen in a recent essay...

"There is one final, and real, AI risk that is probably the scariest at all:

AI isn’t just being developed in the relatively free societies of the West, it is also being developed by the Communist Party of the People’s Republic of China.

China has a vastly different vision for AI than we do – they view it as a mechanism for authoritarian population control, full stop. They are not even being secretive about this, they are very clear about it, and they are already pursuing their agenda. And they do not intend to limit their AI strategy to China – they intend to proliferate it all across the world, everywhere they are powering 5G networks, everywhere they are loaning Belt And Road money, everywhere they are providing friendly consumer apps like Tiktok that serve as front ends to their centralized command and control AI.

The single greatest risk of AI is that China wins global AI dominance and we – the United States and the West – do not.

I propose a simple strategy for what to do about this – in fact, the same strategy President Ronald Reagan used to win the first Cold War with the Soviet Union.

'We win, they lose.'

Rather than allowing ungrounded panics around killer AI, “harmful” AI, job-destroying AI, and inequality-generating AI to put us on our back feet, we in the United States and the West should lean into AI as hard as we possibly can.

We should seek to win the race to global AI technological superiority and ensure that China does not."

I don't think one needs to think too hard about the grave consequences a mission of "We win, they lose" may have for human survival.

Source: https://a16z.com/2023/06/06/ai-will-save-the-world/

JonChesterfield• 3 years ago

Is this a venture capital fund donating money to nvidia in order to support open source? It seems like some wires have got crossed along the way here

Long Live the 'GPU Poor' – Open-Source AI Grants

Related Stories