Meta and Qualcomm team up to run Llama 2 on phones

https://www.cnbc.com/2023/07/18/meta-and-qualcomm-team-up-to-run-big-ai-models-on-phones.html

by tzm • 3 years ago

18 3 3 years ago

To me, this is probably the most important announcement from today. The ability to have an on-device low-latency implementation of an LLM is going to be key to AI/UX. This is similar to Google's various PaLM versions for Android they announced. It will have a huge impact on mobile phones, TVs, smart screens, etc. Obviously the devil will be in the details - like how well will it actually work? But even if it's just the first step, it's an important one.

I bet next up will be at the end of the year when Apple surely jumps into the ring with their version of on-device Siri, powered by their ML chips.

yumraj• 3 years ago

Llama 2, to run on Qualcomm chips on phones and PCs starting in 2024

Are they expecting for Llama 2 to still be relevant in 2024? Shouldn't it have been LLM or Llama (without the version).

brucethemoose2• 3 years ago

Qualcomm wanted to run Stable Diffusion on phones as well, and IIRC they published a demo, but no one cared because it was a barebones demo.

MLC-LLM already runs LlamaV1 (and probably V2?) performantly on Android and iOS. But again, no one seems to care because its a relatively barebones demo.

Qualcomm needs to build and extensive feature set and probably a UI, otherwise no one is going to care about this implementation either.

Meta and Qualcomm team up to run Llama 2 on phones

Related Stories