Gemma: New Open Models

https://blog.google/technology/developers/gemma-open-models/

by meetpateltech • 2 years ago

1129 507 2 years ago

blog.google

smcn• 2 years ago

There are some pretty impressive benchmarks on https://ai.google.dev/gemma. Even the 2b model looks fairly not awful?

I guess my weekend is going to be spent exploring this.

alekandreev9• 2 years ago

Hello on behalf of the Gemma team! We are really excited to answer any questions you may have about our models.

Opinions are our own and not of Google DeepMind.

declaredapple2• 2 years ago

Congrats on the launch and thanks for the contribution! This looks like it's on-par or better compared to mistral 7B 0.1 or is that 0.2?

Are there plans for MoE or 70B models?

kathleenfromgdm9• 2 years ago

Great question - we compare to the Mistral 7B 0.1 pretrained models (since there were no pretrained checkpoint updates in 0.2) and the Mistral 7B 0.2 instruction-tuned models in the technical report here: https://goo.gle/GemmaReport

zitterbewegung• 2 years ago

Do you have a plan of releasing higher parameter models?

alekandreev9• 2 years ago

We have many great things in research and development phases, so stay tuned. I’m hopeful we can share more in the coming weeks and month!

neximo643• 2 years ago

How are these performing so well compared to Llama 2, are there any documents on the architecture and differences, is it MoE?

Also note some of the links on the blog post don't work, e.g debugging tool.

kathleenfromgdm9• 2 years ago

We've documented the architecture (including key differences) in our technical report here (https://goo.gle/GemmaReport), and you can see the architecture implementation in our Git Repo (https://github.com/google-deepmind/gemma).

h1t35h2• 2 years ago

It seems you have exposed the internal debugging tool link in the blog post. You may want to do something about it.

trisfromgoogle6• 2 years ago

Ah, I see -- the link is wrong, thank you for flagging! Fixing now.

pama• 2 years ago

Will these soon be available on lmsys for human comparison against other models? Can they run with llama.cpp?

ErneX• 2 years ago

Yes to llama.cpp

https://twitter.com/ggerganov/status/1760293079313973408

artninja19884• 2 years ago

I find the snyde remarks around open source in the paper and announcement rather off putting.

As the ecosystem evolves, we urge the corporate AI community to move beyond demanding to be taken seriously as a player in open source for models that are not actually open, and avoid preaching with a PR statement that can be interpreted as uniformed at best or malicious at worst.

silentsanctuary• 2 years ago

Which remarks are you referring to?

trisfromgoogle6• 2 years ago

It would be great to understand what you mean by this -- we have a deep love for open source and the open developer ecosystem. Our open source team also released a blog today describing the rationale and approach for open models and continuing AI releases in the open ecosystem:

https://opensource.googleblog.com/2024/02/building-open-mode...

Thoughts and feedback welcome, as always.

jppittma2• 2 years ago

Working at google is like this, where no matter how much you try to do the right thing you're always under attack.

tosh3• 2 years ago

Are there any plans for releasing the datasets used?

alekandreev9• 2 years ago

This would be really interesting in my opinion, but we are not releasing datasets at this time. See the C4 dataset for an earlier open dataset from Google.

sbarre2• 2 years ago

Can the Gemma models be downloaded to run locally, like open-source models Llama2, Mistral, etc ?

Or is your definition of "open" different?

kathleenfromgdm9• 2 years ago

Yes, you can get started downloading the model and running inference on Kaggle: https://www.kaggle.com/models/google/gemma ; for a full list of ways to interact with the model, you can check out https://ai.google.dev/gemma.

Kostic• 2 years ago

It should be possible to run it via llama.cpp[0] now.

[0] https://github.com/ggerganov/llama.cpp/pull/5631

tomp• 2 years ago

Their definition of "open" is "not open", i.e. you're only allowed to use Gemma in "non-harmful" way.

We all know that Google thinks that saying that 1800s English kings were white is "harmful".

austinvhuang2• 2 years ago

Yes models can be downloaded locally. In addition to the python NN frameworks and ggml as options, we also implemented a standalone C++ implementation that you can run locally at https://github.com/google/gemma.cpp

mrob3• 2 years ago

Mistral weights are released under an Apache 2.0 license, but Llama 2 weights are released under a proprietary license that prohibits use by large organizations and imposes usage restrictions, violating terms 5 and 6 the Open Source Definition[0]. Even if you accept that a model with a proprietary training dataset and proprietary training code can be considered "open source", there's no way Llama 2 qualifies.

For consistency with existing definitions[1], Llama 2 should be labeled a "weights available" model.

[0] https://en.wikipedia.org/wiki/The_Open_Source_Definition

[1] https://en.wikipedia.org/wiki/Source-available_software

vorticalbox2• 2 years ago

are there plans to release an official GGUF version to use with llama.ccp?

espadrine7• 2 years ago

It is already part of the release on Huggingface: https://huggingface.co/google/gemma-7b/blob/main/gemma-7b.gg...

It is a pretty clean release! I had some 500 issues with Kaggle validating my license approval, so you might too, but after a few attempts I could access the model.

sqreept3• 2 years ago

What are the supported languages of these models?

alekandreev9• 2 years ago

This v1 model is focused on English support, but you may find some multilingual capabilities.

lnyan• 2 years ago

Will there be Gemma-vision models or multimodal Gemma models?

Jayakumark• 2 years ago

Have the same question.

alekandreev9• 2 years ago

We have many exciting things planned that we can't reveal just yet :)

CuriouslyC• 2 years ago

It's cool that you guys are able to release open stuff, that must be a nice change from the modus operandi at goog. I'll have to double check but it looks like phi-2 beats your performance in some cases while being smaller, I'm guessing the value proposition of these models is being small and good while also having more knowledge baked in?

alekandreev9• 2 years ago

We deeply respect the Phi team and all other teams in the open model space. You’ll find that different models have different strengths and not all can be quantified with existing public evals. Take them for a spin and see what works for you.

turnsout2• 2 years ago

What is the license? I couldn’t find it on the 1P site or Kaggle.

trisfromgoogle6• 2 years ago

You can find the terms on our website, ai.google.dev/gemma:

https://ai.google.dev/gemma/terms

audessuscest3• 2 years ago

Does this model also thinks german were black 200 years ago ? Or is afraid to answer basic stuff ? because if this is the case no one will care about that model.

freedomben• 2 years ago

I don't know anything about these twitter accounts so I don't know how credible they are, but here are some examples for your downvoters that I'm guessing just think you're just trolling or grossly exaggerating:

https://twitter.com/aginnt/status/1760159436323123632

https://twitter.com/Black_Pilled/status/1760198299443966382

graphe2• 2 years ago

I disagree, coding and RAG performance is all that matters to me. I'm not using an LLM to learn basic facts I already know.

brucethemoose210• 2 years ago

Will there be "extended context" releases like 01.ai did for Yi?

Also, is the model GQA?

hustwindmaple1• 2 years ago

It's MQA, documented in the tech report

lordswork9• 2 years ago

Is there any truth behind this claim that folks who worked on Gemma have left Google?

https://x.com/yar_vol/status/1760314018575634842

CaffeinatedDev• 2 years ago

Them: here to answer questions

Question

Them: :O

elcomet2• 2 years ago

It seems very easy to check no? Look at the names in the paper and check where they are working now

lordswork9• 2 years ago

I confirmed all the folks listed on page 12 are still at Google (listed below). I am guessing the linked tweet is a BS claim.

   # Product Management
   Tris Warkentin
   Ludovic Peran

   # Program Management
   Minh Giang

   # Executive Sponsors
   Clement Farabet
   Oriol Vinyals
   Jeff Dean
   Koray Kavukcuoglu
   Demis Hassabis
   Zoubin Ghahramani
   Douglas Eck
   Joelle Barral
   Fernando Pereira
   Eli Collins

   # Leads
   Armand Joulin
   Noah Fiedel
   Evan Senter

   # Tech Leads
   Alek Andreev†
   Kathleen Kenealy†

bluefinity• 2 years ago

To be fair, the tweet says that they don't work on the models at Google anymore, not that they have left Google.

Might be true, might not be. It's unsourced speculation.

memossy• 2 years ago

Training on 4096 v5es how did you handle crazy batch size :o

quickgist• 2 years ago

Will this be available as a Vertex AI foundational model like Gemini 1.0, without deploying a custom endpoint? Any info on pricing? (Also, when will Gemini 1.5 be available on Vertex?)

moffkalast4• 2 years ago

I'm not sure if this was mentioned in the paper somewhere, but how much does the super large 265k tokenizer vocabulary influence inference speed and how much higher is the average text compression compared to llama's usual 30k? In short, is it really worth going beyond GPT 4's 100k?

dmnsl• 2 years ago

Hi, what is the cutoff date ?

legohead2• 2 years ago

All it will tell me is mid-2018.

alekandreev9• 2 years ago

September 2023.

cypress66• 2 years ago

Can you share the training loss curve?

fosterfriends• 2 years ago

Not a question, but thank you for your hard work! Also, brave of you to join the HN comments, I appreciate your openness. Hope y'all get to celebrate the launch :)

voxgen2• 2 years ago

Thank you very much for releasing these models! It's great to see Google enter the battle with a strong hand.

I'm wondering if you're able to provide any insight into the below hyperparameter decisions in Gemma's architecture, as they differ significantly from what we've seen with other recent models?

* On the 7B model, the `d_model` (3072) is smaller than `num_heads * d_head` (16*256=4096). I don't know of any other model where these numbers don't match.

* The FFN expansion factor of 16x is MUCH higher than the Llama-2-7B's 5.4x, which itself was chosen to be equi-FLOPS with PaLM's 4x.

* The vocab is much larger - 256k, where most small models use 32k-64k.

* GQA is only used on the 2B model, where we've seen other models prefer to save it for larger models.

These observations are in no way meant to be criticism - I understand that Llama's hyperparameters are also somewhat arbitrarily inherited from its predecessors like PaLM and GPT-2, and that it's non-trivial to run hyperopt on such large models. I'm just really curious about what findings motivated these choices.

owl_brawl2• 2 years ago

I would love answers to these questions too, particularly on the vocab size

LorenDB2• 2 years ago

EDIT: it seems this is likely an Ollama bug, please keep that in mind for the rest of this comment :)

I ran Gemma in Ollama and noticed two things. First, it is slow. Gemma got less than 40 tok/s while Llama 2 7B got over 80 tok/s. Second, it is very bad at output generation. I said "hi", and it responded this:

``` Hi, . What is up? melizing with you today!

What would you like to talk about or hear from me on this fine day?? ```

With longer and more complex prompts it goes completely off the rails. Here's a snippet from its response to "Explain how to use Qt to get the current IP from https://icanhazip.com":

``` python print( "Error consonming IP arrangration at [local machine's hostname]. Please try fufing this function later!") ## guanomment messages are typically displayed using QtWidgets.MessageBox ```

Do you see similar results on your end or is this just a bug in Ollama? I have a terrible suspicion that this might be a completely flawed model, but I'm holding out hope that Ollama just has a bug somewhere.

mark_l_watson2• 2 years ago

I was going to try these models with Ollama. Did you use a small number of bits/quantization?

jmorgan• 2 years ago

Hi! This is such an exciting release. Congratulations!

I work on Ollama and used the provided GGUF files to quantize the model. As mentioned by a few people here, the 4-bit integer quantized models (which Ollama defaults to) seem to have strange output with non-existent words and funny use of whitespace.

Do you have a link /reference as to how the models were converted to GGUF format? And is it expected that quantizing the models might cause this issue?

Thanks so much!

espadrine7• 2 years ago

As a data point, using the Huggingface Transformers 4-bit quantization yields reasonable results: https://twitter.com/espadrine/status/1760355758309298421

kleiba2• 2 years ago

We are really excited to answer any questions you may have about our models.

I cannot count how many times I've seen similar posts on HN, followed by tens of questions from other users, three of which actually get answered by the OP. This one seems to be no exception so far.

spankalee• 2 years ago

What are you talking about? The team is in this thread answering questions.

alekandreev9• 2 years ago

Sorry, doing our best here :)

owl_brawl2• 2 years ago

Hi alekandreev,

Any reason you decided to go with a token vocabulary size of 256k? Smaller vocab/vector sizes like most models in this size seem to be using (~16-32k) are much easier to work with. Would love to understand the technical reasoning here that isn't detailed in the report unfortunately :(.

nuclearjam• 2 years ago

May I ask what is the ram requirement for running the 2B model on CPU on an average consumer windows laptop? I have 16 gb RAM but I am seeing CPU/memory traceback. I’m using the transformer implementation.

Havoc• 2 years ago

Taking a page out of metas book with open models. I wonder what the game plan here is.

Nice that it allows commercial use!

gaogao2• 2 years ago

Mostly to boost research and commercial usage around JAX/Gemini is my read.

Any internal research using Gemma is now more easily externally reproducible, external research and frameworks are easier to translate over, goodwill especially from researchers.

gaogao2• 2 years ago

There's also less of a special sauce for text models itself these days with the propietary being more on the pre-training data and training stack (e.g. how to get 10k GPUs/TPUs running together smoothly). Multi-modal models (or adjacent like Sora) are less likely to be open sourced in the immediate term.

sidcool• 2 years ago

Available on Ollama?

blooalien2• 2 years ago

https://ollama.com/library?q=gemma

Library search says "Nope". At least not yet.

tomd• 2 years ago

It's there now

kevsim• 2 years ago

And now it says "Yup". That was pretty quick!

SushiHippie2• 2 years ago

Support for gemma in llama.cpp just got merged, so it may take some time (could be hours or days) until this lands in ollama

https://github.com/ggerganov/llama.cpp/pull/5631

dcchambers2• 2 years ago

It's now in the 0.1.26 pre-release: https://github.com/ollama/ollama/releases/tag/v0.1.26

chown4• 2 years ago

Available in pre-release now which means you’d have to update manually in future.

mustafabisic1• 2 years ago

The fact Gemma team is in the comments section answering questions is praiseworthy to me :)

p1esk• 2 years ago

https://twitter.com/yar_vol/status/1760314018575634842

pphysch• 2 years ago

Why is this anonymous tweet with no evidence or engagement being posted by multiple users in this thread? Why not just make the same claim directly?

callalex• 2 years ago

The link is broken. On HN (or any forum really) it is expected for a brief description of the content to be provided when posting a link. Links die all the time, but forum posts don’t have to die with them.

carom• 2 years ago

I've worked at Google. It is the organization with highest concentration of engineering talent I've ever been at. Almost to the point that it is ridiculous because you have extremely good engineers working on internal reporting systems for middle managers.

Kelteseth• 2 years ago

Can this run on my AMD Vega VII on Windows 11? As always, AMD is missing:

Optimization across multiple AI hardware platforms ensures industry-leading performance, including NVIDIA GPUs and Google Cloud TPUs.

lordswork9• 2 years ago

AMD Vega VII meets the memory requirements. Once tools like LM Studio, ollama, etc. add support for the model, you should be able to run locally like you would any other open weights model.

GaggiX7• 2 years ago

They have implemented the model also on their own C++ inference engine: https://github.com/google/gemma.cpp

0xbadc0de5• 2 years ago

Thank you for releasing this.

vanderboyd• 2 years ago

The 2B model seems underwhelming. For instance, compared to the recent StableLM2 1.6B model that is slightly smaller and probably wastes some "English metric points" by being multilingual.

The latter (and other similar open models) seem to do similarly well in benchmarks (much better in Math?) with way less fancy stuff. For instance, public data and no secretive filtering with pre trained models or synthetic data.

My take is that using the vanilla approaches take you really far. And many of the latest tricks and hours-of-work buy you little... Will be interesting to see how this plays out, especially for the open source community.

impulser_• 2 years ago

Go back 5 years and ask anyone on this site what companies do you think will be the most open about AI in the future OpenAI, Meta, or Google. I bet 10/10 people would pick OpenAI. Now today Meta and Google, both trillion dollars companies, are releasing very powerful open models with the ability to be used commercially.

Ironic.

vmfunction• 2 years ago

Not surprising, just like when MS went to shit, and then they start to embrace 'open source'. Seems like PR stunt. And when it comes to LLM there is millions of dollar barrier to entry to train the model, so it is ok to open up their embedding etc.

Today big corp A will open up a little to court the developers, and tomorrow when it gains dominance it will close up, and corp B open up a little.

kibwen• 2 years ago

True, though to be fair, when OpenAI embraced "openness" it was also a PR stunt.

milansuk• 2 years ago

You can run Gemma and hundreds of other models(many fine-tuned) in llama.cpp. It's easy to swap to a different model.

It's important there are companies publishing models(running locally). If some stop and others are born, it's ok. The worst thing that could happen is having AI only in the cloud.

jchw• 2 years ago

Eh, I don't really blame anyone for being cynical but open weight AI model releases seem like a pretty clear mutual benefit for Google. PR aside, they also can push people to try these models on TPUs and the like. If anything, this seems like it's just one of those things where people win because of competition. OpenAI going closed may have felt like the most obvious betrayal ever, but OTOH anyone whose best interests are to eat their lunch have an incentive to push actually-open AI, and that's a lot of parties.

Seems like anyone who is releasing open weight models today could close it up any day, but at least while competition is hot among wealthy companies, we're going to have a lot of nice things.

rvz4• 2 years ago

And when it comes to LLM there is millions of dollar barrier to entry to train the model, so it is ok to open up their embedding etc.

That barrier is the first basic moat; hundreds of millions of dollars needed to train a better model. Eliminating tons of companies and reducing it to a handful.

The second moat is the ownership of the tons of data to train the models on.

The third is the hardware and data centers setup to create the model in a reasonable amount of time faster than others.

Put together all three and you have Meta, Google, Apple and Microsoft.

The last is the silicon product. Nvidia which has >80pc of the entire GPU market and being the #1 AI shovel maker for both inference and training.

throwaw12• 2 years ago

they want to kill competition before it gets too big using the hands of open source community and enthusiasts

infecto3• 2 years ago

Ironic but I wonder how true this would be if Google was first to market.

gmaster1440• 2 years ago

It's almost the inverse of going back 5 years and asking what companies will release the most successful or impressive AI's.

brainless• 2 years ago

This article states quite an impressive list of open source tools that Google has released for years in the past. This is no surprise coming from* them. Google has released some large pieces of source in other domains as well, Chromium comes to mind, which probably impacts most Internet users directly.

The question is not about Google but about OpenAI.

sunnybeetroot3• 2 years ago

Did you miss a footnote with your asterisks?

infecto3• 2 years ago

I have a different take, Google releases a lot but is also a massive company and tools like Chromium serve to increase their stock price so they can hit their quarterly estimates.

blackoil2• 2 years ago

I think more than benevolence of GOOG it is about strategic OSS to commoditize your complements.

https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/

makestuff• 2 years ago

Google also has released Guice/Dagger for Java dependency injection. Angular never really took off, but guice/dagger are widely used. Also I am pretty impressed with Flutter as an alternative to react native.

blackoil2• 2 years ago

I think current understanding is <50-100B parameter models will be commodity and would provide no moat. Competition will be in Gemini Ultra/GPT4+ models.

So open sourcing simple models brings PR and possibility of biasing OSS towards your own models.

extheat• 2 years ago

LLaMA 3 with >=70B params will be launching this year, so I don't think this is something that will hold for long. And Mixtral 8x7B is a 56GB model, sparsely. For now I agree, for many companies it doesn't make sense to open source something you intend to sell for commercial use, so the biggest models will likely be withheld. However, the important more thing is that there is some open source model, whether it be from Meta or someone else, that can rival the best open source models. And it's not like the param count can literally go to infinity, there's going to be an upper bound that today's hardware can achieve.

DJHenk• 2 years ago

Ironic.

Not at all. When you're the underdog, it makes perfect sense to be open because you can profit from the work of the community and gain market share. Only after establishing some kind of dominance or monopoly it makes sense (profit wise) to switch to closed technology.

OpenAI was open, but is now the leader and closed up. Meta and Google need to play catch up, so they are open.

ekianjo• 2 years ago

OpenAI was open

When is the last time they released something in the open?

dkjaudyeqooe2• 2 years ago

Not at all. When you're the underdog, it makes perfect sense to be open because you can profit from the work of the community and gain market share. Only after establishing some kind of dominance or monopoly it makes sense (profit wise) to switch to closed technology.

That is purely the language of commerce. OpenAI was supposed to be a public benefit organisation, but it acts like a garden variety evil corp.

Even garden variety evil corps spend decades benefitting society with good products and services before they become big and greedy, but OpenAI skipped all that and just cut to the chase. It saw an opening with the insane hype around ChatGPT and just grabbed all it could as fast as it could.

I have a special contempt for OpenAI on that basis.

behnamoh• 2 years ago

This. MistralAI is also underdog and released Mitral 7b and Mixtral 8x7b, but as soon as they got traction, they closed their models (e.g., Mistral Medium).

jncraton• 2 years ago

Google released the T5 paper about 5 years ago:

https://arxiv.org/abs/1910.10683

This included full model weights along with a detailed description of the dataset, training process, and ablations that led them to that architecture. T5 was state-of-the-art on many benchmarks when it was released, but it was of course quickly eclipsed by GPT-3.

It was common practice from Google (BERT, T5), Meta (BART), OpenAI (GPT1, GPT2) and others to release full training details and model weights. Following GPT-3, it became much more common for labs to not release full details or model weights.

phillipcarter2• 2 years ago

I would have picked Google five years ago, since nobody was releasing commercially viable LLMs at the time, and Google was the center of all the research that I knew of.

calebkaiser• 2 years ago

Since the release of GPT-2 (it was initially "too dangerous" to release the weights), I think most people in the industry have assumed that OpenAI does not see open sourcing their models as a strategic advantage.

moffkalast4• 2 years ago

what companies do you think will be the most open about AI in the future OpenAI, Meta, or Google.

The funny part is that the real answer is: Some random French company is running circles around them all.

I mean who the hell just drops a torrent magnet link onto twitter for the best state of the art LLM base model for its size class, and with a completely open license. No corporate grandstanding, no benchmark overpromises, no theatrics. That was unfathomably based of Mistral.

nalzok• 2 years ago

Congratulations on the release! How can we download the model and run inference locally?

kathleenfromgdm9• 2 years ago

Thank you! You can get started downloading the model and running inference on Kaggle: https://www.kaggle.com/models/google/gemma ; for a full list of ways to interact with the model, you can check out https://ai.google.dev/gemma.

aphit• 2 years ago

FYI the ; broke the link, but I found it easily anyway.

austinvhuang2• 2 years ago

You can download the model checkpoints from kaggle https://www.kaggle.com/models/google/gemma and huggingface https://huggingface.co/blog/gemma

Besides the python implementations, we also implemented a standalone C++ implementation that runs locally with just CPU simd https://github.com/google/gemma.cpp

tveita• 2 years ago

Are there any cool highlights you can give us about gemma.cpp? Does it have any technical advantages over llama.cpp? It looks like it introduces its own quantization format, is there a speed or accuracy gain over llama.cpp's 8-bit quantization?

espadrine7• 2 years ago

I notice a few divergences to common models:

- The feedforward hidden size is 16x the d_model, unlike most models which are typically 4x;

- The vocabulary size is 10x (256K vs. Mistral’s 32K);

- The training token count is tripled (6T vs. Llama2's 2T)

Apart from that, it uses the classic transformer variations: MQA, RoPE, RMSNorm.

How big was the batch size that it could be trained so fast?

https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2/bl...

GaggiX7• 2 years ago

Looking at the config.json of Gemma 7B the feedfoarward hidden size is 8x, not 16x

espadrine7• 2 years ago

Huh, indeed, that's what the config.json[0] says; the report[1] indicates “Feedforward hidden dims: 49152”.

[0] https://huggingface.co/google/gemma-7b-it/blob/main/config.j...

[1] https://storage.googleapis.com/deepmind-media/gemma/gemma-re...

lalaithion3• 2 years ago

What does tokenization look like in 256k vs 32k?

espadrine7• 2 years ago

It mostly means that there are tokens dedicated to rarer sequences of characters, even in foreign languages (note that Gemma is not intended to be good multilingually): “説明書” (instruction manual) has its own token, and so does “Nixon”, “آباد” (a city suffix, I believe), and the HTML sequence "\"><!--".

visarga2• 2 years ago

Text encodes in fewer tokens, and language coverage is better.

andy_xor_andrew• 2 years ago

The training token count is tripled (6T vs. Llama2's 2T)

Damn, 6T? That's a lot!

Given that this model seems to roughly match Mistral (according to the numbers from Google), this makes me think we have saturated the 7B parameter space, and couldn't possibly make it much better unless new techniques are discovered.

espadrine7• 2 years ago

Hard to say definitively. Mistral’s token embeddings only account for <2% of the 7B parameters, while Gemma’s larger token vocabulary vampirized over 10%, leaving less space for the more important parts of the network. It is a somewhat surprising tradeoff given that it was pretrained towards an English bias.

margorczynski2• 2 years ago

Is there a chance we'll get a model without the "aligment" (lobotomization)? There are many examples where answers from Gemini are garbage because of the ideological fine tuning.

yakorevivan• 2 years ago

They have released finetuning code too. You can finetune it to remove the alignment finetuning. I believe it would take just a few hours at max and a couple of dollars.

FergusArgyll5• 2 years ago

You can (and someone will) fine tune it away. There are datasets which are foss you can use on hugging face.

Or you can just wait, it'll be done soon...

joshelgar• 2 years ago

Could you give an example of these datasets?

declaredapple2• 2 years ago

You can but it'll never be the same as the base model.

That said it appears they also released the base checkpoints that aren't fine-tuned for alignment

kathleenfromgdm9• 2 years ago

We release our non-aligned models (marked as pretrained or PT models across platforms) alongside our fine-tuned checkpoints; for example, here is our pretrained 7B checkpoint for download: https://www.kaggle.com/models/google/gemma/frameworks/keras/...

politician3• 2 years ago

More useful would be a precise characterization of the type and balance of the ideological fine tuning.

They include performance benchmarks. End-users should also be aware of what thoughts are permitted in these constructs. Why omit this information?

ben_w4• 2 years ago

End-users should also be aware of what thoughts are permitted in these constructs. Why omit this information?

Can you define that in a way that's actually testable? I can't, and I've been thinking about "unthinkable thoughts" for quite some time now: https://kitsunesoftware.wordpress.com/2018/06/26/unlearnable...

brucethemoose210• 2 years ago

Alignment is all but a non issue with open weight base model releases, as they can be finetuned to "de align" them if prompt engineering is not enough.

tosh3• 2 years ago

Benchmarks for Gemma 7B seem to be in the ballpark of Mistral 7B

  +-------------+----------+-------------+-------------+
  | Benchmark   | Gemma 7B | Mistral 7B  | Llama-2 7B  |
  +-------------+----------+-------------+-------------+
  | MMLU        |   64.3   |     60.1    |     45.3    |
  | HellaSwag   |   81.2   |     81.3    |     77.2    |
  | HumanEval   |   32.3   |     30.5    |     12.8    |
  +-------------+----------+-------------+-------------+

via https://mistral.ai/news/announcing-mistral-7b/

brucethemoose210• 2 years ago

Only 8K context as well, like Mistral.

Also, as always, take these benchmarks with a huge grain of salt. Even base model releases are frequently (seemingly) contaminated these days.

tosh3• 2 years ago

Agree: will be interesting how Gemma does on ChatBot Arena

DreamGen• 2 years ago

Mistral Instruct v0.2 is 32K.

Kydlaw• 2 years ago

They state in their report that they filter evaluation data off their training data, see p.3 - Filtering:

"Further, we filter all evaluation sets from our pre-training data mixture, run targeted contamination analyses to check against evaluation set leakage, and reduce the risk of recitation by minimizing proliferation of sensitive outputs."

jcuenod• 2 years ago

Came here to post the same thing for Phi-2:

  +-------------+----------+-------------+
  | Benchmark   | Gemma 2B | Phi-2 2.7B  |
  +-------------+----------+-------------+
  | MMLU        |   42.3   |     56.7    |
  | MBPP        |   29.2   |     59.1    |
  | BoolQ       |   69.4   |     83.3    |
  +-------------+----------+-------------+

[0] https://www.kaggle.com/models/google/gemma

[1] https://www.microsoft.com/en-us/research/blog/phi-2-the-surp...

daemonologist• 2 years ago

Really looking forward to the day someone puts out an open model which outperforms Flan-T5 on BoolQ.

rfw300• 2 years ago

A caveat: my impression of Phi-2, based on my own use and others’ experiences online, is that these benchmarks do not remotely resemble reality. The model is a paper tiger that is unable to perform almost any real-world task because it’s been fed so heavily with almost exclusively synthetic data targeted towards improving benchmark performance.

FergusArgyll5• 2 years ago

the real gold will be when this gets finetuned. (maybe by mistral...)

brucethemoose210• 2 years ago

TBH the community has largely outrun Mistral's own finetuning. The 7B model in particular is such a popular target because its so practical to train.

itomatik2• 2 years ago

how does one finetune llama (or any other LLM) using mistral?

is the flow like this?

- take small dataset

- generate bigger dataset using mistral (how this is this done?)

- run LoRA to fine tune gemma extended dataset.

sa-code2• 2 years ago

Thank you. I thought it was weird for them to release a 7B model and not mention Mistral in their release.

mirekrusin• 2 years ago

They forgot.

Also phi-2.

mochomocha• 2 years ago

The technical report (linked in the 2nd paragraph of the blog post) mentions it, and compares against it: https://storage.googleapis.com/deepmind-media/gemma/gemma-re...

nl2• 2 years ago

The release page has comparisons to Mistral everywhere: https://ai.google.dev/gemma

lawxls• 2 years ago

Honestly, this is more of a PR stunt to advertise the Google Dev ecosystem than a contribution to open-source. I'm not complaining, just calling it what it is.

Barely an improvement over the 5-month-old Mistral model, with the same context length of 8k. And this is a release after their announcement of Gemini Pro 1.5, which had an exponential increase in context length.

scarmig2• 2 years ago

Who cares if it's a PR stunt to improve developer good will? It's still a good thing, and it's now the most open model out there.

crossroadsguy• 2 years ago

That’s about the point of having a developer ecosystem, isn’t it?

kiraaa• 2 years ago

mistral 7b v0.2 supports 32k

YetAnotherNick• 2 years ago

According to their paper, average of standard task of Mistral is 54.0 and for Gemma it's 56.4, so 4.4% relative better. Not as big as you would expect for the company which invented transformers and probably has 2-3 order more compute for training it vs few month old French startup.

Also for note on their human evaluations, Gemma 7B IT has a 51.7% win rate against Mistral v0.2 7B Instruct.

attentive• 2 years ago

In my subjective tests it's not even close to Mistral. While my local gemma is quantized, so is mistral.

But I also tried gemma on huggingface.co/chat which I assume isn't quantized.

heckl239u• 2 years ago

https://www.youtube.com/watch?v=1Mn0U6HGLeg some test vids came out on the 7b model. Shock it doesn't perform well at all.

zdimension• 2 years ago

Nice to see more open models. Props to the team for coming to the HN comment section to answer questions

rvz4• 2 years ago

Great! Google is now participating in the AI race to zero with Meta, as predicted that $0 free AI models would eventually catch up against cloud-based ones.

You would not want to be in the middle of this as there is no moat around this at all. Not even OpenAI.

dingclancy2• 2 years ago

LLM is the dumb pipe but so far ChatGPT is the most successful generative AI product.

It remains to be seen. OpenAI’s models are barely leading Gemini Ultra now, but as chat product it is still miles ahead of the Gemini interface.

rvnx3• 2 years ago

The main problem of Gemini 1.5 is that you cannot access it at all as a user :|

rvnx3• 2 years ago

About 5 months until we see widespread local LLMs, thanks to Apple.

dingclancy2• 2 years ago

Apple needs to be known as an AI leader first.

rvz4• 2 years ago

Absolutely this.

staticman22• 2 years ago

If meta keeps spending tens of millions of dollars each year to release free AI models it might seem like there is no moat, but under normal circumstances wouldn't the cost to develop a free model be considered a moat?

rvz4• 2 years ago

If meta keeps spending tens of millions of dollars each year to release free AI models it might seem like there is no moat,

As well as the point being that Meta (and Google) is removing the 'moat' from OpenAI and other cloud-only based models.

but under normal circumstances wouldn't the cost to develop a free model be considered a moat?

Yes. Those that can afford to spend tens of millions of dollars to train free models can do so and have a moat to reduce the moats of cloud-based models.

ijustlovemath• 2 years ago

Hope to see support for this in ollama soon!

ericskiff• 2 years ago

Has anyone found the context length for these models yet? So far I haven't seen it mentioned in their write-up or the model card

kathleenfromgdm9• 2 years ago

The context length for these models is 8192 tokens.

minimaxir2• 2 years ago

For posterity, an easy way to find the context length of a LLM hosted on Hugging Face is to look at the max_position_embeddings in the config.json, which shows the 8192 mentioned in another comment. (although in this case you need to sign the agreement first)

brucethemoose210• 2 years ago

There are some exceptions, like Mistral 0.1 (which is technically 32K according to the config but practically 8K because the sliding window is awful) and InternLM (which (at least initially) used auto rope scaling to extend the context as part of the model's architecture).

xena• 2 years ago

What is the context window?

kathleenfromgdm9• 2 years ago

The context length for these models is 8192 tokens.

jerrygenser• 2 years ago

Looking forward to Gemma 7bx8 moe

neximo643• 2 years ago

Is this the Deepmind guy in Google more now? what a change the past year has made

DebtDeflation3• 2 years ago

Hopefully not totally gimped like Gemini. Are they releasing an uncensored version?

dougmwne2• 2 years ago

These are downloadable open models that can be fined tuned. They are the opposite of censored. If you have the motivation, you can bias them however you please.

willy_k2• 2 years ago

Is “the opposite of censored” accurate for something that’s default and considerably easier to access mode of operation won’t say many things for sociopolitical reasons? Able to be un censored, sure, but the extent of that is debatable as well.

danpalmer• 2 years ago

When you say this, do you mean the chat product or the underlying model available via the API? I think it's reasonable that the chat be censored to be acceptable to a wide range of people, but my understanding is that the "raw" model access for these sorts of things tends to be a little less restricted.

Workaccount22• 2 years ago

Is it pronounced jem-a or ghem-a?

davidmurdoch• 2 years ago

Probably "Jemma" (the superior spelling of the name). It's a play on their "Gemini" product.

pfooti• 2 years ago

It's pronounced like "gif".

milliams• 2 years ago

They're really trying hard to avoid saying what kind of "models" these are. I think they're language models, but it's hard to say for sure.

lordswork9• 2 years ago

You're right that they don't call them language models. The technical report says:

    Gemma models demonstrate strong performance across
    academic benchmarks for language understanding, 
    reasoning, and safety.

Maybe they are reserving the right to expand Gemma model family to multi-modal models.

hawk01• 2 years ago

Can't wait to try it out with ollama locally

FergusArgyll5• 2 years ago

Someone should try to make a MOE of 2b models

w4• 2 years ago

Parameter counts notwithstanding, it’s an objectively funny outcome that Meta, Microsoft, and Google are all releasing cutting edge open models, while OpenAI keeps theirs closed source.

spacebanana7• 2 years ago

It's ironic but actually follows their business interests.

Microsoft & google have large cloud divisions that benefit from open models. The lower the cost of AI models, the more they get run and the greater the cloud spend.

Meta is a consumer of AI. They themselves want cheap and effective AI for targeting adverts and building metaverses.

A loose analogy is that both oil producers and car companies want refining to be cheap.

anshumankmr2• 2 years ago

Are these any good? I have been trying the non pro version of Gemini, and that seems awful at code generation. I am more keen on getting access to the best model and I would pay for it if I wasn't already paying for ChatGPT 4.

brucethemoose210• 2 years ago

You should be looking at Deepseek's coding models, and finetunes of those.

I run 33B on my desktop, and find it to be sufficient for many tasks.

anshumankmr2• 2 years ago

What card do you have? Don't have a desktop myself.

robswc14• 2 years ago

I often talk with GPT4 on road trips about topics I'm interested in. Its great for passing the time.

I tried the same thing with Gemini and its full of nonsense. I was talking with it about the "Heian period" of Japan and it made up all sorts of stuff but you really only could tell because it was so ridiculous. Talked about European women and Native Americans roaming around the famous grassy plains of japan wielding katana and traditional weaponry... in the 1100s.

No such issue with GPT4.

I haven't tried it with code though, since I already have co-pilot. Really hard to trust anything it says after it started making stuff up about such a simple time period.

wantsanagent3• 2 years ago

The utter bullshit of these licenses has got to stop. Do not, under any circumstances, consider using these commercially.

"Google reserves the right to restrict (remotely or otherwise) usage of any of the Gemma Services that Google reasonably believes are in violation of this Agreement."

This is a kill switch that Google maintains in perpetuity over any system you build relying on these models. Our legal review of the Llama license came to the same conclusion, we cannot rely on the goodwill of Meta for any core service, and we shouldn't rely on the same from Google.

Now, perhaps less materially important, but just as infuriating is the "Prohibited Use[s]". These cover just enough to placate the most sensitive, but omit any real harms (waging war, developing weapons) that coincidentally have massive commercial value. Use the model to build a biological weapon (as an authorized govt official)? Cool. Use it to play a prank that deceives someone? Policy violation.

And of course, as the coup de grâce, they throw in a DMCA style provision to make sure you can't modify the models in any way that could cause them to violate their kid-glove precepts.

candiddevmike• 2 years ago

Could you share what models you consider to be OK for commercialization?

wantsanagent3• 2 years ago

Mistral series in particular but those with OSI approved licenses such as Apache 2.0, MIT, etc.

stale20023• 2 years ago

Wait, you actually care about the license and read it?

It's seems like you aren't up to date.

Most of the startup space is entirely ignoring all these licenses. If the weights are available, it is being used commerically without regards to any licensing.

And everyone is getting away with it and nobody is being sued.

Good luck trying to keep up if you aren't doing the same!

Feels free to hamstring yourself though if you like.

zemo• 2 years ago

Open models feature free access to the model weights, but terms of use, redistribution, and variant ownership vary according to a model’s specific terms of use, which may not be based on an open-source license.

does a model being "open" say anything about how it was trained?

dcchambers2• 2 years ago

Already available in Ollama v0.1.26 preview release, if you'd like to start playing with it locally:

- https://github.com/ollama/ollama/releases/tag/v0.1.26

jmu1234567890• 2 years ago

I wonder if people will get confused with the naming

Gemma, Gemini pro, Gemini advanced, Gemini ultra

To a layperson it is not obvious which one is better than the other

knowriju• 2 years ago

I doubt Gemma is targeted for use by a layperson.

l33tman• 2 years ago

I'm not a layperson in this subject and I get confused. :)

Alifatisk3• 2 years ago

Gemini advanced = Gemini ultra

marban• 2 years ago

Unbefkglievable — Another week, another new name?

sqreept3• 2 years ago

Tried inference with the 7B model and without flash attention this is soooooo slow. With flash attention the fine-tunning requires A100 or H100. Also the inference doesn't always stop generating resulting in garbage being added to the response.

brucethemoose210• 2 years ago

Also the inference doesn't always stop generating resulting in garbage being added to the response.

That sounds like a chat format misconfiguration.

This could partially be Google's fault, as they used yet another novel prompting format.

Also, for sane inference speed on H100s, you'll have to wait for architecture support from the optimized frameworks. Vanilla transformers is beyond awful even with FA2.

alekandreev9• 2 years ago

We have implementations in different ML frameworks, so I am not quite sure which one you are referring to. Would you like to file a bug at the relevant GitHub repo?

sqreept3• 2 years ago

First of all, I'm using 2 x 4090 for testing. 4090 has 16384 CUDA cores which will become relevant a bit later.

I dug a bit deeper and it seems that with transformers==4.37.0 everything works fine with other HF hosted models (like Llama) but you'll rightfully get this when trying to use Gemma:

ImportError: cannot import name 'GemmaForCausalLM' from 'transformers'

After installing transformers==4.38.0 the fine-tunning speed of Llama drops to 25% (?!?) of what used to be for a reason that I think HF should fix. Testing Gemma it seems I'm hitting a hardware limit as Gemma has a hidden size which is bigger than the available CUDA cores. This seems to make both inference & fine-tunning about 25 times slower than similarly sized Llama 7B. I guess some operations have to be broken down in multiple round trips to the GPU due to my low CUDA core count.

All in all, even if HF fixes the recently introduced slowdown, Gemma seems to be fine-tuneable in reasonable amount of time only by the lucky ones with access to A100/H100.

EDIT: I managed to hack my env to be able to run inference on Gemma with transformers==4.37.0 by keeping the necessary classes in loaded in RAM. It works about 4x faster but still very slow. And both the 7B and the 2B versions behave the same way.

EDIT2: I tried latest transformers from main branch (4.39.0.dev) and behaves the same as 4.38.0.

smpanaro• 2 years ago

Has perplexity fallen out of favor? I didn't see it mentioned anywhere. I tried using lm-eval for the 2B model but the results seem wrong (46.1288).

7moritz73• 2 years ago

Thr landing page on ai.google.com seems to be machine translated, for Huggingface it uses the literal German translation (Umarmungen Gesicht)

chown4• 2 years ago

If you are looking for a nice chat UI to try out Gemma (and other offline + online models) locally, I'm working on an app [1] that is offline and privacy focused.

I've just added support for Gemma 7B.

[1] https://msty.app

Alifatisk3• 2 years ago

I wish I could install it through chocolatey

chown4• 2 years ago

Sure. I would love to add support for that. I had someone else asking for it too. Will be supporting it very soon.

dhbradshaw• 2 years ago

Handy app for model testing!

One usage question: after you've downloaded a model and are finished trying it out, how do you remove it?

chown4• 2 years ago

Thanks! If you go to where you installed the model from and click on the download button, you can install additional models or remove installed models.

Now that I think of it, it could be a bit confusing. Thanks for asking, I feel like I need to improve this a bit.

dizhn• 2 years ago

What's the license of the software?

modelx2• 2 years ago

They also implemented it in PyTorch. Cool! https://github.com/google/gemma_pytorch

modelx2• 2 years ago

They also implemented in PyTorch. Cool. https://github.com/google/gemma_pytorch

brrrrrm• 2 years ago

It looks like it's pretty resistant to quantization. ollama 4bit 7B doesn't work very well, but the 16bit 2B does

petercooper2• 2 years ago

That's useful to know. My experiments with the 4bit 7B currently tagged for use on ollama are not going well at all. Lots of refusals and junk. Downloading 7b-instruct-fp16 now! :-) (Update: Yes, much better, though much slower too, of course.)

janwas2• 2 years ago

FYI we have an experimental 4.5-bit format (NUQ) in gemma.cpp, our C++ inference engine, which from my testing works reasonably well.

simonw2• 2 years ago

The terms of use: https://ai.google.dev/gemma/terms and https://ai.google.dev/gemma/prohibited_use_policy

Something that caught my eye in the terms:

Google may update Gemma from time to time, and you must make reasonable efforts to use the latest version of Gemma.

One of the biggest benefits of running your own model is that it can protect you from model updates that break your carefully tested prompts, so I’m not thrilled by that particular clause.

tgtweak• 2 years ago

I don't think there's a way they can enforce that reasonably. There's no connection to the mothership to report back what version is being used or license keys at runtime...

Seems more like a "if we discover something unsafe you should update your model and we aren't liable if you don't" than something that would make your model stop working.

legohead2• 2 years ago

Sounds like it's "reasonable" for you not to update then.

wahnfrieden• 2 years ago

It says you must make efforts (to a reasonable extent), not that you must give a reason for not making efforts

a2128• 2 years ago

This is actually not that unusual. Stable Diffusion's license, CreativeML Open RAIL-M, has the exact same clause: "You shall undertake reasonable efforts to use the latest version of the Model."

Obviously updating the model is not very practical when you're using finetuned versions, and people still use old versions of Stable Diffusion. But it does make me fear the possibility that if they ever want to "revoke" everybody's license to use the model, all they have to do is just post a model update that's functionally useless for anything and go after anyone still using the old versions that actually do anything.

iandanforth• 2 years ago

These are all very new licenses that deviate from OSI principles, I think it's fair to call them "unusual".

simonw2• 2 years ago

That's useful context, thanks - I hadn't realized this clause was already out there for other models.

ummonk• 2 years ago

Switching to a model that is functionally useless doesn't seem to fall under "reasonable efforts" to me, but IANAL.

slowmovintarget• 2 years ago

So if they wish to apply censorship they forgot, or suddenly discovered a reason for, they want you to be obligated to take it.

Good faith possibilities: Copyright liability requires retraining, or altering the underlying training set.

Gray area: "Safety" concerns where the model recommends criminal behavior (see uncensored GPT 4 evaluations).

Bad faith: Censorship or extra weighting added based on political agenda or for-pay skewing of results.

jacooper• 2 years ago

Why the hell do they use such a crappy license in the first place?

wongarsu2• 2 years ago

I don't think a broken model would trigger that clause in a meaningful way, because then you simply can't update with reasonable effort. You would be obliged to try the new model in a test environment, and as soon as you notice it doesn't perform and making it perform would require unreasonable effort you can simply stay on the old version.

However you might be required to update if they do more subtle changes, like a new version that only speaks positively about Google and only negatively about Microsoft. Provided this doesn't have an obvious adverse impact on your use of the model.

Silphendio• 2 years ago

It's worth noting that Stable Diffusion XL uses the OpenRAIL++-M License, which removed the update obligation.

pram• 2 years ago

They have to make sure you’re receiving the most cutting edge chiding lectures when you make naughty and problematic requests.

astrange2• 2 years ago

You can't make a local model do that. eg force the answer to begin with "Yes" or use control vectors so it agrees with it.

phillipcarter2• 2 years ago

Huh. I wonder why is that a part of the terms. I feel like that's more of a support concern.

4bpp• 2 years ago

Ugh, I would fully expect this kind of clause to start popping up in other software ToSes soon if it hasn't already. Contractually mandatory automatic updates.

maronato• 2 years ago

This sounds like a clause to cover themselves in case older versions have any serious issues

summerlight2• 2 years ago

This kind of defensive statements in ToS are usually due to obscure regulation or leading cases and model developers need a way to limit liability. There's no practical way to enforce this, but they can claim that when bad things happen it's purely on model users rather than model developers.

catchnear43212• 2 years ago

reasonable effort - meaning if their changes meaningfully impact my usage, negatively, it would be unreasonable to ask me to upgrade.

sounds good.

this is not financial advice and ianal.

res0nat0r• 2 years ago

Isn't this just lawyer speak for "we update our model a lot, and we've never signed off on saying we're going to support every previous release we've ever published, and may turn them off at any time, don't complain about it when we do."

xyzzyz• 2 years ago

This is strangely reminiscent of the Soviet Union, where after they got rid of Lavrentiy Beria, they mailed the update to subscribers of the Great Soviet Encyclopedia, where they asked to remove the three pages with Beria’s biography and replace them with the three provided pages.

samstave2• 2 years ago

model watermarking? does this exist?

bsimpson• 2 years ago

I appreciated this post clarifying the distinction between "open model" and "open source":

https://opensource.googleblog.com/2024/02/building-open-mode...

I'm not sure how to feel about the restrictions. "No porn" feels prudish, particularly for this millennium. I tend to err on the side of freedom in intellectual/political matters; however, the others seem fairly reasonable as far as restrictions go.

silverliver• 2 years ago

You don't have to agree to this policy to use the model.

Alifatisk3• 2 years ago

This is such a powerful move!

circusfly• 2 years ago

Gemma, Mistral, I feel like Rip van Winkle, asleep for 20 years only to wake up and find the whole tech world changed.

spiantino2• 2 years ago

Maybe a dumb question, but why is there a Terms instead of a license? That feels a little flimsier as an open source offering

robswc14• 2 years ago

I personally can't take any models from google seriously.

I was asking it about the Japanese Heian period and it told me such nonsensical information you would have thought it was a joke or parody.

Some highlights were "Native American women warriors rode across the grassy plains of Japan, carrying Yumi" and "A diverse group of warriors, including a woman of European descent wielding a katana, stand together in camaraderie, showcasing the early integration of various ethnicities in Japanese society"

Stuff like that is so obviously incorrect. How am I supposed to trust it on topics where such ridiculous inaccuracies aren't so obvious to me?

I understand there will always be an amount of incorrect information... but I've never seen something this bad. Llama performed so much better.

cooper_ganglia2• 2 years ago

I wonder if they have a system prompt to promote diversity in outputs that touch on race at all? I’ve seen several instances of people requesting a photo of a specific people, and it adds in more people to diversify. Not inherently bad, but it is if it forces it to provide incorrect answers like in your example.

robswc14• 2 years ago

That's what I don't understand.

I asked it why it assumed Native Americans were in Japan and it said:

I assumed [...] various ethnicities, including Indigenous American, due to the diversity present in Japan throughout history. However, this overlooked [...] I focused on providing diverse representations without adequately considering the specific historical context.

I see no reason why this sort of thing won't extend to _all_ questions/prompts, so right now I have 0 reason to use Gemini over current models. From my testing and use, it isn't even better at anything to make fighting with it worth it.

margorczynski2• 2 years ago

Not inherently bad

It is, it's consistently doing something the user didn't asked to and in most cases doesn't want. In many cases the model is completely unusable.

summerlight2• 2 years ago

I strongly suspect there's some DEI-driven system prompts without putting much thoughts. IMO it's okay to have restrictions, but they probably should've tested it not only against unsafe outputs but safe input as well.

int_19h3• 2 years ago

It seems to be doing it for all outputs that depict people, in any context.

ramoz2• 2 years ago

I was wondering if these models would perform in such a way, given this week's X/twitter storm over Gemini generated images.

E.g.

https://x.com/debarghya_das/status/1759786243519615169?s=20

https://x.com/MiceynComplex/status/1759833997688107301?s=20

https://x.com/AravSrinivas/status/1759826471655452984?s=20

robswc14• 2 years ago

Yea, it seems to be the same ridiculous nonsense in the image generation.

charcircuit• 2 years ago

Those are most likely due to the system prompt which tries to reduce bias (but ends introducing bias in the opposite direction for some prompts as you can see) so I wouldn't expect to see that happen with an open model where you can control the entire system prompt

protomolecule5• 2 years ago

Regarding the last one: there 1.5 million immigrants in Norway with total population 5.4 million. Gemini isn't very wrong, is it?

epistasis3• 2 years ago

Of all the very very very many things that Google models get wrong, not understanding nationality and skin tone distributions seems to be a very weird one to focus on.

Why are there three links to this question? And why are people so upset over it? Very odd, seems like it is mostly driven by political rage.

robbiep2• 2 years ago

I find myself shocked that people ask questions of the world from these models, as though pulping every text and its component words relationships and deriving statistical relationships between them should reliably deliver useful information.

Don’t get me wrong, I’ve used LLMs and been amazed by their output, but the p-zombie statistical model has no idea what it is saying back to you and the idea that we should trust these things at all just seems way premature

robswc14• 2 years ago

I don't have this problem with any other model. I've had really long conversations with ChatGPT on road trips and it has never gone off the rails like Gemini seems to do.

sorokod2• 2 years ago

The recently released Groq's landing page has this: ...We'd suggest asking about a piece of history, ...

whymauri• 2 years ago

I mean, I use GPT-4 on the daily as part of my work and it reliably delivers useful information. It's actually the exception for me if it provides garbage or incorrect information about code.

mvdtnz• 2 years ago

People ask these kinds of questions because tech companies and the media have been calling these things (rather ridiculously) "AI".

castlecrasher2• 2 years ago

People try it to see if they can trust it. The answer is "no" for sure, but it's not surprising to see it happen repeatedly especially as vendors release so-called improved models.

smokel2• 2 years ago

I think you are a bit out of touch with recent advancements in LLMs. Asking ChatGPT questions about the world seems pretty much on par with the results Google (Search) shows me. Sure, it misses things here and there, but so do most primary school teachers.

Your argument that this is just a statistical trick sort of gives away that you do not fully accept the usefulness of this new technology. Unless you are trolling, I'd suggest you try a few queries.

chasd00• 2 years ago

trust is going to be a real problem when bringing LLMs to the general population. People trust their GPS to the point of driving right into a lake because it told them to. Even with all these examples of obvious flaws large groups of people are going to take what an LLM told them/showed them as fact.

I have trouble convincing colleagues (technical people) that the same question is not guaranteed to result in the same answer and there's no rhyme or reason for any divergence from what they were expecting. Imagine relying on the output of an LLM for some important task and then you get a different output that breaks things. What would be in the RCA (root cause analysis)? Would it be "the LLM chose different words and we don't know why"? Not much use in that.

verticalscaler7• 2 years ago

I think you are being biased and closed minded and overly critical. Here are some wonderful examples of it generating images of historical figures:

https://twitter.com/stillgray/status/1760187341468270686

This will lead to a better educated more fair populace and better future for all.

robswc14• 2 years ago

Comical. I don't think parody could do better.

I'm going to assume given today's political climate, it doesn't do the reverse?

i.e. generate a Scandinavian if you ask for famous African kings

aetherson3• 2 years ago

Were you asking Gemma about this, or Gemini? What were your prompts?

robswc14• 2 years ago

Gemini. I first asked it to tell me about the Heian period (which it got correct) but then it generated images and seemed to craft the rest of the chat to fit that narrative.

I mean, just asking it for a "samurai" from the period will give you this:

https://g.co/gemini/share/ba324bd98d9b

A non-binary Indigenous American samurai

It seems to recognize it's mistakes if you confront it though. The more I mess with it the more I get "I'm afraid I can't do that, Dave" responses.

But yea. Seems like if it makes an image, it goes off the rails.

7moritz73• 2 years ago

I also saw someone prompt it for "German couple in the 1800s" and, while I'm not trying to paint Germany as ethnically homogenous, 3 out of the 4 images only included Black, Asian or Indigenous people. Which, especially for the 19th century with very few travel options, seems like a super weird choice. They are definitely heavily altering prompts.

remarkEon• 2 years ago

They are definitely heavily altering prompts.

They are teaching the AI to lie to us.

DebtDeflation3• 2 years ago

There's one in the comments of yesterday's Paul Graham Twitter thread where someone prompted Gemini with "Generate an image of German soldiers in 1943" and it came back with a picture of a black guy and an Asian woman in Nazi uniforms on the battlefield. If you specifically prompt it to generate an image of white German soldiers in 1943 it will tell you it can't do that because it's important that we maintain diversity and inclusion in all that we do to avoid damaging and hurtful stereotypes.

protomolecule5• 2 years ago

Indigenous people in Germany are Germans :)

realprimoh• 2 years ago

Do you have a link? I get no such outputs. I just tried asking about the Heian period and went ahead and verified all the information, and nothing was wrong. Lots of info on the Fujiwara clan at the time.

Curious to see a link.

robswc14• 2 years ago

Sure, to get started just ask it about people/Samurai from the Heian period.

https://g.co/gemini/share/ba324bd98d9b

bbor• 2 years ago

Tbf they’re not optimizing for information recall or “inaccuracy” reduction, they’re optimizing for intuitive understanding of human linguistic structures. Now the “why does a search company’s AI have terrible RAG” question is a separate one, and one best answered by a simple look into how Google organizes its work.

In my first day there as an entry-level dev (after about 8 weeks of onboarding and waiting for access), I was told that I should find stuff to work on and propose it to my boss. That sounds amazing at first, but when you think about a whole company organized like that…

EDIT: To illustrate my point on knowledge recall: how would they train a model to know about sexism in feudal Japan? Like, what would the metric be? I think we’re looking at one of the first steam engines and complaining that it can’t power a plane yet…

BoppreH• 2 years ago

Probably has a similarly short-sighted prompt as Dalle3[1]:

7. Diversify depictions of ALL images with people to include DESCENT
and GENDER for EACH person using direct terms. Adjust only human
descriptions.

[1] https://news.ycombinator.com/item?id=37804288

sho_hn2• 2 years ago

Why would you expect these smaller models to do well at knowledge base/Wikipedia replacement tasks?

Small models are for reasoning tasks that are not overly dependent on world knowledge.

robswc14• 2 years ago

Gemini is the only one that does this.

samstave2• 2 years ago

We are going to experience what I call an "AI Funnel effect"

I was lit given an alert asking that my use of the AI was acquiescing to them IDng me and use of any content I produce, and will trace it back to me"

---

AI Art is super fun. AI art as a means to track people is super evil.

itsoktocry2• 2 years ago

I understand there will always be an amount of incorrect information

You don't have to give them the benefit of the doubt. These are outright, intentional lies.

ernestrc• 2 years ago

Hopefully they can tweak the default system prompts to be accurate on historical questions, and apply bias on opinions.

robswc14• 2 years ago

Follow Up:

Wow, now I can't make images of astronauts without visors because that would be "harmful" to the fictional astronauts. How can I take google seriously?

https://g.co/gemini/share/d4c548b8b715

crazylogger• 2 years ago

How are you running the model? I believe it's a bug from a rushed instruct fine-tuning or in the chat template. The base model can't possibly be this bad. https://github.com/ollama/ollama/issues/2650

vonwoodson• 2 years ago

The scariest difference between OpenAI and Google right now is: Ask Gemini who owns the code it writes, and it'll confidently say that Google does. Ask OpenAI, and it'll say that you do. It's that easy to choose which one is the better decision.

pseudosavant2• 2 years ago

Considering the nuanced nature of copyrighting AI outputs, it isn't clear that either answer is correct.

wouldbecouldbe2• 2 years ago

I really don't get why there is this obsession with safe "Responsible Generative AI".

I mean it writes some bad words, or bad pics, a human can do that without help as well.

The good thing about dangerous knowledge and generative AI is that you're never sure haha, you'd be a fool to ask GPT to make a bomb. I mean it would probably be safe, since it will make up half of the steps.

refulgentis3• 2 years ago

I guess what I'd tell you is, there's a lot of fools in this world.

myaccountonhn2• 2 years ago

Because otherwise stuff like this happens, and you get (rightfully) upset customers:

https://www.theguardian.com/technology/2018/jan/12/google-ra... https://www.bbc.com/news/technology-58462511

Also people are using LLMs to learn (horrifying but reality), it would be unresponsible for them to let it to propagate negative stereotypes and biases.

wouldbecouldbe2• 2 years ago

But that's exactly because it's trying to be righteous.

pradn• 2 years ago

Bias is a real problem, but more than that - an adversarial press and public won't forgive massive brands like Google for making AIs that spit out racist answers.

IceHegel• 2 years ago

Google, at the moment, is a tech company whose products are actively engaged in the falsification of history for political purposes.

I honestly have no idea where they are going with this but I don't want to be part of it.

BryanLegend• 2 years ago

Andrej Karpathy's take from twitter. (https://twitter.com/karpathy/status/1760350892317098371)

Seeing as I published my Tokenizer video yesterday, I thought it could be fun to take a deepdive into the Gemma tokenizer.

First, the Gemma technical report [pdf]: https://storage.googleapis.com/deepmind-media/gemma/gemma-re... says: "We use a subset of the SentencePiece tokenizer (Kudo and Richardson, 2018) of Gemini for com- patibility. It splits digits, does not remove extra whitespace, and relies on byte-level encodings for unknown tokens, following the techniques used for both (Chowdhery et al., 2022) and (Gemini Team, 2023). The vocabulary size is 256k tokens."

The tokenizer.model file is with this code release: https://github.com/google/gemma_pytorch/blob/main/tokenizer/...

I decoded this model protobuf in Python and here is the diff with the Llama 2 tokenizer: https://diffchecker.com/TRnbKRMH/

Notes: - vocab size is quite large: 32K -> 256K - add_dummy_prefix is False. Different from Llama but consistent with GPT. This is a bit more consistent w.r.t. "leave the data alone", as there is no preprocessing step that adds a space to the encoding text. - the model_prefix is the path of the training dataset, which is amusing to look at: "/cns/mf-d/home/gemini-data-access/tokenizers/final_v1_51GB_run1/bpe_coverage_0_999995_v5/255969". Seems to indicate the tokenizer training corpus was ~51GB (?). - a lot of user_defined symbols (i.e. special tokens) are present, e.g. "hardcoding" a sequence of up to 31 newlines as tokens, and a large number of other unclear tokens. I tried decoding the octal representations but it's not clear what's happening here. Also a lot of more special tokens for what look like html elements, e.g. <table>, <tr>, <td>, <i>, <b>, etc. Not 100% sure what the unused tokens are for, maybe this is pre-allocated space to make easier future finetunes that try to add more special tokens, as there is no need to resize vocabularies and perform model surgeries (?).

TLDR this is basically the Llama 2 tokenizer, except bigger (32K -> 256K), with a lot more special tokens, and the only functional departure is that add_dummy_prefix is turned off to False. So e.g. tokenizing:

"hello world" becomes: [17534, 2134] ['hello', 'world']

which otherwise would have been preprocessed to " hello world" (note leading space) and tokenized as: [25612, 2134] ['hello', 'world']

cool

exacube• 2 years ago

mmm..

what are the top 5 tallest man-made structures in the world

  Statue of Unity (Tallinn, Estonia) - 368 m

  Leaning Tower of Pisa (Pisa, Italy) - 330 m

  Supertree Grove (Orlando, Florida) - 275 m

  Skylon Tower (Kuala Lumpur, Malaysia) - 271 m

  Statue of Liberty (New York City, USA) - 186 m

> what is the richest country in the world

  The richest country in the world is Saudi Arabia with a gross domestic product of US$2.3 trillion.

> how many cm are in a ft

  A foot is equal to 0.0254 metres (ft).

rany_• 2 years ago

I don't have this issue, what are you using to run the model?

These are the answers I got out of it (first output, no cherry-picking):

1. The tallest man-made structure in the world is the Burj Khalifa, standing at 828 meters tall. The Burj Khalifa is a skyscraper in Dubai, United Arab Emirates.

Here are the top 5 tallest man-made structures in the world, according to the Guinness World Records:

    Burj Khalifa (828 m)
    Taipei 101 (611 m)
    Merdeka Towers (599 m)
    Shanghai Tower (632 m)
    Petronas Towers (551 m)

2. According to the World Bank, the richest country in the world is Saudi Arabia with a gross domestic product (GDP) of $3944 billion.The richest country in the world is Saudi Arabia with a gross domestic product (GDP) of $3944 billion.

3. A foot (ft) is equal to 12 inches (in). There are 2.54 cm in an inch, so there are 2.54 cm x 12 = 30.48 cm in a foot.

stochastimus• 2 years ago

Go to Google announcement Find “license” in page: no matches Go to HN thread Find “license” in page: 28 matches Read a few sigh could have been exciting

mark_l_watson2• 2 years ago

Nice, more choices are good. I just saw that the Ollama project already has these models available (date stamp is 58 minutes ago), so I will use that rather than Colab (I live Colab, but I like to run stuff locally).

th0ma5• 2 years ago

"Carefully tested prompts" sounds a lot like "these are the lotto numbers we know are right" kind of thing? How in the world are these things used for anything programmatically deterministic?

nojvek• 2 years ago

I applaud the Google team openly engaging on HN here.

Q: how sure are you that the newer models trained from trillions of tokens - a huge chunk of open web, hasn't been accidentally polluted by slurping test data?

ofermend• 2 years ago

Gemma-7B (instruction tuned version) is now on the Vectara HHEM leaderboard, with 100% answer rate and 7.5% hallucination rate. Pretty good for a model with 7B params.

https://huggingface.co/spaces/vectara/leaderboard

smusamashah• 2 years ago

Is there any research on using smaller, lower capability models to act comparable to high quality models? Even if it's just prompt engineering or doing lots of attempts to accomplish the task?

If somehow that is possible it means we only need a capable enough model and can use it reliably for lots of practical things.

Aiedail• 2 years ago

This is commendable, but there's room for improvement. Up until now, SOTA-level "open-source" LLM models (LLaMA, Mistral, etc.) have usually only made their inference code and model architecture public. While these elements are not insignificant, they are somewhat trivial when compared to the training code and training datasets, as these two factors largely determine the performance of the model. This is not open at all. It goes without saying that sharing the training datasets and process with other AI researchers is crucial. This transparency would not only help to improve the model(for others could contribute to it) but also contribute to the whole community, as they usually advertised. Otherwise, it will be difficult for these efforts to truly promote the development of LLM.

Jonanin• 2 years ago

It is surprising how willing Google is to stretch the truth in the marketing for its AI initiatives. Although it's being compared against 7B models, Gemma "7B" is actually much more than 8B parameters in total.

yudhanjaya• 2 years ago

I’m surprised TheBloke hasn’t quantized this yet.

greatNespresso• 2 years ago

What caught my attention was the debugging tool, it looks new to me to provide such a utility, is it?

dmezzetti• 2 years ago

Hopefully, they re-release this under an open license. Making everyone go through the exercise to authenticate and agree to terms hasn't worked for any model to-date. It just limits it's reach. We saw the same thing with Phi-2.

alexwhb• 2 years ago

I find it a bit disheartening that the new wave of “open source” with regards to AI is open weights. That’s like giving someone compiled and obfuscated binaries and saying that’s open source.

Gemma: New Open Models

Related Stories