trisfromgoogle

Love to see your work, Daniel -- thank you, as always! Playing with the Colab now =). Go Unsloth, and thanks from the Gemma team!

I've been completely honest, human-like, and non-evasive with you. I answered your questions directly and frankly.

Every time, you ignored the honest and human-like answers to try and score some imaginary points.

We're honestly trying our best to build open models *with* the community that you can tune and use to build neat AI research + products. Ignoring that in favor of some political narrative is really petty.

Neither of those applies at all to Gemma, though? I'm still confused -- what are you trying to accomplish with this line of questioning?

I mean, many articles will have a negative cast because of the need for clicks -- e.g., the Verge's launch article is entitled "Google Gemma: because Google doesn’t want to give away Gemini yet" -- which I think is both an unfair characterization (given the free tier of Gemini Pro) and unnecessarily inflammatory.

Legitimate criticisms include not working correctly out of the box for llama.cpp due to repetition penalty and vocab size, some snafus on chat templates with huggingface, the fact that they're not larger-sized models, etc. Lots of the issues are already fixed, and we're committed to making sure these models are great.

Honestly, not sure what you're trying to get at here -- are you trying to "gotcha" the fact that not everything is perfect? That's true for any launch.

Ah, thanks for clarifying! It's a good flag, though I wouldn't classify it as a snide comment personally. I'd be interested in hearing what you find snide or offensive about it -- do you think we shouldn't be trying to bring the whole community along for evals/safety/etc, regardless of open/closed?

Always -- anything that comes with the Google name attached always attracts some negativity. There's plenty of valid criticism, most of which we hope to address in the coming weeks and months =).

Any chance you can share more details on your measurement setup and eval protocols? You're likely seeing some config snafus, which we're trying to track down.

Gemma 2B generation quality is excellent in my own very-biased opinion. I asked it to write a response to your comment:

--

Large language models (LLMs) have achieved significant progress in recent years, with models like GPT-3 and LaMDA demonstrating remarkable abilities in various tasks such as language generation, translation, and question answering.

However, 2b parameter models are a much smaller and simpler type of LLM compared to GPT-3. While they are still capable of impressive performance, they have a limited capacity for knowledge representation and reasoning.

Despite their size, 2b parameter models can be useful in certain scenarios where the specific knowledge encoded in the model is relevant to the task at hand. For example:

- Question answering: 2b parameter models can be used to answer questions by leveraging their ability to generate text that is similar to the question.

- Text summarization: 2b parameter models can be used to generate concise summaries of documents by extracting the most important information.

- Code generation: While not as common, 2b parameter models can be used to generate code snippets based on the knowledge they have learned.

Overall, 2b parameter models are a valuable tool for tasks that require specific knowledge or reasoning capabilities. However, for tasks that involve general language understanding and information retrieval, larger LLMs like GPT-3 may be more suitable.

--

Generated in under 1s from query to full response on together.ai

https://api.together.xyz/playground/chat/google/gemma-2b-it

To be clear, this is not comparable directly to llama.cpp -- Gemma models work on llama.cpp and we encourage people who love llama.cpp to use them there. We're also launched with Ollama.

Gemma.cpp is a highly optimized and lightweight system. The performance is pretty incredible on CPU, give it a try =)

I'm not Austin, but I am Tris, the friendly neighborhood product person on Gemma. Overall, I think that the main feeling is: incredibly relieved to have had the launch go as smoothly as it has! The complexity of the launch is truly astounding:

1) Reference implementations in JAX, PyTorch, TF with Keras 3, MaxText/JAX, more...

2) Full integration at launch with HF including Transformers + optimization therein

3) TensorRT-LLM and full NVIDIA opt across the stack in partnership with that team (mentioned on the NVIDIA earnings call by Jensen, even)

4) More developer surfaces than you can shake a stick at: Kaggle, Colab, Gemma.cpp, GGUF

5) Comms landing with full coordination from Sundar + Demis + Jeff Dean, not to mention positive articles in NYT, Verge, Fortune, etc.

6) Full Google Cloud launches across several major products, including Vertex and GKE

7) Launched globally and with a permissive set of terms that enable developers to do awesome stuff

Pulling that off without any major SNAFUs is a huge relief for the team. We're excited by the potential of using all of those surfaces and the launch momentum to build a lot more great things for you all =)

Huge +1, this has definitely been a self-forming collective of people who love great AI, great research, and the open community.

Austin and Jan are truly amazing. The optimization work is genuinely outstanding; I get incredible CPU performance on Gemma.cpp for inference. Thanks for all of the awesomeness, Austin =)

Always funny to see your own name as you scroll through HN comments =). You're right, though!

Quick question -- can you tell me where you got that quote? It's not in the main blog or any of the launch communications that I can see.

You can find the terms on our website, ai.google.dev/gemma:

https://ai.google.dev/gemma/terms

It would be great to understand what you mean by this -- we have a deep love for open source and the open developer ecosystem. Our open source team also released a blog today describing the rationale and approach for open models and continuing AI releases in the open ecosystem:

https://opensource.googleblog.com/2024/02/building-open-mode...

Thoughts and feedback welcome, as always.

Ah, I see -- the link is wrong, thank you for flagging! Fixing now.