News for hackers. Stuff that matters.
Re:Gemma.cpp: lightweight, standalone C++ inference engine f... by trisfromgoogle on Friday February 23, 2024 @03:07PM (#39485458)
Always -- anything that comes with the Google name attached always attracts some negativity. There's plenty of valid criticism, most of which we hope to address in the coming weeks and months =).
Re:Gemma.cpp: lightweight, standalone C++ inference engine f... by verticalscaler on Friday February 23, 2024 @03:08PM (#39485479)
Can you without addressing just acknowledge some of it here? Specific examples? =)

> not to mention positive articles in NYT, Verge, Fortune, etc.

You in fact are mentioning them and only them. I was wondering if you can simply mention the negative ones. Otherwise it sort of sounded at first like its all roses. ;)

Re:Gemma.cpp: lightweight, standalone C++ inference engine f... by trisfromgoogle on Friday February 23, 2024 @03:28PM (#39485682)
I mean, many articles will have a negative cast because of the need for clicks -- e.g., the Verge's launch article is entitled "Google Gemma: because Google doesn’t want to give away Gemini yet" -- which I think is both an unfair characterization (given the free tier of Gemini Pro) and unnecessarily inflammatory.

Legitimate criticisms include not working correctly out of the box for llama.cpp due to repetition penalty and vocab size, some snafus on chat templates with huggingface, the fact that they're not larger-sized models, etc. Lots of the issues are already fixed, and we're committed to making sure these models are great.

Honestly, not sure what you're trying to get at here -- are you trying to "gotcha" the fact that not everything is perfect? That's true for any launch.

Re:Gemma.cpp: lightweight, standalone C++ inference engine f... by verticalscaler on Friday February 23, 2024 @03:37PM (#39485795)
I mean, take for example Paul Graham the guy who made this website you're posting on:

  "Wow, Gemini is a joke. 

  The ridiculous images generated by Gemini aren't an anomaly. 

  They're a self-portrait of Google's bureaucratic corporate culture.

  The bigger your cash cow, the worse your culture can get without driving you out of business. And Google's cash cow, search advertising, is one of the biggest the world has ever seen." - https://twitter.com/paulg/status/1760078920135872716
Are these legitimate criticisms? I don't think he needs clicks.

  "The AI behavior guardrails that are set up with prompt engineering and filtering should be public — the creators should proudly stand behind their vision of what is best for society and how they crystallized it into commands and code.

  *I suspect many are actually ashamed*

  The thousands of tiny nudges encoded by reinforcement learning from human feedback offer a lot more plausible deniability, of course." 
  
    - https://twitter.com/ID_AA_Carmack/status/1760360183945965853
Neither does John Carmack.
Re:Gemma.cpp: lightweight, standalone C++ inference engine f... by summerlight on Friday February 23, 2024 @03:55PM (#39485976)
It looks like you're trying to get some sort of "confession" from relevant people based on recent memes against the company? The reality is likely that the developers sincerely believe in the value of this product and are proud of its launch. You're just adding uninteresting, irrelevant noise to the discussion and you probably won't get what you want.
Re:Gemma.cpp: lightweight, standalone C++ inference engine f... by trisfromgoogle on Friday February 23, 2024 @03:50PM (#39485920)
Neither of those applies at all to Gemma, though? I'm still confused -- what are you trying to accomplish with this line of questioning?
Re:Gemma.cpp: lightweight, standalone C++ inference engine f... by verticalscaler on Friday February 23, 2024 @04:20PM (#39486252)
I'm confused about your confusion ("Still confused"? When did you suddenly become confused? First you've mentioned this).

Gemini is a nerfed, for lack of better term, model released by Google to overwhelming negative response from the public. Not just the click hungry press, users. Including highly knowledgable ones.

Said nerfing "a self-portrait of Google's bureaucratic corporate culture" according to Paul Graham.

Same company. Same culture. Releases a toy open model, a mini Gemini. The concerns around it are the same.

> Launched globally and with a permissive set of terms that enable developers to do awesome stuff

I think by permissive set of terms you're referring to the license, not the model itself. Not entirely sure how developers can trust the model itself given recent events. Seems like a reasonable line of questioning.

I'm trying to establish whether honest human-like non-evasive communication is on offer along with the model so developers might contemplate trusting your work output.

Otherwise, besides being a technical curio, unclear what anybody can do with it. Most wouldn't feel comfortable rolling a Gemma model into a product without a lot more clarity.

Re:Gemma.cpp: lightweight, standalone C++ inference engine f... by trisfromgoogle on Friday February 23, 2024 @04:56PM (#39486624)
I've been completely honest, human-like, and non-evasive with you. I answered your questions directly and frankly.

Every time, you ignored the honest and human-like answers to try and score some imaginary points.

We're honestly trying our best to build open models *with* the community that you can tune and use to build neat AI research + products. Ignoring that in favor of some political narrative is really petty.

Re:Gemma.cpp: lightweight, standalone C++ inference engine f... by SquareWheel on Friday February 23, 2024 @06:02PM (#39487266)
That user is clearly not working with the principle of charity in mind. I've flagged their comments, and would suggest disengaging with them.
Re:Gemma.cpp: lightweight, standalone C++ inference engine f... by verticalscaler on Friday February 23, 2024 @05:34PM (#39487013)
> not to mention positive articles in NYT, Verge, Fortune, etc.

> There's plenty of valid criticism, most of which we hope to address in the coming weeks and months =).

> many articles will have a negative cast because of the need for clicks

This simply comes across as, at best, spin. Given what just happened with Gemini I urge you not to communicate in this style.

> doesn't apply at all to Gemma

That would be the crux of the matter. As a sibling comment states:

> a larger unreleased version of it (Gemma) is (likely) used as part of the Gemini product

There is no reason for anybody outside Google to think otherwise, these are blackboxes.

> Ignoring that in favor of some political narrative is really petty.

You can have assume any politics you wish, the issues with Gemini have already been acknowledged by Google along with an apology.

https://blog.google/products/gemini/gemini-image-generation-...

  over time, the model became way more cautious than we intended and refused to answer certain prompts entirely — wrongly interpreting some very anodyne prompts as sensitive.
There is no information for me or anybody else to go on that your Gemma models are not nerfed - whether maliciously or accidentally nerfed - is immaterial.

Instead of addressing that you and you alone eventually brought it back to politics.

> We're honestly trying our best to build open models with the community

Amongst other things this requires much more trust and transparency and an altogether different communication style.

Re:Gemma.cpp: lightweight, standalone C++ inference engine f... by wokwokwok on Friday February 23, 2024 @10:35PM (#39488918)
Let it go.

You’re being flagged and downvoted on your threads because you’re being unreasonable, not because of some moderator level conspiracy.

The argument you’re making is not falsifiable.

If you want to pursue this line of argument, you need to pause, think about how you can confidently and unambiguously make substantive claims; for example, substantive examples from actually using these models.

The way you’re currently arguing is not meaningful or compelling.

Re:Gemma.cpp: lightweight, standalone C++ inference engine f... by jph00 on Friday February 23, 2024 @04:42PM (#39486497)
These comments appear to be about Gemini's image generation, IIUC. Gemma, however, is a language model -- whilst I believe that a larger unreleased version of it is used as part of the Gemini product, it doesn't seem relevant to these criticisms. Also, the Gemma base model is released, which doesn't AFAIK contain any RLHF.

The impression I have is that you're using the release of Gemma to complain about tangentially related issues about Google and politics more generally. The HN guidelines warn against this: "Eschew flamebait. Avoid generic tangents... Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something. Please don't use Hacker News for political or ideological battle."

Re:Gemma.cpp: lightweight, standalone C++ inference engine f... by Permik on Friday February 23, 2024 @07:13PM (#39487807)
I thought that reflecting what went poorly is really informative and inspiring! It really shows how you are taking this seriously, and iterating on a great project that you're building better and better, thanks for the insight and for a peek behind the curtain
Re:Gemma.cpp: lightweight, standalone C++ inference engine f... by cornel_io on Friday February 23, 2024 @09:08PM (#39488486)
I think he's trying to bring up the racial image gen bias stuff that's going on with Gemini, but for some reason won't say it. He also doesn't appear to realize there is a difference between the two products/teams/launches...
Re:Gemma.cpp: lightweight, standalone C++ inference engine f... by jart on Friday February 23, 2024 @10:27PM (#39488885)
> Legitimate criticisms include not working correctly out of the box for llama.cpp

I don't think that's a legitimate criticism. Especially not for something that just launched. You should be helping your own project first, before you help others. For example, it'd be nice to see AVX2 work as well as AVX512 with gemma.cpp.