News for hackers. Stuff that matters.

HereBePandas

Re:Gemini 3 Pro Model Card on Tuesday November 18, 2025 @08:41AM (#45965797)
Yes, two things: 1. GPT-5.1 Codex is a fine tune, not the "vanilla" 5.1 2. More importantly, GPT 5.1 Codex achieves its performance when used with a specific tool (Codex CLI) that is optimized for GPT 5.1 Codex. But when labs evaluate the models, they have to use a standard tool to make the comparisons apples-to-apples.

Will be interesting to see what Google releases that's coding-specific to follow Gemini 3.

Re:Gemini 3 Pro Model Card on Tuesday November 18, 2025 @08:30AM (#45965612)
You're right on SWE Bench Verified, I missed that and I'll delete my comment.

GPT 5.1 Codex beats Gemini 3 on Terminal Bench specifically on Codex CLI, but that's apples-to-oranges (hard to tell how much of that is a Codex-specific harness vs model). Look forward to seeing the apples-to-apples numbers soon, but I wouldn't be surprised if Gemini 3 wins given how close it comes in these benchmarks.

Re:Gemini 3 Pro Model Card on Tuesday November 18, 2025 @08:27AM (#45965537)
Not apples-to-apples. "Codex CLI (GPT-5.1-Codex)", which the site refers to, adds a specific agentic harness, whereas the Gemini 3 Pro seems to be on a standard eval harness.

It would be interesting to see the apples-to-apples figure, i.e. with Google's best harness alongside Codex CLI.

Re:Gemini 3 Pro Model Card on Tuesday November 18, 2025 @08:17AM (#45965388)
[comment removed]
Re:Our next-generation model: Gemini 1.5 on Thursday February 15, 2024 @12:02PM (#39385142)
They explicitly address this in page 11 of the report. Basically perfect recall for up to 1M tokens; way better than GPT-4.
Re:Gemini Ultra Released on Thursday February 08, 2024 @10:44AM (#39303262)
Yes
Re:Gemini Ultra Released on Thursday February 08, 2024 @09:37AM (#39302382)
Weird - just tried this and it worked for me.
Re:Gemini – Google DeepMind on Wednesday December 06, 2023 @10:58AM (#38545578)
Tech report seems to hint at the fact that GPT-4 may have had some training/testing data contamination and so GPT-4 performance may be overstated.
I'd be shocked - given the incentives - if it hasn't already happened to a great extent. Many of the types of people Google DeepMind hires are also the types of people hedge funds hire.
Sure, but this time in 2022, you probably wouldn't have let it do that much (or even had this thought).

Progress!

?

BERT was 5 years ago. Of course it's worse than anything introduced more recently (both inside and outside Google).

https://en.m.wikipedia.org/wiki/BERT_(language_model)

Re:Democracy Is the Solution to Vetocracy on Sunday June 04, 2023 @04:33PM (#36189322)
The US has had the Bill of Rights and its constitutional system / property protection for a long time but only recently has had the degree of NIMBYism that it's had.

It's easy to pretend the bug is actually an important and noble feature, but sometimes it's just a bug that needs fixing.

Re:Democracy Is the Solution to Vetocracy on Sunday June 04, 2023 @04:24PM (#36189246)
What kinds of specific policies do you have in mind?

Seems like the reason people in the US don't build houses in the empty patches in the US is because the economics of agglomeration mean much greater efficiency/economic growth when you build them in existing large cities. How do you fight this without making the country much less economically dynamic?

Re:Statement on AI Risk on Tuesday May 30, 2023 @12:13PM (#36126609)
> If only we would fight for the real issues like...

I've heard these arguments many times and they never make sense to me. Most of the people I know working on AI do so precisely because they want to solve the "real issues" like climate change and believe that radically accelerating scientific innovation via AI is the key to doing so.

And some fraction of those people also worry that if AI -> AGI (accidentally or intentionally), then you could have major negative side effects (including extinction-level events).

Re:Jane Street and the OCaml Compiler (2018) [video] on Thursday March 26, 2020 @05:59AM (#22692032)
Two Sigma and Renaissance (albeit not Medallion) primarily invest outside capital.
Re:TensorFlow 2.0 Deep Learning Video Tutorial on Friday March 29, 2019 @08:35PM (#19526783)
Doesn’t work on mobile (iOS) for me.