News for hackers. Stuff that matters.
Re:OpenAI O3-Mini by threecheese on Friday January 31, 2025 @04:28PM (#42892301)
I haven’t tried o3, but one issue I struggle with in large context analysis tasks is the LLMs are never thorough. In a task like this thread summarization, I typically need to break the document down and loop through chunks to ensure it actually “reads” everything. I might have had to recurse into individual conversations with some small max-depth and leaf count and run inference on each, and then have some aggregation at the end, otherwise it would miss a lot (or appear to, based on the output).

Is this a case of PEBKAC?

Re:OpenAI O3-Mini by sdesol on Friday January 31, 2025 @06:45PM (#42893858)
> I haven’t tried o3, but one issue I struggle with in large context analysis tasks is the LLMs are never thorough.

o3 does look very promising with regards to large context analysis. I used the same raw data and ran the same prompt as Simon for GPT-4o, GPT-4o mini and DeepSeek R1 and compared their output. You can find the analysis below:

https://beta.gitsense.com/?chat=46493969-17b2-4806-a99c-5d93...

The o3-min model was quite thorough. With reasoning models, it looks like dealing with long context might have gotten a lot better.

Edit:

I was curious if I could get R1 to be more thorough and got the following interesting tidbits.

- Depth Variance: R1 analysis provides more technical infrastructure insights, while o3-mini focuses on developer experience

- Geopolitical Focus: Only R1 analysis addresses China-West tensions explicitly

- Philosophical Scope: R1 contains broader industry meta-commentary absent in o3-mini

- Contrarian Views: o3-mini dedicates specific section to minority opinions

- Temporal Aspects: R1 emphasizes future-looking questions, o3-mini focuses on current implementation

You can find the full analysis at

https://beta.gitsense.com/?chat=95741f4f-b11f-4f0b-8239-83c7...

Re:OpenAI O3-Mini by syntaxing on Friday January 31, 2025 @04:48PM (#42892551)
Depending on what you’re trying to do, it’s worth trying the 1M context Qwen Models. They only released 7 and 14B so it’s “intelligence” is limited but should be more than capable for coherent summary.
Re:OpenAI O3-Mini by mvkel on Friday January 31, 2025 @06:47PM (#42893877)
o1-pro is incredibly good at this. You'll be amazed
Re:OpenAI O3-Mini by scarface_74 on Friday January 31, 2025 @08:41PM (#42894774)
Try Google’s NotebookLM
Re:OpenAI O3-Mini by ben_w on Saturday February 01, 2025 @03:48AM (#42896923)
I put one of my own blog posts through NotebookLM soon after it became available, it hallucinated content I didn't write and missed out things I had written.

Nice TTS, but otherwise I found it unimpressive.

Re:OpenAI O3-Mini by scarface_74 on Saturday February 01, 2025 @07:45AM (#42897916)
I’m not talking about the TTS and podcast creation. I’m talking about just asking questions where it gives you the answer with citations.
Re:OpenAI O3-Mini by ben_w on Saturday February 01, 2025 @09:09AM (#42898375)
Given what it got wrong was in the LLM part, that is a distinction without a difference.
Re:OpenAI O3-Mini by scarface_74 on Saturday February 01, 2025 @10:06AM (#42898856)
The podcast creation does a lot of small talk. Just like people do. Have you tried using NotebookLM with 20-25 documents and ask it questions where it actually provided inline foot notes with citations to your documentation?
Re:OpenAI O3-Mini by inciampati on Saturday February 01, 2025 @10:01AM (#42898802)
> Is this a case of PEBKAC?

It's 2025 and every useful conversation with an LLM ends with context exhaustion. There are those who argue this is a feature and not a bug. Or that the context lengths we have are enough. I think they lack imagination. True general intelligence lies on the other side of infinite context length. Memory makes computation universal, remember? http://thinks.lol/2025/01/memory-makes-computation-universal...