News for hackers. Stuff that matters.

A comment on Gwern Branwen – How an Anonymous Researcher Predicted AI's Trajectory.

Re:Gwern Branwen – How an Anonymous Researcher Predicted AI'... by nutanc on Thursday November 14, 2024 @05:46AM (#42134935)
Experimenting with creating semantic chunks of large podcasts. Got the following chunks, https://gist.github.com/nutanc/a9e6321649be5ea9806b4450b0bd6...

Dwarkesh has 18 splits. https://www.dwarkeshpatel.com/i/151435243/timestamps

I got 171. So roughly 9 context discussions in one time stamp.

[ Parent | on HN ]
Re:Gwern Branwen – How an Anonymous Researcher Predicted AI'... by maujim on Thursday November 14, 2024 @02:59PM (#42140515)
What did you use to create the chunks?
[ Parent | on HN ]
Re:Gwern Branwen – How an Anonymous Researcher Predicted AI'... by nutanc on Friday November 15, 2024 @01:02AM (#42144308)
It's a new approach I am experimenting with.

https://gpt3experiments.substack.com/p/a-new-chunking-approa...

[ Parent | on HN ]
Re:Gwern Branwen – How an Anonymous Researcher Predicted AI'... by maujim on Friday November 15, 2024 @02:05PM (#42149835)
Interesting approach. Have you tried using existing clustering algorithms to determine the chunks instead of using distance or slope as a measure?
[ Parent | on HN ]