steinsgate

We found something surprising about ARC AGI 2: the benchmark aiming to measure human-like fluid intelligence. Just enabling a stateful Python tool boosts performance across models. We got > 4x performance improvement in GPT OSS 120B (high). The effect continues well into frontier territory (GPT 5.2) with double digit gains.

We aren't sure whether these gains happen because code execution is a stronger form of verification compared to pure CoT or because it encourages qualitatively different thinking patterns.

Another interesting finding: interleaved thinking, the model capability behind these gains, seems fragile at the infra/client layer. Soft failures can make capable models look much worse than they actually are.

Just wanted to mention that there are indications that universal constants e.g. fine structure constant, gravitational constant etc. are time varying [0].

[0] https://en.wikipedia.org/wiki/Time-variation_of_fundamental_...

Ask HN threads contain a lot of valuable information. So I decided to categorize and organize this information in a GitHub repo. Would appreciate your feedback on the following:

1. Do you find this to be of any value?

2. Any suggestions on what you would like to see in such repositories?

Hi everyone. I recently created Ask HN summaries. It is a Medium publication that turns an Ask HN thread into a blog post and an accompanying GitHub repository. I would love to hear your feedback on it.

I have been working on a NLP project where I needed to identify different forms of the same word. Typically, this is done by Stemming and Lemmatization. These methods are not accurate, and I needed high accuracy in my project. Since I found no libraries/packages that can do this, I decided to write a Python package myself. It works quite well now. It is also trending in /r/python. Feel free to check it out, I would love to hear your feedback.

I have been working on a NLP project where I needed to identify different forms of the same word. Typically, this is done by Stemming and Lemmatization. These methods are not accurate, and I needed high accuracy in my project. Since I found no libraries/packages that can do this, I decided to write a Python package myself. It works quite well now. Feel free to check it out, I would love to hear your feedback.

If someone told me that there would come a day when two of the first three HN posts will be about Bob Dylan and Leonard Cohen, I would have dismissed them as high or delusional. Turns out that I would be very wrong.

Nice work! You said that you avoided machine learning because labeled data is hard to find. What about unsupervised approaches?

Frankly speaking, I am a bit skeptical about pattern matching algorithms for answering questions. It would help if you showed some kind of stats about your algorithm's performance on a diverse question set. For example, you can scrape simple quiz questions (and answers) from quiz sites [1] and report back on the performance.

[1] http://www.quiz-zone.co.uk/questionsbydifficulty/1/0/answers...

It is a free service as of now (at least for me, initially they said I would have to pay after an year, but this never happened). Technically, they don't even have a business model till now. What they have is scale. I think they are afraid that a subscription model will make users churn and they would end up losing the scale. The only other alternative is an ad supported free model or a freemium model. I am wondering if a freemium model would make sense for WhatsApp.

What's the value of WhatsApp? It is the scale of the product. I use it because most of my friends are on it. There are many other alternatives to WhatsApp as pointed out by many in this thread, but none of them have nearly the same traction.

The only business model that allows such scale is the ad supported model. If WhatsApp was subscription based, I am sure they could not have achieved this scale. And I wouldn't have found value in it.

I realize that the ad supported model is what indirectly adds so much value to WhatsApp. Therefore, it makes sense to me to support that model if the model is reasonable enough. They have end to end encryption, which means my content is safe from prying eyes. That's already huge. So I really don't mind if they share my number with Facebook.

another TL;DR : Making mistakes is a great way of learning. To learn from mistakes, the first step is to identify that you have made an error. The second step is to correct it. When we grow older, we become worse at the first step i.e. identifying the error. Without identification, there can be no correction and no learning.

This is one of my favorite posts from Paul Graham. I would add one more point to it based on my experience.

- Make something that has value from day one. Many tech based startups have significant value for users only when many other users are also using it. Tech based political startups, for example, suffer from this problem. These kind of startups are harder to build because getting and retaining the seed users can drain you easily. It is easier if you build something that adds value to the life of the very first user, even if he is the only one using it.