fatso784

This looks vibe-written. Why should I trust it then? Some guys at Stanford decided to feed Claude their ideas and ask it to write a lengthy textbook for them?

Notational programming explored this, also with quantum computing, except the spatial notation was interleaved in a well-established 1d language (Python): https://dl.acm.org/doi/10.1145/3526113.3545619

My problem with this kind of work is—-obviously they do. Did anyone seriously think otherwise? I’m shocked why these are even questions deserving scientific scrutiny. Have people truly lost their critical thinking that badly already?

Thanks!

Sorry, but these people are not victims. I went through a tech PhD; it was well-known how fast the wind changes and the trendy topic falls by the wayside. Big data and crowdsourcing pre-AI boom, then ML, then ethics of AI (around 2020-21), now we are at AI agents which is inherently more product-oriented. Beyond this, we had COVID and a now brutal tech market, despite the US economy seemingly way up and the AI craze continuing. If you went into a CS PhD thinking nothing changes and people will beg you to take cushy salary positions with total freedom, then you simply didn’t do your research.

Wow. This actually disproves a key subtext of the match mentioned by some commentators: that Ding failed to convert winning positions to wins. Instead, it shows that Ding converted more often than Gukesh. The fact that Gukesh won seems more a statistical anomaly in light of this evidence. We are indeed probably post-hoc rationalizing the winner.

“Do politics have artifacts?” was the rejoinder article. IMO that article should be as widely read as the main one, because it provides a warning to those who take the main one as gospel. Link: https://journals.sagepub.com/doi/abs/10.1177/030631299029003...

This seems like a sheets implementation of something like ChainForge (https://github.com/ianarawjo/ChainForge).

It's curious that Anthropic is entering the LLMOps tooling space ---this definitely comes as a surprise to me, as both OpenAI and HuggingFace seem to avoid building prompt engineering tooling themselves. Is this a business strategy of Anthropic's? An experiment? Regardless, it's cool to see a company like them throw their hat into the LLMOps space beyond being a model provider. Interested to see what comes next.

The original poster making this claim used a t-test to compare means (https://x.com/RobLynch99/status/1734278713762549970?s=20). Turns out the data is not normally distributed, making a t-test worthless (https://www.statology.org/t-test-assumptions/).

There might be other tests to do, but for this specific setup, the claim has been debunked.

Can’t reproduce this. See for yourself: https://x.com/IanArawjo/status/1734307886124474680?s=20

Inspectable evaluation flow in ChainForge: https://chainforge.ai/play/?f=2yvqkpe1vpus8

This seems like a sheets implementation of something like ChainForge (https://github.com/ianarawjo/ChainForge).

It's curious that Anthropic is entering the LLMOps tooling space ---this definitely comes as a surprise to me, as both OpenAI and HuggingFace seem to avoid building prompt engineering tooling themselves. Is this a business strategy of Anthropic's? An experiment? Regardless, it's very cool to see a company like them throw their hat into the LLMOps space beyond being a model provider. Interested to see what comes next.

ChainForge lets you do this, and also setup ad-hoc evaluations with code, LLM scorers, etc. It also shows model responses side-by-side for the same prompt: https://github.com/ianarawjo/ChainForge