Sometimes I wonder if _they’re_ sapient.
One interesting trick you can do with LLMs is to require an ensemble of solutions sampled at higher temperature, and choose the most consistent answer. This boosts the model by a huge margin and provides confidence scores. So I see it improving by external means.
Sorry for adding to the dogpile, but that is a big claim without anything to back it up. It's been clear for a while now that we have no objective measure of the ability of language models. See, for example, the discussion in this article:
What Will it Take to Fix Benchmarking in Natural Language Understanding?
https://aclanthology.org/2021.naacl-main.385/
Especially so with the shift to large language models that can only realistically be created by big corporations with vast resources and no interest in scientific questions, the whole business of language modelling has escaped from the confines of science. We just have no way to know how good, or bad, those models really are. All we have is nothing more than feelings: the subjective opinion of people who use them, and then tweet about them.
So let's just not make such grandiose claims, right? As the authors of the paper above point out:
Performance on popular benchmarks is extremely high, but experts can easily find issues with high-scoring models. (...) Ample evidence has emerged that the systems that have topped these leaderboards can fail dramatically on simple test cases that are meant to test the very skills that the leaderboards focus on (McCoy et al., 2019; Ribeiro et al., 2020). This result makes it clear that our systems have significant room to improve.
What I would want from a tool like this is that it should base its reasoning on more than just the variable names, and if it cannot deduce what the code does, just tell me that instead of confidently bullshitting something.
The given code is a C-style implementation of the "fast inverse square root" algorithm. This algorithm computes an approximation of the inverse square root of a given number (1/sqrt(x)). The code does not actually compute the floor of a floating-point number, so the function name "Floor" is misleading.
https://news.ycombinator.com/item?id=35164407
Edit: for anybody who is claiming GPT "understands" the Fast Inverse Square Root algorithm, try changing all the variable names not just one.
It is still very interesting that it is able to recognize the original at all and chase down a plagiarized explanation but this isn't magic. That function has a wiki entry and is widely discussed.
This is a leap in semantic search and summarization not a NN that can code.
Reminds me of many “devs” I’ve worked with in the past 10 years. Kidding of course, but also not. ducks
But the worst are people that post answers generated by ChatGPT.
https://meta.stackoverflow.com/questions/421831/temporary-po...
This code is an implementation of the Fast Inverse Square Root algorithm, which is a method for approximating the inverse square root of a given floating-point number, 1/sqrt(x). This method is well-known for its use in the 3D computer graphics of the video game Quake III Arena, developed by id Software in 1999.
Btw: I don’t quite get the need for these frontends. I literally input this to gpt: “ Wxplain this code:” (Yes, with typo)
You need your own OpenAI API key to use GPT directly. There's some value in not managing that yourself.
EDIT: but GPT4, GPT5 and GPT6 will still get stuff wrong. But so do humans. I don't think the existence of errors is a problem, we just have to get accurate enough to get useful.
The reason is the variable names exist solely to define connections between inputs and outputs, and renaming them is sane as renumbering a graph.
The initial paragraph is wrong though. It takes markdown headings as input, it doesn't generate markdown headings. This confusion makes it a lot less trustworthy.
Is that the kind of flaw you were pointing out?
https://whatdoesthiscodedo.com/g/24c7ac3
Original: https://gitlab.com/taco.steemers/md_toc/-/blob/master/genera...
Currently it looks like it is only capable of writing text that states everything as a matter of pure fact, probably because most training data is written by authors who have already checked that their statements are correct (or at least believe them to be).
That is the famous fast inverse square root [1] from Quake, with a handful of completely inconsequential changes. I renamed the function from "Q_rsqrt" to "Floor", and I renamed one variable from "threehalfs" to "magicNumber". I also removed the various comments. The intent was to demonstrate the flaws in the current "guess the next word" style algorithm of large language models. The result:
-----
The code is an implementation of the floor function for floating-point numbers. The floor function returns the largest integer value that is less than or equal to a given input number.
The code works by first dividing the input number by 2 and storing the result in a variable called x2. It then stores the original input number in a variable called y.
Next, it performs some bit manipulation on the binary representation of y to obtain an approximation of its inverse square root. This is done using the following "magic number" constant: 0x5f3759df. The resulting value is stored in i.
The code then converts the binary representation of i back into a float value and multiplies it by a correction factor, which is based on the original input number and the inverse square root approximation. This correction factor is computed as follows:
magicNumber - (x2 * y * y)
Finally, the resulting value is returned from the function as the floor of the input number.
-----
[1] - https://en.wikipedia.org/wiki/Fast_inverse_square_root