News for hackers. Stuff that matters.
Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by layman51 on Monday February 10, 2025 @05:11PM (#43005956)
Right, the other side is when you end up with rediscoveries of the same ideas. The example that comes to my mind is when a medical researcher found the trapezoidal rule for integration again[1].

[1]: https://fliptomato.wordpress.com/2007/03/19/medical-research...

Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by sitkack on Monday February 10, 2025 @08:08PM (#43007676)
I think that shows how great the trapezoidal rule is. I feel like this is brought out too many times, that now it is used to make fun of people. It is 18 years old at this point.
Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by FartyMcFarter on Monday February 10, 2025 @08:13PM (#43007723)
I mean, it sort of deserves being made fun of. 18 years ago Google existed, surely you'd search for "area under a curve" before going through all the effort of writing a paper reinventing integrals?

Edit: actually I just realised the paper was written in 1994, but still, peer review existed and so did maths books...

Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by fragmede on Tuesday February 11, 2025 @02:09AM (#43009872)
I'd argue this is an argument against purely peer review, as her peers also weren't mathematicians.

Some of us when learning calculus wonder if we'd been alive before it was invented, if we'd be smart enough to invent it. Dr. Tai provably was. (the trapezoid rule, anyway) So I choose to say xkcd 1053 to her, rather than bullying her for not knowing advanced math.

Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by eru on Tuesday February 11, 2025 @08:02AM (#43012275)
> Dr. Tai provably was.

No, we have no proof of that. We just know that she published a paper explaining the trapezoidal rule.

(A) That approximation for 'nice' curves was known long before calculus. Calculus is about doing this in the limit (or with infinitesimals or whatever) and also wondering about mathematical niceties, and also some things about integration. (B) I'm fairly certain she would have had a bit of calculus at some point in her education, even if she remembered it badly enough, to think she found something new.

Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by fragmede on Wednesday February 12, 2025 @05:17AM (#43023846)
I mean, it's possible she reinvented the wheel because what she really needed in her life is for the math department to laugh at her, but that seems far fetched to me.
Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by ashoeafoot on Tuesday February 11, 2025 @02:15AM (#43009920)
Does it make you less of a peer to others who found it before ? At leas the author showed ability to think creative for himself , not paralyzed by the great stagnation like the rest of us.
Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by mpweiher on Tuesday February 11, 2025 @02:58AM (#43010178)
Herself. Mary Tai.

And what makes you less of a peer is not knowing the basics. And being so unaware of apparently not knowing the basics, and/or uninterested, that you don't bother to check something that is highly checkable.

Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by eru on Tuesday February 11, 2025 @07:59AM (#43012247)
Even worse: you didn't just think so in private, but you decided to publish your 'great' discovery.
Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by fooker on Tuesday February 11, 2025 @01:02PM (#43015948)
The blame is on the reviewers.

This is why peer review exists. One can not known everything themselves. It's fairly common for CS paper submissions to reinvent algorithms and then tone down the claims after reviewers suggest that variants already exist.

Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by fragmede on Tuesday February 11, 2025 @01:05PM (#43015990)
> highly checkable

in 1994?

Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by reshlo on Wednesday February 12, 2025 @04:26AM (#43023520)
Calculus textbooks existed in 1994. It just took me 30 seconds to find “area under a curve” in the index of the one I own, and another 30 seconds to find the section on numerical integration, which describes the trapezoidal approximation.
Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by fragmede on Wednesday February 12, 2025 @05:16AM (#43023839)
So you already know the particular area of the larger topic of mathematics that you need to look for, you already have a textbook for that particular subject in your possession, meaning you don't need to go to the library and somehow figure out the right book to choose from the thousands the 510 section; you know what you are looking for exists, and then you aren't surprised you can find it?

I know how to find the area under the curve, but there's so much biology I don't know jack shit about. Back in 1994, It would have been hopeless for me to know the Michaelis-Menten model even existed if it had been relevant to my studies in computer science. That you can right click on those words in my comment in 2025 and get an explanation of what that is and can interrogate chatgpt to get a rigorous understanding of it shouldn't make it seem like finding someone in the math department to help you in 1994 was easier than just thinking logically and reinventing the wheel.

Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by Joker_vD on Wednesday February 12, 2025 @11:07AM (#43026726)
There is thing called "higher education". Ostensibly one of its chief purposes is to arm you with all that interconnected knowledge and facts that is useful in your chosen field of study. To the boot, you get all of that from several different human beings who you can converse with, to improve the scope and precision of the knowledge you're receiving. You know, "standing on the shoulders of the giants" and all of that stuff?
Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by fragmede on Wednesday February 12, 2025 @03:31PM (#43029446)
ostensibly.
Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by reshlo on Wednesday February 12, 2025 @01:10PM (#43028009)
> So you already know the particular area of the larger topic of mathematics that you need to look for

So did the author of the paper. The paper’s title itself mentions the area under a curve. It would not have been difficult to find information about how to calculate an approximation of the area under a curve in the library.

Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by rchard2scout on Tuesday February 11, 2025 @03:46AM (#43010480)
The "18 years" probably refers to the date since the linked blogpost was published, March 2007.
Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by mgens on Tuesday February 11, 2025 @04:19PM (#43018518)
Unfortunately quite common to see serious mathematical issues in the medical literature. I guess due to a combination of math being essential to interpreting medical data and trial results, but most practitioners not having much depth of math knowledge. Just this week I came across the quote "Frequentist 95% CI: we can be 95% confident that the true estimate would lie within the interval." This is an incorrect interpretation of confidence intervals, but the amusing part is that it is from a tutorial paper about them, so the authors should have known better. And cited by 327! https://pmc.ncbi.nlm.nih.gov/articles/PMC6630113/
Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by gsf_emergency_2 on Monday February 10, 2025 @10:29PM (#43008605)
This is the the strongest argument for not shaming reinvention...

Unless the victims are world-class..? (Because it's not entirely not self-inflicted)

https://news.ycombinator.com/item?id=42981356

Shades of the strong-link weak-link dilemma too

Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by eru on Tuesday February 11, 2025 @08:07AM (#43012306)
> This is the the strongest argument for not shaming reinvention...

Sounds like a pretty weak argument? I'm sure there are some good arguments for re-invention. But this ain't one of them.

Basically, re-invention for fun or to help gain understanding is fine. But when you publish a 'new' method, it helps to do a bit of research about prior work. Especially when the method is something you should have heard about during your studies.

Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by Shorel on Tuesday February 11, 2025 @05:06AM (#43010961)
That's not really a problem.

In one hand, it shows the idea is really useful on its own.

And on the other hand, it shows that currently forgotten ideas have a chance to being rediscovered in the future.

Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by blablablerg on Tuesday February 11, 2025 @05:34AM (#43011202)
It is, because you are wasting time reinventing the wheel. Also if something is already well researched, you might miss intricacies, traps, optimizations etc. previous researchers have stumbled upon.
Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by dwaltrip on Tuesday February 11, 2025 @11:48AM (#43015004)
It isn’t necessarily “wasted” time. There are more ways to look at it, as well as 2nd order and 3rd order effects (and so on).

It’s a powerful skill to be able to try to solve things from first principles. And it’s a muscle you can strengthen.

It would be a bit silly to never look anything up, but it isn’t so black and white.

Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by Brian_K_White on Tuesday February 11, 2025 @12:46PM (#43015736)
This is a good example of how the most obvious intuition can be wrong, or at best incomplete.
Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by Shorel on Tuesday February 11, 2025 @05:00PM (#43018901)
You need to be able to do both.

Only reading the existing literature is not good enough.

The capacity to create ideas is also something that needs to be practiced.

Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by p00dles on Tuesday February 11, 2025 @05:58AM (#43011378)
I agree -> even if someone spends their time "rediscovering" an existing solution, I think that the learning experience of coming up with a solution without starting from current best solution is really valuable. Maybe that person doesn't reach the local maximum on that project, but having a really good learning experience maybe enables them to reach a global maximum on one of their next projects.

If I want some novel ideas from a group of people, I'm going to give them the framework of the problem, split them into groups so that they don't bias each other, and say: go figure it out.

Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by bluGill on Tuesday February 11, 2025 @09:12AM (#43012894)
It is not a problem if you are a student learning how to solve problems. Solving previously solved problems is often a good way to learn - because there is a solution you know you are not hitting something that cannot be solved, and your teacher can guide you if you get stuck.

For real world everyday problems normally it is an application of already solved theory or it isn't worth working on at all. We still need researchers to look at and expand our theory which in turn allows us to solve more problems in the real world. And there are real world problems that we pour enormous amounts of effort into solving despite lacking theory, but these areas move much slower than the much more common application of already solved theory and so are vastly vastly more expensive. (this is how we get smaller chip architectures, but it is a planet scale problem to solve)

Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by dwaite on Wednesday February 12, 2025 @03:49PM (#43029644)
If I have seen what others have seen, it is by replicating the work of giants
Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by lobocinza on Friday February 14, 2025 @11:52AM (#43050344)
The cost of creativity/innovation.
Re:Undergraduate Upends a 40-Year-Old Data Science Conjecture by djmips on Wednesday February 12, 2025 @03:52AM (#43023302)
I find a lot of the time concepts and patterns can be the same in different fields, just hidden by different nomenclature and constructed upon a different pyramid of knowledge.

It's nice we have a common language that is mathematics, the science of patterns, to unify such things but it's still going to be a challenge because not everyone is fluent in all the various branches of mathematics.

It's even mind-blowing how many ways you can approach the same problem with equivalencies between different types of mathematics itself.