News for hackers. Stuff that matters.
Re:How to (seriously) read a scientific paper by AlotOfReading on Thursday February 08, 2024 @04:51PM (#39308231)
This misses the best skimming trick I learned from an advisor: After you read the abstract, read the last couple lines/paragraphs of the introduction. That's where you'll find the authors best summary of the paper's contributions and novelty. For instance, the first paper that popped up with some random search terms [0]:

    The present paper attempts to provide a structured and comprehensive overview of state-of-the-art automated driving related hardware-software practices. [...] The aim of this paper is to fill this gap in the literature with a thorough survey.

    The remainder of this paper is written in eight sections. Section II is an overview [...] Details [...] are given in Section III. Section IV presents [...] etc.
When you have a meter high stack of papers for your lit review, comprehending long introductions and conclusions gets too expensive. You want to filter the irrelevant stuff as quickly as possible so you can spend time focused on the couple hundred papers you might actually read and cite.

[0] https://doi.org/10.1109/ACCESS.2020.2983149

Re:How to (seriously) read a scientific paper by ziddoap on Thursday February 08, 2024 @05:40PM (#39308790)
Maybe I'm misunderstanding what you mean, but I don't think your trick was missed at all?

The second paragraph says:

>I start by reading the abstract. Then, I skim the introduction [...]

A few paragraphs down, another quote says:

>I always start with title and abstract. [...] I then read the introduction [...]

Another:

>I nearly always read the abstract first [...] I generally skim the introduction, reading its last paragraph

Is there something significantly different about what you are suggesting?

Re:How to (seriously) read a scientific paper by AlotOfReading on Thursday February 08, 2024 @06:33PM (#39309289)
What I'm saying is similar to the third quote, where you start after the abstract by reading the last 1-5 lines of the introduction. Only if you want to proceed further do you dedicate the time to read or skim the entire intro, which can span multiple pages. You save tens of seconds and a lot of thinking this way, plus it helps frame things by giving a concise statement you can reference as you ask "how does what I'm reading build towards the paper's goals?"
Re:How to (seriously) read a scientific paper by timr on Thursday February 08, 2024 @05:56PM (#39308944)
Maybe it depends on the literature. I almost never read the introduction, unless I am unclear on the motivation for the work from the abstract.

The fundamental problem with abstracts and introductions is that they're sales content. You can't trust them. The only thing you can do is decide if you wish to read further, and see if the paper says what they claim it says. For that, an abstract is sufficient about 99% of the time. Spending lots (or...any amount) of time on the intro before you read the data tables is basically asking to have time stolen from you.

For what it's worth, the proportion of papers where the body doesn't support the claims in the abstract is...high. Like way above 50%, in my experience (approaching nearly 100% for any random paper you find on HN or X.)

My algorithm is:

* read abstract. decide to continue.

* formulate my own version of the experiment in my head -- how would I go about answering this question?

* read all data tables, including supplemental materials. see if anything stands out as weird.

* look at all figures, including supplemental materials. see if anything stands out as weird.

* do I understand the data? if not, search for table/figure references from results, until I do (generally I end up reading most of the results, out-of-order).

* is this paper consistent with the way I'd have approached the problem? If not, is that important? what are the possible methodological flaws? what would they look like in the data?

* re-examine data to look for signs of flaws.

* does the data support the conclusion? if not, why not? what would I do differently or as a follow-up?

* finally, decide if the paper worth reading in detail.

Re:How to (seriously) read a scientific paper by AlotOfReading on Thursday February 08, 2024 @06:24PM (#39309221)
In the literature I spend the most time with (archaeology/anthropology), the abstract is often focused on the broad applicability of the research to the field, which I either don't care about or already know (often both). The last line/paragraph of the introduction contains the specific contributions/findings of the paper. If and only if that's reasonable might it be worth investing the time to read further. I only rarely touch the data tables (assuming they even exist), because I don't want to spend an hour trying to understand whatever software nightmare the probably-not-computer-literate author(s) used and miss the details hidden outside the data in the explanatory text (e.g. samples were processed by throwing away all bits we couldn't identify!) that make the time spent useless.
Re:How to (seriously) read a scientific paper by timr on Thursday February 08, 2024 @06:36PM (#39309314)
Yeah, we come from very different fields. But I'll say this:

> the probably-not-computer-literate author(s) used and miss the details hidden outside the data in the explanatory text (e.g. samples were processed by throwing away all bits we couldn't identify!) that make the time spent useless.

This is super common in my own field, and it's top-five indication that the paper is bad. If I can't understand how they manipulated the data, unless I have some specific other interest, the entire paper goes in the trash. There's no excuse.

For example, it's pretty routine in biology and medicine to see high profile papers that invent some wild/complex "statistical pseudo-control", instead of actually doing a good experiment (which is hard, and probably why the question is unpublished). These papers are basically never worth reading.

See also: not using conventional statistical tests, not including uncertainty estimates, not using standard/vanilla models before jumping to crazy stuff. Not to be too glib, but "we entered the data into SAS/Excel/R/Matlab" is like a bright, flashing warning sign that you're about to see a clown show.

Re:How to (seriously) read a scientific paper by AlotOfReading on Thursday February 08, 2024 @07:16PM (#39309659)
Unfortunately, most of the foundational papers in the field did exactly that, usually without explicitly stating. It was pretty much the standard of practice for all bone analysis prior to ~2010 or so when microstructure analysis became a practical thing. Similar issues in other types of samples exist and it's still common today, it was just what came to my mind as the first example.
Re:How to (seriously) read a scientific paper by ants_everywhere on Thursday February 08, 2024 @06:39PM (#39309343)
Yeah it depends heavily on the field and sometimes on the subfield.

Also the approach I use depends on how much I know about the field. If I know less, I weight the introduction more. If I know a lot, I skip almost all the sales and stage setting content. E.g. if it's a psych paper, often I jump directly to the results and only read other content if the results look reasonable. And so on.

So, I guess my only real point is that it's hard to take advice for how best to read a paper from someone (like your advisor) who knows the field much better than you do. You should probably gather as many tricks from the advice as you can, but maybe don't treat it as gospel and do lots of experimenting.

Re:How to (seriously) read a scientific paper by timr on Thursday February 08, 2024 @07:11PM (#39309624)
It's certainly true that it depends on the subfield and somewhat on your level of experience in that field. But the approach I outlined works pretty well, generally, and only serves to underscore how bad many papers are.

For example, I'm not going to be able to pick up a paper from CERN and do most of the steps I outlined, but if I did do that, and found an obvious statistical anomaly in the first table, well...now I'd have something interesting to ask about and dig into.

Re:How to (seriously) read a scientific paper by ants_everywhere on Thursday February 08, 2024 @08:00PM (#39310045)
oh yeah, I agree. I was agreeing and expanding on your comment, not disagreeing with it.
Re:How to (seriously) read a scientific paper by gcr on Thursday February 08, 2024 @10:27PM (#39311018)
In the early-2010s, computer vision papers started putting a bulleted summary of that paper’s contributions at the end of the intro. It’s an enormously helpful (and fairly recent) invention. I’ve been insisting on using that pattern in the papers I write since, not just because it helps readers understand the key ideas better, but because it also forces authors to clarify the main points to themselves.
Re:How to (seriously) read a scientific paper by bbsz on Friday February 09, 2024 @01:45PM (#39318640)
That train's probably gone with the rise of LLMs? As in, in 2020s it would be more than often just delegated to the chatbot and called quits. At least that's my thought.