Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by baobabKoodaa on Sunday March 17, 2024 @08:02AM
(#39733861)
There are only a few search results on this list which appear to be ChatGPT generated. If you filter the list by "after 2020", you will see the number of results drop from 657 to 71 (ChatGPT did not exist in 1962, so it could not have possibly written scholarly articles in 1962).
Here is an excerpt from one of the results after constraining the search to "after 2020":
Hundreds of young men are crowding them, to prepare for the army and navy, to represent their country in some future war. Certainly here is ample preparation for the patriot of war. But is our country offering an equal opportunity to the one who would serve herin times of peace?
This is similar to the case where a journalist typed "15 year old" on Pornhub search, didn't understand how search works, and didn't look at the results.
But even then, I don't think this is the gotcha OP thinks it is: LLM generation and summarizing is a legit tool in the paper writing process, especially for conclusions.
In my academic experience, the paper introduction and conclusion is often written last-minute, close to the submission deadline. While a good intro and conclusion are important for the reader, very little time is made available for it since researchers spend maximal time on expanding material findings and experiments. No wonder that assistive writing tools are used to speed things up. As long as proper post-editing and proof-reading is applied.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by semanticist on Sunday March 17, 2024 @08:46AM
(#39734121)
> As long as proper post-editing and proof-reading is applied for course.
The whole point of the original article is that people aren't doing even that.
Plus in an academic context, it's important to acknowledge when parts of the paper are not your own work, especially in cases where the generated summary may not accurately reflect the actual paper. Saying that ChatGPT has been used to generate summaries would allow people to judge if the summary alone can be trusted to represent the content of the paper.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by aredox on Sunday March 17, 2024 @08:22AM
(#39733965)
Exactly. A correct analysis would be to determine the baseline pre-chatGPT, examine if there is a trend (some language "clichés" evolve with time) and then check if there is a clear shift post-ChatGPT.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by DataDive on Sunday March 17, 2024 @08:33AM
(#39734039)
When I looked at the examples, very few seemed to be fully ChatGPT generated.
That being said, most scientists are very smart and recognize the power of ChatGPT. They are most likely heavily relying on it when producing summaries, abstracts, hypotheses, etc., and because they can and will edit it you won't be able to tell from words alone.
ChatGPT is an equalizer between native English speakers and scientists speaking English as a second language.
But both groups will use it heavily and nobody can foresee what the net effect of it is on science.
Having had that experience, I can tell you that it takes weeks or months to write up results as a paper and go through the most boring steps of formatting and organizing them into antiquated presentation formats: Intro/Discussion/Results, dumbass citations, labeling, backward wordings ... etc
What should be a five-paragraph discovery on a blog has to take the form of a multipage, heavily formatted document that contains enormous, unrelated, and useless information.
With ChatGPT, you can do it in a day. If you are not using ChatGPT, you lose out.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by noobcoder on Sunday March 17, 2024 @08:09AM
(#39733893)
One of my close friends (Ivy League guy) did the same with his PhD thesis on Siamese networks in improving robotic perception. With the final presentation upcoming, in 2-3 days he's got a whole polished thesis ready to go and even convinced his guide that this was what he was working on it since 3 months. He was recently offered a doctorate
I wonder how many people adeptly use GPTs and manage to navigate through their academic landscapes nowadays. I bet its majority
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by dougb5 on Sunday March 17, 2024 @08:05AM
(#39733882)
After late '22 I noticed a surge in superlatives like "vital", "essential", "crucial", and "pivotal" in essays and cover letters from students. ChatGPT uses words like these whenever you ask it to write an essay, and it's a dead giveaway. It's ruined the legitimate uses of the words.
Pick one of the Google Scholar results from the article and you'll find something awful like this (the start of the second paper): "The carriage of goods under international commercial law is a complex and essential aspect of international trade."
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by Sharlin on Sunday March 17, 2024 @09:09AM
(#39734249)
To be fair, replacing "important" with some less generic synonym is like one of the first tips in any Copy Editing for Dummies book, and definitely suggested by Grammarly, for example.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by peter_vukovic on Monday March 18, 2024 @03:03AM
(#39741092)
Any word can become generic when overused. As more people get access to Grammarly and CharGPT, more of them will apply the same recommendations, turning previously rare words into generics.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by 6510 on Monday March 18, 2024 @06:07AM
(#39742065)
Chose your words carefully, if lucky you might find some that only fit you.
Thus therefore to provide an example thereof one may replaces all instances of "important" or "essential" with "splendid", "brilliant" "magnificent","gorgeous" or "sophisticated".
—"The carriage of goods under international commercial law is a sophisticated and gorgeous aspect of international trade."
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by nomonnai on Sunday March 17, 2024 @09:43AM
(#39734436)
> surge in superlatives like "vital", "essential", "crucial", and "pivotal"
This is very interesting. It could also be an effect of Grammarly, which suggests these replacements for the generic "important." Perhaps it's a combination of several effects.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by dougb5 on Sunday March 17, 2024 @01:13PM
(#39736017)
My guess is that ChatGPT does it because RLHF rewards strongly stated opinions, because that's what humans prefer. It's a kind of "sycophantic behavior" that researchers have observed in these models (https://arxiv.org/abs/2310.13548)
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by dougb5 on Sunday March 17, 2024 @01:09PM
(#39735974)
Heh, or maybe in some general sense, everything is essential, crucial, vital, constantly pivoting, etc., and these are the best words to use. Some day ChatGPT will tell me "I told you so!"
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by dougb5 on Sunday March 17, 2024 @01:22PM
(#39736101)
"Superlative" can mean not just the grammatical category that includes "*est" words, but also, more informally, adjectives that live at an extreme, as "critical" does within the concept of "importance".
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by yesco on Sunday March 17, 2024 @01:44PM
(#39736316)
I've been interacting with ChatGPT and other LLMs so much I've already caught myself getting influenced by their word choice in the same way I might start subconsciously adopting the vocabulary of a friend. Food for thought .
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by Tubbe on Sunday March 17, 2024 @07:41AM
(#39733736)
Crazy how broken the entire publication and review process is, that this stuff goes through without anyone noticing, and some even in well recognized journals.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by brohee on Sunday March 17, 2024 @08:22AM
(#39733961)
It's basically organised crime. It's public money funding scientific publishing, and a huge part of the industry is just here to scam public money.
The industry is ripe for public prosecution when an article submitter didn't read it, an article reviewer didn't read it, neither did the publisher yet it is published in a journal with a subscription price in the thousands payed by publicly funded libraries.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by esalman on Sunday March 17, 2024 @09:06AM
(#39734239)
So many students on every level of higher education, probably even starting from high school, are now using chat GPT. Does this make education a crime?
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by throw9988998899 on Sunday March 17, 2024 @07:43AM
(#39733757)
I know a guy who is currently on a higher technical school. He talks very open how he and all in his class use Chat GPT for all the bloat they have to write
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by admissionsguy on Sunday March 17, 2024 @08:02AM
(#39733859)
Students at universities do that now. And they get passed. It exposes the whole charade that much of the higher education has become.
The idea behind writing a lot has been that it forces students to think systematically and thereby learn. But they are not typically interested in that, they just need a diploma in order to get a job. Before computers they were copying books verbatim and since plagiarism software they paraphrase books. And now chatgpt removes the middle man - the student. It doesn’t make much sense either way. The source of the problem is really the conflation of higher learning with vocational training.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by tulio_ribeiro on Sunday March 17, 2024 @08:00AM
(#39733847)
I'm fine with and approve the usage of LLMs in academia, as long as they provide genuine value and something new to the field. These tools should be embraced when they can augment human intellect.
However, I draw a firm line at using them to generate complete academic works or nonsensical content, as that undermines the integrity of research and renders it devoid of originality. LLMs should serve as invaluable assistants to free up scholars for higher-order analysis, not as replacements for human ingenuity.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by regularfry on Sunday March 17, 2024 @08:38AM
(#39734072)
The problem here is not necessarily the use of chatgpt at all. It's that academic idiom requires content-free linguistic noise for acceptance, and people are quite naturally turning to mechanistic routes to automate out the drudgery.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by rich_sasha on Sunday March 17, 2024 @08:45AM
(#39734114)
IMO academic papers are too long. The decorum dictates that a core of real, novel content must be surrounded in tons of fluff. Most people don't ever read that fluff (some do, and it's not totally useless, but not to most people).
It doesn't actually bother me if some of that fluff is ChatGPT-generated, provided the author actually read it and accepts the autogenerated content.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by anal_reactor on Monday March 18, 2024 @04:55AM
(#39741667)
I'm having flashbacks from writing my Master's thesis. The experimental part was done in two weeks, then I spent a week describing the results, then a few months going from 15 pages to 80 pages and at least 30 citations, with the latter being surprisingly difficult because of uniqueness of my research topic.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by CuriouslyC on Sunday March 17, 2024 @08:53AM
(#39734157)
Buckle up, because we're not far away from autonomous research agents. It'll start with computational analysis, plot generation and creation of data discovery documents, but soon models will learn how to request simple physical experiments (sequencing some DNA, mass spec a sample, etc) and plan research based around the outcome.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by rchaud on Sunday March 17, 2024 @10:08AM
(#39734592)
You may as well say that you only want dynamite to be used for nonviolent purposes, like demolishing condemned buildings. It doesn't work that way, which is exactly what AI ethicists and researchers have been warning about - AI should be regulated, because once its in somebody's hands, you don't control how they use it, and the repercussions could be much wider than people imagine.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by perihelions on Sunday March 17, 2024 @08:14AM
(#39733914)
The upside of all this is that professionals are communicating their points better. Lots of examples from non-English (-language) nationalities using LLM's to write exposition sections for English journals. This lowers the communications friction across the language barrier; and lowers the implicit bias and penalization when people read a (technically correct and useful) paper that has weak grammar, and is confusing to parse.
These aren't purely bad things! I wonder what future social conventions for this kind of thing will look like.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by vitiral on Sunday March 17, 2024 @08:59AM
(#39734192)
Perhaps two people can who don't fluently speak the same language can better communicate. But that's not really the bar is it? These are writing articles on behalf of fluent speakers for other fluent speakers to read. Is it improving THEIR "communication".
I'd say by definition the answer is no, since it's no longer humans communicating with each other at all.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by MarkSweep on Sunday March 17, 2024 @09:56AM
(#39734504)
I agree that LLM can be used as a tool by non-native English speakers.
But the examples cited in the article are indefensible. The authors of the paper we either so care less they did not notice the the “certainly, here is…” preamble made no sense in a paper. Or they have such a poor grasp of English that they are unable to comprehend what they copy-pasted to notice. Either way this suggests that nothing in the paper can be relied upon.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by admissionsguy on Sunday March 17, 2024 @08:12AM
(#39733909)
Hopefully this leads to a shorter communication form for presenting new results. No need for each paper to have a half-assed intro/review of the field. Since now even the authors skip that part.
Leave reviews for quality papers by top experts in the area.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by rzzzt on Sunday March 17, 2024 @07:51AM
(#39733796)
First hit, in the keywords section (Ключевые слова): "methodology. Certainly / here is the English translation of the provided text"
Second hit: "Certainly! Here is a review of the literature on the carriage of goods under international commercial law:"
Third hit: "Certainly, here is the tabular representation of intraocular pressure (IOP) at 1 and 4 hours for Group X and Group Y:"
Fourth hit: "Certainly, here is a list of both Geo-natural and geosynthetic
materials commonly used in civil engineering and environmental applications."
Fifth hit: "Certainly, here is a concise summary of the provided sections:"
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by baobabKoodaa on Sunday March 17, 2024 @08:04AM
(#39733878)
If you keep going, you will see that the list contains less than 10 "damning examples" like the ones you posted here. The remaining 740 entries do not appear to be ChatGPT-generated.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by rzzzt on Sunday March 17, 2024 @08:21AM
(#39733957)
Sixth hit: "Certainly, here is a list of various types of polymers:"
Seventh hit: <not found in paper or abstract>
Eighth hit: "Certainly, here is the essential information provided for reference to the proceedings of the conference on the topic of university book production:"
Ninth hit: "Certainly, here is the methodology for conducting a literature review on the attitudes of healthcare workers toward COVID-19 vaccination:"
Tenth hit: <paid access>
Eleventh hit: "Furthermore, to confirm the advantages of the new model, Certainly! Here is the rephrased sentence with improved language quality and clarity:"
I don't think that all results contain such wording, and surely daghamm's statement can't be disproven by listing these. But there is something that (also) rubs me the wrong way about these sentences getting into research papers.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by rzzzt on Sunday March 17, 2024 @08:39AM
(#39734088)
Good for them and their readers.
Re-reading Simon's post, the only word that I think others might find contentous is "huge". Would it be better if it read the search expression turns up "a number" of papers, without a qualifier?
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by baobabKoodaa on Sunday March 17, 2024 @08:05AM
(#39733881)
Out of the 750 articles listed, 90% of those were written before 2020 (before ChatGPT was released). You claim that you picked 10 out of these 750 articles at random, and all 10 of them contained very obvious ChatGPT artifacts. Even if we assume 100% of articles written after 2020 have been generated with ChatGPT, your claim is completely implausible. It would require you to randomly land on after-2020 article 10 times out of 10 (quick napkin math: 0.1^10).
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by 10c8 on Sunday March 17, 2024 @08:00AM
(#39733844)
If you actually take time to go and read through them, you can see that the writing pattern is clearly different to that of ChatGPT, contrary to the recent ones:
- "[...] Certainly here is a vast and important project for research."
Versus:
- "Certainly! Here is the text with spaces added after each word:"
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by throwawayyy9237 on Sunday March 17, 2024 @07:50AM
(#39733791)
I don't have time to properly go through the entire list, of course, but the first paper that I've opened seemed to use ChatGPT to format a table of values, presumably not in tabular format originally. It didn't _seem_ like any of the ideas or conclusions were GPT-created.
I'm perfectly fine with this usage, although it is very sloppy if the authors and reviewers didn't pick the "certainly" in the final text.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by AlecSchueler on Sunday March 17, 2024 @08:16AM
(#39733920)
You can ask it for code that will convert your input into tabulated output. Easier to check the logic of the code than all of the values in the table, then just plug your data into it.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by bigyikes on Sunday March 17, 2024 @07:40AM
(#39733734)
How can someone be intelligent enough to submit a research paper but stupid enough to fail at taking the most basic steps to hide their AI plagiarism?
What do we even do about something like this? Is there an organization that has the power to go through this list and blacklist each of the authors from future publications? Our society should have a zero tolerance policy for drivel like this. It cheapens the entire institution.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by muldvarp on Sunday March 17, 2024 @08:27AM
(#39734000)
I wouldn't call it plagiarism, but not even reading the output of the LLM before pasting it into your paper shows a lack of professionalism and indicates that the data in that table might as well be the hallucination of an LLM. Not exactly a good look.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by muldvarp on Monday March 18, 2024 @06:22AM
(#39742143)
I disagree. LLMs fundamentally change the situation: They make it easier to "write" low quality publications and they make it harder to immediately recognize that a publication is low quality.
Before LLMs, one of the main deterrents of low quality work was that the effort required to write a low quality paper is within an order of magnitude compared to the effort for a high quality paper. Using LLMs, however, I can "write" hundreds of low quality papers (that look okay-ish on the surface) in the time it takes me to write a single high quality paper.
I also don't think that researchers publishing high quality papers will benefit that much from LLMs. I tried getting ChatGPT to reproduce an argument I made in my PhD thesis and it took me many tries before ChatGPT even produced something without factual errors. As for padding: Papers should ideally not contain any padding. If a section in your paper is so devoid of actual information that an LLM can write it, you should probably remove it altogether.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by gordian-mind on Tuesday March 19, 2024 @03:05AM
(#39753538)
> Using LLMs, however, I can "write" hundreds of low quality papers (that look okay-ish on the surface) in the time it takes me to write a single high quality paper.
I've been thinking lately that one could probably leverage the same technology to evaluate the quality of research, possibly as a post-hoc verification.
> I also don't think that researchers publishing high quality papers will benefit that much from LLMs.
Oh, but they do! If not for anything, LaTeX formatting is tedious.
> Papers should ideally not contain any padding. If a section in your paper is so devoid of actual information that an LLM can write it, you should probably remove it altogether.
It is mostly correct, except for perhaps the abstract and some parts that provide structure for the paper (Outline, maybe Conclusion, ...). However, you can always seed an LLM with the bits of information you want explained and use the result as a basis for improvement. It can seed writing: there is no obligation to copy-paste and leave it as is.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by muldvarp on Wednesday March 20, 2024 @10:39AM
(#39767257)
> I've been thinking lately that one could probably leverage the same technology to evaluate the quality of research, possibly as a post-hoc verification.
I doubt it. They'll be just as effective as ChatGPT-detectors are. Completely useless.
> If not for anything, LaTeX formatting is tedious.
LaTeX formatting is a very, very, very small part of publishing research. At least it was for me.
> However, you can always seed an LLM with the bits of information you want explained and use the result as a basis for improvement.
That's what I tried to do with the argument in my PhD thesis. It didn't work. Also, at the point where you have meticulously thought through how to structure your argument (and that's what I needed to do to get ChatGPT to produce anything of value at all) you did 95% of the work already. Actually producing the text isn't the hard part of writing.
My point still stands: LLMs help way more if you want to produce low-quality papers than when you want to produce high-quality papers and as a consequence, the percentage of low-quality work will increase.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by TylerE on Sunday March 17, 2024 @08:01AM
(#39733854)
Hell, better that they DO include the prompt. Since that A: clearly discloses AI usage and B: If someone attempts to duplicate the experiment (if relevant), they can take their data (perhaps give the prompt input, at least a subset showing all structure, in an appendix) and run it through the prompt.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by bluish29 on Sunday March 17, 2024 @08:03AM
(#39733870)
I would very much argue that ChatGPT answering questions about how to make this table in latex, how to move this table by resizing into IEEE template...etc the most useful(and ethical) way of using LLMs to help with writing papers.
If you try that and was writing papers and academic manuscripts before in latex you will see a boost in your writing. I remember that I spent more time searching on how to do stuff in latex or fix image relative locations on my thesis than actual writing.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by twroijewrlk3 on Sunday March 17, 2024 @08:21AM
(#39733956)
Kind of sad that a lot of these papers are from India.
This is probably related to the policy change that has aimed to increase publication numbers from India (and so IIRC there's like a quota that needs to be met for promotions etc.).
Not surprised that Indians use this. Their obsession with English-medium education means that understanding has little value compared to memorization. This'd be okay for English-grammar (as with Japanese/Chinese), but it's a bit worrying that this is the case even in "real" technical subjects.
They'll pay for their linguistic-apartheid system when all the hype about the "next China" inevitably comes crashing down. The "cream" elites who've made it big in Anglo-Saxon nations and/or run the Indian state, and push for these discriminatory policies that allow education only for the <5% of Indians who can speak English, are so disconnected from the ground any way (like neighboring Pakistan) that they'll move onto US etc.
Worrying still is that this absurdity has come down all the way to kindergarten/primary education even in rural center (most teachers in these schools/universities can't speak a sentence of English). How they'll industrialize with literally an idiot population, that knows neither numeracy, nor a language, neither one's own or almighty English, is anybody's guess. Add to it all the socialist policies from 1st world nations that inevitably sell well with such large unproductive uncivilized mass of population (civilization = education).
This is how you destroy a country that gave the world Linguistics and Calculus (though they'll blame every useless invader from outside for this).
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by tetris11 on Sunday March 17, 2024 @07:37AM
(#39733724)
Damming. What is research even becoming at this stage? I hope open peer review is the golden bullet here, since the paid journals are clearly struggling under the deluge of AI spam
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by TimTheTinker on Sunday March 17, 2024 @07:49AM
(#39733784)
It's not that they're struggling, it's that by and large they don't care. They're making money from publishing scientific papers (or whatever they can be called), and that's all that matters.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by kettleballroll on Sunday March 17, 2024 @08:00AM
(#39733851)
N. Usually THE AUTHOR pays the journal to publish anything. However, the professors need publications for their CV (number of published articles is a very important metric in the academic world), to acquire new funds ("give us money to do research on topic X. Look at how much we published on this related topic before, we clearly know what we're doing") or advance in their career (ASA function of the former two points). Students need publications as part of their graduation requirements (in a lot of fields, you are required to have N publications in a peer reviewed venue to get your PhD).
So in both cases, you don't make money directly from publications, but from the prestige they bring and the credibility they're associated with ("this guy must be an expert on X, because they published on the topic").
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by qarl on Sunday March 17, 2024 @02:32PM
(#39736749)
I asked ChatGPT to consider your point:
Understandably, concerns about the quality of LLM-generated content are valid. However, it's crucial to recognize that academic research goes through rigorous processes before publication, including oversight by advisors and peer review. These layers of scrutiny help ensure that any content, whether initially drafted with the aid of AI like GPT-4 or not, meets the high standards of accuracy, reliability, and originality expected in scholarly work. AI tools are used as aids in the research process, not replacements for human intellect, and the responsibility for the final output always rests with the human authors. Hence, the academic community already has strong mechanisms in place to filter out inaccuracies or "bullshit," ensuring that the integrity of published research is maintained.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by jordanpg on Sunday March 17, 2024 @09:36AM
(#39734396)
Putting aside the well-known issues with academic publication that this problem highlights, another aspect of modern publications more generally that LLMs are going to disrupt is word bloat. Too many words.
For example, many academic papers are pages and pages of background, discussion, citations, etc. Very often the actual new material can fit on an index card + 1 figure. I understand why all the background is needed, in the abstract, but maybe this traditional way of thinking about incremental academic thought needs to shift in light of the reality of LLMs: LLMs will be used to write needless fluff. Indeed, it's what they're good at.
I'm a patent attorney and I can say that the same thing applies there. Patents are enormous documents filled with legalese, background, description, vague, conditional language, etc. There are good legal reasons for that, but again, often the actual novel invention can fit on an index card.
For that reason, it's self-evident to me that my job will be replaced, at least to some significant extent, by LLMs. But why not just cut out all the needless words? I could imagine patent law that added a "parsimony" requirement to patent filings. Examiners could reject filings that contained extraneous words. Then maybe patent lawyers would still be needed: to write the shortest possible patent that still contained the novelty.
This is just a toy thought experiment, not a serious proposal. My point is that LLMs expose as trivially bankrupt some aspects of academic and patent writing (and probably other places: marketing copy, some nonfiction essay-style writing, etc.).
Maybe LLMs will force us to embrace or even require brevity and conciseness to ensure that human beings stay in the loop.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by sorenjan on Sunday March 17, 2024 @09:59AM
(#39734527)
Humans will use LLMs to produce long texts from a brief description, and then other humans will use LLMs to make a summary of that text. Maybe eventually we can make this communication more effective.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by krastanov on Sunday March 17, 2024 @10:06AM
(#39734578)
How many of these are in serious journals that actually are read by other scientists and how many of these are in low-quality predatory journals that simply help unsuccessful scientists inflate their publication numbers for a fee?
Looking at the search results, none of these are in "real" science journals. This is just yet another way in which predatory journals are terrible for science and academia (and its public perception), but the existence of garbage journals has been a problem for 2 decades (or more?).
On the bright side, productivity at respected institutions has not been significantly hampered by predatory journals. On the other hand, academia as a whole has obviously failed to teach incoming members that this is basically fraud (partially due to the publish-or-perish culture that is now even endorsed by governments).
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by timkam on Sunday March 17, 2024 @10:14AM
(#39734620)
Note that "false positives" may appear in the Google Scholar results for such queries. For example, a high-profile AI researcher shared a screenshot of https://scholar.google.com/scholar?as_sdt=0%2C5&as_ylo=2023&... (on X/Twitter: https://twitter.com/MelMitchell1/status/1768422636944499133).
The second entry is an ironically written abstract of a talk and the author clearly (and consciously) plays with the cliché. Obviously, most of the fans of the person who shared the screenshot to thousands of followers will not notice this.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by 082349872349872 on Sunday March 17, 2024 @10:26AM
(#39734689)
Is the adjacency in Scholar limited to intra-sentence fragments? It'd be problematic if not, certainly. Here is another example of the sibboleth: when we use "certainly" here. Is this a query hit?
It's a constant crawl of the literature that uses hand-written phrase detectors. Many of the phrases it looks for are caused by using spinners on plagiarised text, like this one:
"At first, the input picture is transformed into the ruddy, green, and blue organize, and the clamor within the green band is evacuated using a median channel"
Here "noise" has been turned into "clamor". Nonsense text like this is easy to find in journals published by well known brands, in this case Springer Nature. A search for "profound learning" (deep learning) yields over 2000 results:
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by beginning_end on Sunday March 17, 2024 @10:24AM
(#39734678)
Until this point in history there has been a reasonable correlation between the quality of writing and the "quality" of the underlying work in most text. If effort was put into the writing (and learning to write), that in itself used to be a decent indicator that effort and skill also was put into the data/ideas that the text conveyed.
That rule no longer works. People are still going to rely on it for a while though, and I'm worried it's going to break some stuff over the next few years.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by empath-nirvana on Sunday March 17, 2024 @10:58AM
(#39734871)
I think probably the solution here is to stop using publishing in peer reviewed journals as a job requirement for academics. Even before ChatGPT, academic publishing was already falling victim to Goodhart's law. Remove the perverse incentives and it should cut down on this by quite a lot.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by 2cynykyl on Sunday March 17, 2024 @12:07PM
(#39735402)
Another way to look at the problem of LLM-generated garbage papers is that it doesn't actually change anything, because serious researchers in a given field already didn't trust work coming from unknown labs in shady journals. Researchers need to earn a reputation first by doing all the usual things like going to conferences (to build personal relationships), publishing reproducible results in real journals, making an actual impact on important problems, etc. Basically, LLM papers are just a new contribution to the already existing background noise.
Re:Google Scholar search: "certainly, here is" -chatgpt -llm
by geor9e on Sunday March 17, 2024 @01:23PM
(#39736105)
Most of the first results have Cyrillic letters in the Author's name. I think these folks perhaps aren't fluent in English, and think "Certainly, here is a summary of…" is a normal thing people would say. They probably see it as a monologue flourish like "Secondly, here is a summary…", and don't realize it is actually a subservient conversational reply, a dead giveaway of an LLM chat.
Here is an excerpt from one of the results after constraining the search to "after 2020":
Hundreds of young men are crowding them, to prepare for the army and navy, to represent their country in some future war. Certainly here is ample preparation for the patriot of war. But is our country offering an equal opportunity to the one who would serve herin times of peace?
This is similar to the case where a journalist typed "15 year old" on Pornhub search, didn't understand how search works, and didn't look at the results.
Better queries would include proper timeframing and all LLM-preference-tuning phrases from https://github.com/AlpinDale/gptslop
But even then, I don't think this is the gotcha OP thinks it is: LLM generation and summarizing is a legit tool in the paper writing process, especially for conclusions.
In my academic experience, the paper introduction and conclusion is often written last-minute, close to the submission deadline. While a good intro and conclusion are important for the reader, very little time is made available for it since researchers spend maximal time on expanding material findings and experiments. No wonder that assistive writing tools are used to speed things up. As long as proper post-editing and proof-reading is applied.
The whole point of the original article is that people aren't doing even that.
Plus in an academic context, it's important to acknowledge when parts of the paper are not your own work, especially in cases where the generated summary may not accurately reflect the actual paper. Saying that ChatGPT has been used to generate summaries would allow people to judge if the summary alone can be trusted to represent the content of the paper.
Which already filters after 2023.
That being said, most scientists are very smart and recognize the power of ChatGPT. They are most likely heavily relying on it when producing summaries, abstracts, hypotheses, etc., and because they can and will edit it you won't be able to tell from words alone.
ChatGPT is an equalizer between native English speakers and scientists speaking English as a second language.
But both groups will use it heavily and nobody can foresee what the net effect of it is on science.
Having had that experience, I can tell you that it takes weeks or months to write up results as a paper and go through the most boring steps of formatting and organizing them into antiquated presentation formats: Intro/Discussion/Results, dumbass citations, labeling, backward wordings ... etc
What should be a five-paragraph discovery on a blog has to take the form of a multipage, heavily formatted document that contains enormous, unrelated, and useless information.
With ChatGPT, you can do it in a day. If you are not using ChatGPT, you lose out.
Maybe even one day society will re-value conciseness and precision of thought over linguistic diarrhea.
I wonder how many people adeptly use GPTs and manage to navigate through their academic landscapes nowadays. I bet its majority
Pick one of the Google Scholar results from the article and you'll find something awful like this (the start of the second paper): "The carriage of goods under international commercial law is a complex and essential aspect of international trade."
Thus therefore to provide an example thereof one may replaces all instances of "important" or "essential" with "splendid", "brilliant" "magnificent","gorgeous" or "sophisticated".
—"The carriage of goods under international commercial law is a sophisticated and gorgeous aspect of international trade."
This is very interesting. It could also be an effect of Grammarly, which suggests these replacements for the generic "important." Perhaps it's a combination of several effects.
The industry is ripe for public prosecution when an article submitter didn't read it, an article reviewer didn't read it, neither did the publisher yet it is published in a journal with a subscription price in the thousands payed by publicly funded libraries.
The idea behind writing a lot has been that it forces students to think systematically and thereby learn. But they are not typically interested in that, they just need a diploma in order to get a job. Before computers they were copying books verbatim and since plagiarism software they paraphrase books. And now chatgpt removes the middle man - the student. It doesn’t make much sense either way. The source of the problem is really the conflation of higher learning with vocational training.
However, I draw a firm line at using them to generate complete academic works or nonsensical content, as that undermines the integrity of research and renders it devoid of originality. LLMs should serve as invaluable assistants to free up scholars for higher-order analysis, not as replacements for human ingenuity.
It doesn't actually bother me if some of that fluff is ChatGPT-generated, provided the author actually read it and accepts the autogenerated content.
But better still, cut the fluff.
But it has been done for at lest a decade or so I think
These aren't purely bad things! I wonder what future social conventions for this kind of thing will look like.
I'd say by definition the answer is no, since it's no longer humans communicating with each other at all.
It implies that the authors did not care to thoroughly review related work. They basically didn't read any papers.
If you try to have a technical conversation with this sort of a person, they'll communicate their points out of their behind.
But the examples cited in the article are indefensible. The authors of the paper we either so care less they did not notice the the “certainly, here is…” preamble made no sense in a paper. Or they have such a poor grasp of English that they are unable to comprehend what they copy-pasted to notice. Either way this suggests that nothing in the paper can be relied upon.
Leave reviews for quality papers by top experts in the area.
But a quick browsing shows that not all these papers contain that phrase and not all that do seem to be machine generated.
Second hit: "Certainly! Here is a review of the literature on the carriage of goods under international commercial law:"
Third hit: "Certainly, here is the tabular representation of intraocular pressure (IOP) at 1 and 4 hours for Group X and Group Y:"
Fourth hit: "Certainly, here is a list of both Geo-natural and geosynthetic materials commonly used in civil engineering and environmental applications."
Fifth hit: "Certainly, here is a concise summary of the provided sections:"
Seventh hit: <not found in paper or abstract>
Eighth hit: "Certainly, here is the essential information provided for reference to the proceedings of the conference on the topic of university book production:"
Ninth hit: "Certainly, here is the methodology for conducting a literature review on the attitudes of healthcare workers toward COVID-19 vaccination:"
Tenth hit: <paid access>
Eleventh hit: "Furthermore, to confirm the advantages of the new model, Certainly! Here is the rephrased sentence with improved language quality and clarity:"
I don't think that all results contain such wording, and surely daghamm's statement can't be disproven by listing these. But there is something that (also) rubs me the wrong way about these sentences getting into research papers.
Re-reading Simon's post, the only word that I think others might find contentous is "huge". Would it be better if it read the search expression turns up "a number" of papers, without a qualifier?
I randomly picked 10 articles and all of them had the phrase in a manner that indicated it was directly pasted from ChatGPT.
In summary: you are lying.
https://scholar.google.fr/scholar?hl=fr&as_sdt=0%2C5&as_ylo=...
- "[...] Certainly here is a vast and important project for research."
Versus:
- "Certainly! Here is the text with spaces added after each word:"
I'm perfectly fine with this usage, although it is very sloppy if the authors and reviewers didn't pick the "certainly" in the final text.
But I was talking about the intended usage, it doesn't sit as fraudulent to me in the scientific sense.
What do we even do about something like this? Is there an organization that has the power to go through this list and blacklist each of the authors from future publications? Our society should have a zero tolerance policy for drivel like this. It cheapens the entire institution.
Before LLMs, one of the main deterrents of low quality work was that the effort required to write a low quality paper is within an order of magnitude compared to the effort for a high quality paper. Using LLMs, however, I can "write" hundreds of low quality papers (that look okay-ish on the surface) in the time it takes me to write a single high quality paper.
I also don't think that researchers publishing high quality papers will benefit that much from LLMs. I tried getting ChatGPT to reproduce an argument I made in my PhD thesis and it took me many tries before ChatGPT even produced something without factual errors. As for padding: Papers should ideally not contain any padding. If a section in your paper is so devoid of actual information that an LLM can write it, you should probably remove it altogether.
I've been thinking lately that one could probably leverage the same technology to evaluate the quality of research, possibly as a post-hoc verification.
> I also don't think that researchers publishing high quality papers will benefit that much from LLMs.
Oh, but they do! If not for anything, LaTeX formatting is tedious.
> Papers should ideally not contain any padding. If a section in your paper is so devoid of actual information that an LLM can write it, you should probably remove it altogether.
It is mostly correct, except for perhaps the abstract and some parts that provide structure for the paper (Outline, maybe Conclusion, ...). However, you can always seed an LLM with the bits of information you want explained and use the result as a basis for improvement. It can seed writing: there is no obligation to copy-paste and leave it as is.
I doubt it. They'll be just as effective as ChatGPT-detectors are. Completely useless.
> If not for anything, LaTeX formatting is tedious.
LaTeX formatting is a very, very, very small part of publishing research. At least it was for me.
> However, you can always seed an LLM with the bits of information you want explained and use the result as a basis for improvement.
That's what I tried to do with the argument in my PhD thesis. It didn't work. Also, at the point where you have meticulously thought through how to structure your argument (and that's what I needed to do to get ChatGPT to produce anything of value at all) you did 95% of the work already. Actually producing the text isn't the hard part of writing.
My point still stands: LLMs help way more if you want to produce low-quality papers than when you want to produce high-quality papers and as a consequence, the percentage of low-quality work will increase.
If you try that and was writing papers and academic manuscripts before in latex you will see a boost in your writing. I remember that I spent more time searching on how to do stuff in latex or fix image relative locations on my thesis than actual writing.
https://twitter.com/pmarca/status/1768405485705650309
This is probably related to the policy change that has aimed to increase publication numbers from India (and so IIRC there's like a quota that needs to be met for promotions etc.).
Not surprised that Indians use this. Their obsession with English-medium education means that understanding has little value compared to memorization. This'd be okay for English-grammar (as with Japanese/Chinese), but it's a bit worrying that this is the case even in "real" technical subjects.
They'll pay for their linguistic-apartheid system when all the hype about the "next China" inevitably comes crashing down. The "cream" elites who've made it big in Anglo-Saxon nations and/or run the Indian state, and push for these discriminatory policies that allow education only for the <5% of Indians who can speak English, are so disconnected from the ground any way (like neighboring Pakistan) that they'll move onto US etc.
Worrying still is that this absurdity has come down all the way to kindergarten/primary education even in rural center (most teachers in these schools/universities can't speak a sentence of English). How they'll industrialize with literally an idiot population, that knows neither numeracy, nor a language, neither one's own or almighty English, is anybody's guess. Add to it all the socialist policies from 1st world nations that inevitably sell well with such large unproductive uncivilized mass of population (civilization = education).
This is how you destroy a country that gave the world Linguistics and Calculus (though they'll blame every useless invader from outside for this).
So in both cases, you don't make money directly from publications, but from the prestige they bring and the credibility they're associated with ("this guy must be an expert on X, because they published on the topic").
Understandably, concerns about the quality of LLM-generated content are valid. However, it's crucial to recognize that academic research goes through rigorous processes before publication, including oversight by advisors and peer review. These layers of scrutiny help ensure that any content, whether initially drafted with the aid of AI like GPT-4 or not, meets the high standards of accuracy, reliability, and originality expected in scholarly work. AI tools are used as aids in the research process, not replacements for human intellect, and the responsibility for the final output always rests with the human authors. Hence, the academic community already has strong mechanisms in place to filter out inaccuracies or "bullshit," ensuring that the integrity of published research is maintained.
For example, many academic papers are pages and pages of background, discussion, citations, etc. Very often the actual new material can fit on an index card + 1 figure. I understand why all the background is needed, in the abstract, but maybe this traditional way of thinking about incremental academic thought needs to shift in light of the reality of LLMs: LLMs will be used to write needless fluff. Indeed, it's what they're good at.
I'm a patent attorney and I can say that the same thing applies there. Patents are enormous documents filled with legalese, background, description, vague, conditional language, etc. There are good legal reasons for that, but again, often the actual novel invention can fit on an index card.
For that reason, it's self-evident to me that my job will be replaced, at least to some significant extent, by LLMs. But why not just cut out all the needless words? I could imagine patent law that added a "parsimony" requirement to patent filings. Examiners could reject filings that contained extraneous words. Then maybe patent lawyers would still be needed: to write the shortest possible patent that still contained the novelty.
This is just a toy thought experiment, not a serious proposal. My point is that LLMs expose as trivially bankrupt some aspects of academic and patent writing (and probably other places: marketing copy, some nonfiction essay-style writing, etc.).
Maybe LLMs will force us to embrace or even require brevity and conciseness to ensure that human beings stay in the loop.
Looking at the search results, none of these are in "real" science journals. This is just yet another way in which predatory journals are terrible for science and academia (and its public perception), but the existence of garbage journals has been a problem for 2 decades (or more?).
On the bright side, productivity at respected institutions has not been significantly hampered by predatory journals. On the other hand, academia as a whole has obviously failed to teach incoming members that this is basically fraud (partially due to the publish-or-perish culture that is now even endorsed by governments).
https://dbrech.irit.fr/pls/apex/f?p=9999:1::::::
It's a constant crawl of the literature that uses hand-written phrase detectors. Many of the phrases it looks for are caused by using spinners on plagiarised text, like this one:
https://link.springer.com/article/10.1007/s11042-024-18524-1
"At first, the input picture is transformed into the ruddy, green, and blue organize, and the clamor within the green band is evacuated using a median channel"
Here "noise" has been turned into "clamor". Nonsense text like this is easy to find in journals published by well known brands, in this case Springer Nature. A search for "profound learning" (deep learning) yields over 2000 results:
https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&as_ylo...
That rule no longer works. People are still going to rely on it for a while though, and I'm worried it's going to break some stuff over the next few years.