News for hackers. Stuff that matters.
Re:Transformers as Support Vector Machines by kristopolous on Sunday September 03, 2023 @05:53AM (#37369086)
I really really wish this culture of expressing simple things in ornate ways would die. All it does is make knowledge less accessible
Re:Transformers as Support Vector Machines by tinco on Sunday September 03, 2023 @06:38AM (#37369272)
Of the excuse is true and the "ornate" language really is a dense representation of information then it should be fairly trivial to have an LLM agent unsummarize it.

There could be a webservice that offers a parallel track of layman's translations of any paper.

Re:Transformers as Support Vector Machines by heydenberk on Sunday September 03, 2023 @08:22AM (#37369824)
Sounds like you're referring to https://www.explainpaper.com/ :)
Re:Transformers as Support Vector Machines by noduerme on Sunday September 03, 2023 @08:30AM (#37369888)
An LLM won't tell you that the authors obfuscated it because they don't know what the fuck they did. You need a human for that.
Re:Transformers as Support Vector Machines by estebarb on Sunday September 03, 2023 @11:53AM (#37371385)
I haven't tried other models, but if you prompt a recent ChatGTP with "academic style" and ask it to "review and provide feedback" of a paragraph you wrote it will reword it using the most fancy, overselling words it can find. I liked to use it for improving grammar and style, but in later iterations ChatGTP started writing garbage...

I'm not sure if that is because training, feedback from users or an attempt to make usage is LLMs obvious to teachers.

Re:Transformers as Support Vector Machines by jimkoen on Sunday September 03, 2023 @05:58AM (#37369102)
People have to get their PhD's somehow... ;)
Re:Transformers as Support Vector Machines by noduerme on Sunday September 03, 2023 @08:23AM (#37369834)
I think this applies to everything right now. Papers like this are just ridiculous examples. In like, 6th grade I won second place at the LA county science fair for coding a simulation of a coyote's life in hypercard (with tons of graphs). Yay. Y'know what? That shit and those graphs would've been incomprehensible to the judges if it hadn't been written in plain language, in an attempt to make them understand what they were looking at. My entire career since has been an attempt to communicate and alleviate the pain points in communication between parties, by way of writing software that encapsulated their descriptions of what they needed. And likewise I never pretended to be smarter or know more than my clients did: Everything must be explained and comprehensible in normal people language. People need to know how shit works, especially if they're paying for it.

Or they should.

Or if they don't know and don't care, they're fucking negligent.

Especially if they say "wow that sounds smart, let's let these guys run our weapons program".

To your point, the reason this ornate language thrives and people get away with complacency about how their own systems work, boils down to a silent pact between managers and engineers to sweep everything under the rug out of laziness and ill-will. There's something blatantly mendacious and evil (in the banal way) about the agreement that managers approve black boxes which were approved by complex-sounding papers so that upper management can wash their hands of the results.

[edit] maybe I'm just bitter because I spent hours today pondering exactly how many engineers at Monsanto must have known about the dangers of the astroturf, and how many raised their hand, or hid behind a spreadsheet

https://frontofficesports.com/investigation-links-astroturf-...

Re:Transformers as Support Vector Machines by pedrosorio on Sunday September 03, 2023 @09:29AM (#37370272)
“Engineers who like to pretend to be mathematicians” - I heard once.
Re:Transformers as Support Vector Machines by kristopolous on Sunday September 03, 2023 @09:45AM (#37370387)
In math, this is mostly an English problem I think. Next time you find a Wikipedia math page to be an impenetrable wall of jargon, click the Wikipedia language tool and choose another language, any will do.

Then use Chrome's tool to machine translate the foreign language version back to English. I've found invariably this makes the article more coherent then the native English language Wikipedia math page.

It says something about the culture for sure.

Re:Transformers as Support Vector Machines by novalis78 on Sunday September 03, 2023 @10:41AM (#37370808)
This works for philosophers too. Try Schopenhauer (already easy to read in comparison) in English translation.
Re:Transformers as Support Vector Machines by thechao on Sunday September 03, 2023 @10:31AM (#37370735)
I know a lot of professional mathematicians who can't make their way through the Wikipedia articles of adjacent fields. The English entries appear to be written by the ghost of Bourbaki!
Re:Transformers as Support Vector Machines by consilient on Sunday September 03, 2023 @10:50AM (#37370882)
Some of them are: I've run into sections lifted straight of of Lang's Algebra several times.
Re:Transformers as Support Vector Machines by djmips on Monday September 04, 2023 @03:07AM (#37377914)
Since LLM are good at translating perhaps that's one of the reasons they seem to do a decent job of explaining things.
Re:Transformers as Support Vector Machines by CuriouslyC on Sunday September 03, 2023 @09:53AM (#37370444)
That's literally the entire field of philosophy after the ~18th century.
Re:Transformers as Support Vector Machines by FrustratedMonky on Sunday September 03, 2023 @11:12AM (#37371051)
Yep ~18 century Didn't Wittgenstein and/or Nietzsche say something similar. Words are in-adequate for communication, and all philosophy is playing with words.

But, Language is all we have to communicate, so guess we are stuck with it.

Re:Transformers as Support Vector Machines by TrackerFF on Sunday September 03, 2023 @10:45AM (#37370844)
"Not math-y enough"/ "Needs more math" is a very common feedback ML/AI researchers get when writing papers.

The other day I was watching a live-stream of a doctoral defense, as the thesis was quite relevant to my work.

So one of the committee members would really pick and criticize the math - ask questions like "You are supposed to be the bleeding edge on this topic, why was the math so simple? Did you research more rigorous theories to explain the math?" etc. (He was awarded the doctorate though)

So, I dunno, if that's how things are now - it makes sense to me that the authors go overboard with complicated notation, even if they could have written it much simpler. Probably makes the work seem more rigorous and legit.

Doesn't really take that much more time, and it covers your ass from "not rigorous enough" gotchas - though at the expense of readability.

Re:Transformers as Support Vector Machines by kristopolous on Sunday September 03, 2023 @11:00AM (#37370957)
Go read any article in the first 200 years or so of philtrans. There's lots of crucial science there written in a way that doesn't have the modern trappings of the form. It's good reading. Maybe some style perturbations borrowed from earlier eras would be good

https://www.biodiversitylibrary.org/bibliography/62536 menu on the right

Benjamin Franklin, Robert Boyle, Isaac Newton, Maxwell, Ohm and Volt - they're all there. If that style was good enough for them ...

Re:Transformers as Support Vector Machines by FrustratedMonky on Sunday September 03, 2023 @11:08AM (#37371021)
I wish also. When I was young and new, so much wasted time trying to parse the 'arcane' math that was really something simple bug dressed up as complicated to give it weight.
Re:Transformers as Support Vector Machines by mattew on Sunday September 03, 2023 @11:25AM (#37371137)
What I find entertaining/confounding is how difficult the abstracts to these new AI papers are to understand. It feels like academia is pushing this style, so it’s hard to blame the authors since they have to play the game.

For reference I have an undergrad degree in computer science, have been working professionally for 25 years, and am fairly data centric in my work.

I’m hoping when I run this through GPT4 to get an explanation for a mortal software developer something sensible comes out the other end.