News for hackers. Stuff that matters.
Re:Recent Advances in Natural Language Processing by FiberBundle on Sunday August 16, 2020 @05:10PM (#24181347)
I found the science exams results interesting and skimmed the paper [1]. They report an accuracy of >90% on the questions. What I found puzzling was that they have a section in the experimental results part where they test the robustness of the results using adverserial answer options, more specifically they used some simple heuristic to choose 4 additional answer options from the set of other questions which maximized 'confusion' for the model. This resulted in a drop of more than 40 percentage points in the accuracy of the model. I find this extremely puzzling, what do these models actually learn? Clearly they don't actually learn any scientific principles.

[1] https://arxiv.org/pdf/1909.01958.pdf

Re:Recent Advances in Natural Language Processing by wrs on Sunday August 16, 2020 @05:26PM (#24181474)
I would be interested in hearing the results from humans presented with adversarial answer options. You may say that a machine learning correlations between words isn’t really learning science, but I wonder how many human students aren’t either, just pretty much learning correlations between words to pass tests...
Re:Recent Advances in Natural Language Processing by FiberBundle on Sunday August 16, 2020 @05:45PM (#24181624)
They do give an example of a question, in which the model chose an incorrect answer in the adversarial setting:

"The condition of the air outdoors at a certain time ofday is known as (A) friction (B) light (C) force (D)weather[correct](Q) joule (R) gradient[selected](S)trench (T) add heat"

I assume this might be characteristic for other questions as well, although I don't know anything about the Regents Science Exam and whether there are multiple questions about closely related topics.

Re:Recent Advances in Natural Language Processing by taneq on Sunday August 16, 2020 @08:16PM (#24182532)
That’s a terribly worded question anyway. Of the original answers, ‘weather’ is the least worst but it’s still vague.
Re:Recent Advances in Natural Language Processing by mannykannot on Sunday August 16, 2020 @09:36PM (#24182996)
It is a well-worded question for its purpose. The whole point is that, of all the options given, only one is justifiable (and it does not require a tendentious stretch to justify it, either.) Even “light” (which was not chosen) only applies half the time, on average. This is a valid test of natural language understanding.
Re:Recent Advances in Natural Language Processing by rvense on Monday August 17, 2020 @04:17AM (#24185142)
Remember when IBM went on Jeopardy? There was a question about which Egyptian pharaoh. A human with some knowledge of history might mix up Ramses and Seti, or whatever, or just not know the answer, but know that they didn't know. Watson answered "What are trousers?" with supreme confidence.

Jeopardy is fun and games and it was great for the blooper reel, but they're trying to sell this stuff to diagnose cancer and guide police efforts. Failure modes are kind of important.

Re:Recent Advances in Natural Language Processing by jacobwilliamroy on Sunday August 16, 2020 @06:03PM (#24181748)
Most multiple choice math problems can be completely circumvented by simply finding the digital root of the expression in the problem. I was surprised to find this to be true, even on college entrance exams.
Re:Recent Advances in Natural Language Processing by flir on Sunday August 16, 2020 @06:06PM (#24181770)
Seriously? Can you explain the mechanism? 'cos that sounds like - well, numerology, to be honest.
Re:Recent Advances in Natural Language Processing by jacobwilliamroy on Sunday August 16, 2020 @07:04PM (#24182141)
The digital root of the problem expression will be the digital root of the answer, because the answer IS the problem expression. They're the same number, just with all the pieces jumbled up. Comparing against digital roots of answer choices will usually eliminate most of the wrong answers. Any remaining wrong answers tend to be obviously wrong. 2 + 2 != 13 obviously.
Re:Recent Advances in Natural Language Processing by plutonorm on Sunday August 16, 2020 @07:41PM (#24182365)
Can you give an example? I am intrigued, although suspect you are making a joke!
Re:Recent Advances in Natural Language Processing by jacobwilliamroy on Sunday August 16, 2020 @08:47PM (#24182696)
You can check the properties of digital roots on wikipedia:

https://en.wikipedia.org/wiki/Digital_root#Properties

Example from ASVAB practice math test:

(x+4)(x+4) =

A. x^2+16x+8

B. x^2+16x+16

C. x^2+8x+16

D. x^2+8x+8

Since we don't have to solve for X in this problem, we can just assume x is 1, which would make the digital root of the problem expression 7. Assuming x = 1, the digital roots of A, B, C and D are 7, 6, 7 and 8 respectively. C is the answer because its last term is the square of 4.

Re:Recent Advances in Natural Language Processing by plutonorm on Sunday August 16, 2020 @09:14PM (#24182846)
Perhaps I am being dumb, but I don't see what the digital root brings to this.

You can do the same thing without the digital root. substitute 1 into all the possible answers gives: A: 25 B: 33 C: 25 D: 17

Substitute 1 into the question gives 25. Therefore only A and C are possible answers and it must be C because 4*4 = 16

This is the same process as your answer but without the digital root!

Re:Recent Advances in Natural Language Processing by taneq on Sunday August 16, 2020 @11:17PM (#24183526)
Agreed. The real trick here is knowing that, since we don't need to find x, we can sub in an arbitrary x and do the comparison numerically instead of algebraically.

Of course there's a tiny chance that one of the other equations would be equal at your arbitrary x value. If so and you get two 'right' answers just try again with a different x.

Re:Recent Advances in Natural Language Processing by benibela on Monday August 17, 2020 @09:00AM (#24186785)
It is the DeMillo-Lipton-Schwartz–Zippel lemma

If two polynomials evaluate to equal numbers with random variables values, the polynomials are almost certainly equal themselves

Ofc, x = 0 and x = 1 are hardly random

Re:Recent Advances in Natural Language Processing by jacobwilliamroy on Sunday August 16, 2020 @09:54PM (#24183108)
I suppose such is possible without digital roots. Let's try the same problem, but this time we'll multiply our constants by 3,571:

(x + 14284) (x + 14284)=

A: x^2 + 204032656x + 28568

B: x^2 + 204032656x + 204032656

C: x^2 + 28568x + 204032656

D: x^2 + 204032656x + 204032656

The digital root of the problem expression is 4. The digital roots of A, B, C and D are 4, 6, 4 and 3 respectively. The digital root of the square of 14284 is 1, and the digital roots of the last term of A and C are 2 and 1, respectively. The answer is C.

Re:Recent Advances in Natural Language Processing by dragonwriter on Sunday August 16, 2020 @09:59PM (#24183137)
Of course, it is quicker just to multiply and add the last digit of the constant to find the only possible answer (or count digits to get an order of magnitude estimate, which will also show C as the only possible answer).

When a problem of this form uses big enough numbers to warrant a shortcut, digital root is almost always a suboptimal shortcut.

Re:Recent Advances in Natural Language Processing by jacobwilliamroy on Monday August 17, 2020 @02:11AM (#24184465)
Yes you are the heuristics master!
Re:Recent Advances in Natural Language Processing by callmekit on Sunday August 16, 2020 @09:49PM (#24183083)
Maybe is case of large numbers you can compute digital root easier than the answer.

For example, 25 * 26 = 650, digital root 2. Or take digital root of 25 and 26 first, get 7 * 8 = 56, same digital root 2.

Re:Recent Advances in Natural Language Processing by flir on Sunday August 16, 2020 @08:54PM (#24182740)
I can't decide if I've just seen magic, or conjuring. I'm gonna have to go away and think about this.
Re:Recent Advances in Natural Language Processing by dragonwriter on Sunday August 16, 2020 @08:51PM (#24182714)
This is true, but in many cases amounts to reducing the problem to a more time consuming problem, though one that requires less knowledge. On timed tests, this can be a big negative.
Re:Recent Advances in Natural Language Processing by 0-_-0 on Sunday August 16, 2020 @06:41PM (#24181991)
Adversiarial training means that they specifically search for answers that the network would misunderstand. If this only leads to a 40 percent loss (the network still answers correctly 50% of the time) I still consider that remarkable.

Choosing the best from 8 answers where some of them were adversarially derived should be equivalent to choosing the best from all possible answers, of which there could be tens of thousands. How would a human do in that situation?

Although the kinds of mistakes the network makes seem like a mistake you would never do (i.e. you wouldn't call the condition of air outdoors gradient), the opposite could also be true, that it would easily answer questions you would have a problem with.

Re:Recent Advances in Natural Language Processing by mannykannot on Sunday August 16, 2020 @10:29PM (#24183304)
I admire the way that you have reinterpreted the issue, but I don’t buy it.

For one thing, I can imagine a knowledgeable human working through ten thousand alternatives and picking the best, (if, objectively, there is one) - it would just take a long time. On the other hand, It does not seem obvious to me that current NLP systems would do better than humans on such a task, and nor is it obvious to me that one can conclude that from the fact that the machine sometimes ignores adversarial examples (if the assumption is that most of the ten thousand would be de-facto adversarial, that is a lot to not make one mistake on; that’s not so much a problem for a human with an understanding of the issue, and who might well come up with the right answer unprompted.)

This is probably all moot, however, as the most obvious way to compare the machine results to those of humans is to give both the actual tests in question, rather than substitute a dubious “equivalent” test.

Re:Recent Advances in Natural Language Processing by teej on Sunday August 16, 2020 @06:19PM (#24181851)
That’s the thing. Machine Learning is a misnomer, the models don’t “learn” anything about the domain they operate in. It’s just statistical inference.

A dog can learn to turn left or to turn right for treats. But they don’t understand the concept of “direction”, their brain isn’t wired that way.

Machine learning models perform tricks for treats. The tricks they do get more impressive by the day. But don’t be deceived, they aren’t wired to gain knowledge.

Re:Recent Advances in Natural Language Processing by plutonorm on Sunday August 16, 2020 @06:38PM (#24181972)
My god if I hear this argument one more time I'm going to pop. What on earth gives you the idea that you are something more than statistical inference?
Re:Recent Advances in Natural Language Processing by teej on Sunday August 16, 2020 @06:53PM (#24182083)
Yes, it’s widely believed that statistical inference is a part of how the brain operates. But we have barely scratched the surface in our understanding how the human brain works.

Do you honestly believe statistical inference completely explains a human’s ability to learn?

Re:Recent Advances in Natural Language Processing by plutonorm on Sunday August 16, 2020 @07:37PM (#24182338)
Of course.
Re:Recent Advances in Natural Language Processing by ggrrhh_ta on Sunday August 16, 2020 @10:00PM (#24183144)
I think that the reason your mind pops is that you are not trying to understand what the person saying that is actually meaning.

Stating that we are no more than statistical inference is no different to saying that we are no more math - really, you can build the deepest mathematical truths on statistical inference.

The thing is that the math/s.i. that ML models do is too trivial in comparison to the math that would be required to describe a human identity. It can detect faces very well, it can calculate fifths roots very quickly, it can solve discrete problems quickly, yes, but its achievements are, in comparison to actual human understanding (and purpose), more closer to those of another tool, like a hammer, than those of humans.

edit: typos/grammar

Re:Recent Advances in Natural Language Processing by plutonorm on Monday August 17, 2020 @12:50AM (#24184102)
I hear what they are saying and I reject it as trivially incorrect. Of course all we are is math. What else do you think there is?

"The thing is that the math/s.i. that ML models do is too trivial in comparison to the math that would be required to describe a human identity" why do you think this? How do you know the expressive power of neural networks? How have you guaged the expressive power of biological neural networks. How have you made any of these conclusions?

I am exasperated.

Re:Recent Advances in Natural Language Processing by ggrrhh_ta on Monday August 17, 2020 @09:59AM (#24187259)
You are exasperated because when I wrote "that ML do is too trivial..." you did not realize that for ML I meant "the current state of the art of ML"...

I never assumed we are more than math. Just that there is a lot to still learn about the structure in which those buildings blocks are organized in order to fully tame what is understood as human intelligence. If we could simulate all the neurons of a real adult living brain accurately enough it would be intelligent from a human perspective, and still, both the model and us would be equally far, very far, from understanding how to arrive to that point of neural organization to achieve that kind of intelligence.

Re:Recent Advances in Natural Language Processing by 20after4 on Monday August 17, 2020 @02:46AM (#24184639)
For a moment I had myself convinced that your response was written by GPT-3. Part of me hopes that is the actual fact.
Re:Recent Advances in Natural Language Processing by bsenftner on Sunday August 16, 2020 @11:41PM (#24183679)
Because statistical inference does not create complex operating models designed to fit a subset of reality from a larger encompassing model of reality which humans base our daily decisions and life philosophies. Just because we do not deep dive into the complexities of existential questions every waking moment does not say we're not operating scaffolds which depend upon prior existential examinations. We're complex to say the least, and we require reasons - and that very idea: reason is what is not present in statistical inference. Without a prior body of human conversation, statements, and larger expressions of philosophy based upon human reason, statistical inference goes nowhere.
Re:Recent Advances in Natural Language Processing by hackinthebochs on Monday August 17, 2020 @01:10AM (#24184195)
>reason is what is not present in statistical inference.

But why think reason isn't a kind of statistical inference? GPT-3 demonstrates a rudimentary capacity for reasoning and its "merely" a statistical inference engine.

Re:Recent Advances in Natural Language Processing by randcraw on Sunday August 16, 2020 @09:56PM (#24183118)
Because it is possible to invent or imagine new things without resorting to endless random walks or blind trial and error.

Induction isn’t probabilistic. It’s the origin of all discovery and based in sussing out what’s important in patterns, selectively proposing causal mechanisms and testing those hypotheses by following priors and implications — an essential basis for original and creative thought that no purely probabilistic engine can employ.

Re:Recent Advances in Natural Language Processing by FeepingCreature on Monday August 17, 2020 @12:51AM (#24184107)
You think a probabilistic engine can't model a human doing induction? At a certain level of capability, mere pattern imitation closes upwards.
Re:Recent Advances in Natural Language Processing by 0-_-0 on Sunday August 16, 2020 @06:45PM (#24182026)
Doesn't your brain derive knowledge through statistical inference? How else? I can imagine the AGI of the future (having perfected knowledge gathering through statistical inference) some day saying: "Humans perform tricks for treats. The tricks they do are impressive. But don’t be deceived, they aren’t wired to gain knowledge."
Re:Recent Advances in Natural Language Processing by teej on Sunday August 16, 2020 @06:57PM (#24182108)
I didn’t claim that the way brains work didn’t include statistical inference. That’s why the dog metaphor works. Both the dog and the machine learning model use statistical inference to perform tricks.

Dogs haven’t been to the moon, however. There’s more to the brain we don’t understand.

Re:Recent Advances in Natural Language Processing by esrauch on Sunday August 16, 2020 @07:31PM (#24182307)
I'm curious why you think a "difference in kind" rather than simply "difference in degree" must exist between us and dogs.
Re:Recent Advances in Natural Language Processing by MiroF on Sunday August 16, 2020 @09:08PM (#24182810)
Adversarial examples probably exist for humans too, they are just less easy to find since we can't easily backprop through human judgements like we can a bunch of array multiplications and dot products.

Doesn't mean that they are not "actually" learning any more than we are.

Re:Recent Advances in Natural Language Processing by callmekit on Sunday August 16, 2020 @09:56PM (#24183117)
These are "adversarial answers" - additional answers to a multi-choice test. If you know the correct answer to the test, adding more possible answers shouldn't make you change your selection.