News for hackers. Stuff that matters.
Re:Tricks That Can Outsmart Deepfake Videos for Now by amluto on Saturday October 20, 2018 @09:49AM (#18263288)
One technique I haven’t seen mentioned would be to take advantage of the fact that the world is 3D but neural networks that make fake videos are somewhat fundamentally 2D. Making good 3D computer graphics is hard and involves a lot of lighting calculations that are very different from what neural networks do. It seems like it should be possible to detect the errors that a neural network will introduce that will make the scene unphysical, at least in cases where more is modified than just small bits of geometry like the position of someone’s lips and other facial features.
Re:Tricks That Can Outsmart Deepfake Videos for Now by chongli on Saturday October 20, 2018 @10:14AM (#18263382)
If you can write a function to quantify those errors, then you can use it to train the neural networks that produce the deepfakes. The better your error function, the better the deepfakes.
Re:Tricks That Can Outsmart Deepfake Videos for Now by nothrabannosir on Saturday October 20, 2018 @10:25AM (#18263419)
Is that always true ? I know nothing about AI whatsoever but wouldn’t the function have to satisfy certain properties for a NN to be able to learn from it? Properties that the discussed “3D verifier function “ wouldn’t necessarily have?

It sounds very much like complexity theory, where you definitely have problems that are easier to verify than solve (see NP hard).

(Otherwise I could do stuff like use a hash function to train a NN to compute input for hashes , for example, no?)

Re:Tricks That Can Outsmart Deepfake Videos for Now by avivo on Saturday October 20, 2018 @01:32PM (#18264351)
I think this is a valuable framing. In other words, is there any computation problem which physics implicitly solves in the process of generating an image or video—and which is:

1) computationally intractable to forge. (e.g. requires simulating trillions of photons)

2) computationally tractable to verify

Sadly, my immediate gut intuition is that there is not such a problem, for a variety of reasons; but hopefully I'm wrong!

Re:Tricks That Can Outsmart Deepfake Videos for Now by kiallmacinnes on Saturday October 20, 2018 @10:41AM (#18263485)
I don't know much about AI and NN's either, so grain of salt and all that..

In theory, yes - you can. The training function could return a score - say the number of matching digits in the hash - and the NN will in theory learn what inputs produce the better output. If somehow, there is a weakness in the hash algorithm, it could stumble onto it - allowing it to get better at producing the right input to get the required output.

The simpler it is to deterministically manipulate, the easier it should be for a NN to learn to manipulate - even of the function is just returning a boolean or a "rating between 1 and 10". So yea, good hash functions are unlikely to be learned and solved, but photos and videos aren't designed to be hash functions.

(All that said, I'm willing to be bet the time and computing power you'd need to pour into this NN to break a good hash function is likely more than has ever existed in the sum of all past time + computing resources ;))

Re:Tricks That Can Outsmart Deepfake Videos for Now by chongli on Saturday October 20, 2018 @10:50AM (#18263518)
It sounds very much like complexity theory, where you definitely have problems that are easier to verify than solve (see NP hard).

I think you mean NP. NP hard includes problems that are not in NP, such as the halting problem.

I'm not aware of any proof that photorealistic 3D rendering is in NP. If it is not, then verifying 3D cannot be said to be easy.

Re:Tricks That Can Outsmart Deepfake Videos for Now by nothrabannosir on Saturday October 20, 2018 @11:31AM (#18263700)
Well, not exactly regular NP, as was pointed out. I guess NP complete would make more sense. Although the set of problems that are easier to validate than solve reaches much further than NP complete; see basically any Σ in the polynomial hierarcy. It's about the existential quantifier.
Re:Tricks That Can Outsmart Deepfake Videos for Now by krastanov on Saturday October 20, 2018 @10:58AM (#18263552)
Nitpicking on your nitpick: you mean NP-complete, not NP (for instance, any P problem is an NP problem, but NP-complete problems are probably not P)
Re:Tricks That Can Outsmart Deepfake Videos for Now by mygo on Saturday October 20, 2018 @11:25AM (#18263669)
You’re correct, but I’m gonna have to nitpick your nitpick and say that NP-complete problems have not been proven (or disproven) to be in P. “Probably not” is an opinion.
Re:Tricks That Can Outsmart Deepfake Videos for Now by valvar on Saturday October 20, 2018 @01:24PM (#18264302)
It's a well-informed opinion that is based on the fact that what we do know indicates[1] that, probably, P≠NP.

[1] https://en.wikipedia.org/wiki/P_versus_NP_problem#Reasons_to...

Re:Tricks That Can Outsmart Deepfake Videos for Now by mygo on Saturday October 20, 2018 @09:40PM (#18266367)
Publish a paper with the mathematical proof. “probably not” is neither here nor there.

The main argument from your link is nobody has found an efficient algorithm for any of the 3,000+ studied problems after all this time.

And yet it’s also true that in the same span of time, nobody has been able to prove the that it’s not, either. No matter how hard they have tried.

“probably not” is just an opinion. I’ll wait for the formal proof. Till then it simply is not known whether or not.

Re:Tricks That Can Outsmart Deepfake Videos for Now by currymj on Saturday October 20, 2018 @11:13AM (#18263614)
its not always true (hash functions are in some sense designed by brilliant people to be as hard to learn as possible) but it’s true often enough to be surprising.

if there was some score that would look at lighting etc, you probably could just tack it on to one of the loss functions somewhere and expect to see improvement.

maybe not enough to beat the system; or perhaps it would increase the lighting score but make the image obviously unrealistic in other ways we haven’t thought of.

Re:Tricks That Can Outsmart Deepfake Videos for Now by confiscate on Saturday October 20, 2018 @03:41PM (#18265020)
ya but in this case, wouldn't you have to program a really complex loss function, to track a lot of things like lighting, edge detection, granularity, video compressions artifacts etc.

basically at this point you would have a lot of if-else statements, like a "Expert System" AI in the 80s.

so much so that it would be better to use a NN to replace the loss/error/verification function instead of coding it by hand?

Re:Tricks That Can Outsmart Deepfake Videos for Now by sniuff on Saturday October 20, 2018 @10:33AM (#18263449)
It will keep generating stuff until it improves score of the function, not try to satisfy the function.
Re:Tricks That Can Outsmart Deepfake Videos for Now by 3pt14159 on Saturday October 20, 2018 @01:07PM (#18264213)
Yeah, but we don't generate prime numbers with them by scoring them based on how few common factors they have. Pattern matching only works for certain things. At some point it's necessary to conceptualize at which point I'm pretty convinced you have hard AI.
Re:Tricks That Can Outsmart Deepfake Videos for Now by dagss on Saturday October 20, 2018 @10:38AM (#18263464)
Neural networks are not magic.

You can't solve the travelling salesman problem in polynomial time just by throwing machine learning at it...

On a mathematical level you can prove that machine learning will do a bad job for certain things, and this is what the parent talked about.

Re:Tricks That Can Outsmart Deepfake Videos for Now by lmm on Saturday October 20, 2018 @04:54PM (#18265410)
Is there any reason to believe generating 2D video with the properties that a recording of a 3D world has is a difficult mathematical problem though?
Re:Tricks That Can Outsmart Deepfake Videos for Now by amluto on Saturday October 20, 2018 @01:25PM (#18264309)
But you can solve TSP pretty well with a TSP solver. I’d be impressed if a neural network did that well. But maybe it’s possible.
Re:Tricks That Can Outsmart Deepfake Videos for Now by currymj on Saturday October 20, 2018 @03:53PM (#18265080)
See Dai et al, "Learning Combinatorial Optimization Algorithms Over Graphs", experiment section A.1.3 "Traveling Salesman Problem".

https://www.cc.gatech.edu/~lsong/papers/arxiv_rl_combopt.pdf

It does at least respectably well against the Concorde TSP solver although (unsurprisingly) doesn't beat it. Note that this also isn't a neural network that just maps a graph to a TSP solution; instead, the neural network function is a heuristic that guides the greedy construction of a solution one vertex at a time.

Re:Tricks That Can Outsmart Deepfake Videos for Now by Nasrudith on Saturday October 20, 2018 @07:27PM (#18266006)
That sounds like an approach prone to backfiring if not done carefully. The real world is noisy - something too perfect would stand out like a sore thumb. Training on an overly large set would likely wind up inconsistent and overly small one would probably be too biased.

For instance one fundamental of forensic accounting is Benford's law [1]. Even in peripherally related with artistry it stands out. one of my never off the ground webcomic attempts with digital art I tried making backgrounds with perspective and I calculated the pixel spacing completely evenly - it stood out as unnaturally regular compared to doing it with just a straight-edge and a ruler on a cheap Wacom tablet.

[1]: https://en.wikipedia.org/wiki/Benford%27s_law

Re:Tricks That Can Outsmart Deepfake Videos for Now by netheril96 on Saturday October 20, 2018 @12:10PM (#18263908)
Except that the error function is not differentiable.
Re:Tricks That Can Outsmart Deepfake Videos for Now by malmsteen on Saturday October 20, 2018 @04:20PM (#18265212)
Thats what a gan could do automatically nah ?
Re:Tricks That Can Outsmart Deepfake Videos for Now by IshKebab on Saturday October 20, 2018 @11:32AM (#18263705)
That is what capsule networks were designed to solve. So far though I've only heard they got good results on MNIST and promising results on CIFAR10.

Anyway I'm not sure that you can conclude that just because convolutions are 2D that deep neural networks are unable to understand 3D relations. For example this network seems to have a pretty good understanding of the 3D world:

https://deepmind.com/blog/neural-scene-representation-and-re...

Re:Tricks That Can Outsmart Deepfake Videos for Now by ezoe on Saturday October 20, 2018 @12:27PM (#18263995)
I imagined the interview video where multiple light sources constantly moving for the proof that the lighting is real.

Or more likely these NN-generated video fake the poor quality video and audio. The fake video author claims it was captured by hidden camera and faked person admit the fake illegal act.

Re:Tricks That Can Outsmart Deepfake Videos for Now by umanwizard on Saturday October 20, 2018 @05:54PM (#18265700)
What do you mean by neural networks being 2D? How do you define the dimensionality of a neural network?
Re:Tricks That Can Outsmart Deepfake Videos for Now by amelius on Saturday October 20, 2018 @02:05PM (#18264552)
Look up "adversarial networks". If you have a classifier that tells when an image is faked, then you can use that classifier to train a network to generate images which make the classifier say the images are real.
Re:Tricks That Can Outsmart Deepfake Videos for Now by confiscate on Saturday October 20, 2018 @03:34PM (#18264986)
ya but isn't the adversarial network in this case just a more sophisticated version of the error function (i.e. verifying function) the others have talked about?