As @braddwyer himself notes, you can probably get a much better mesh integrating over time. It depends how long it takes to capture a single frame, but I imagine that's not long, so getting an order of magnitude improvement is probably quite easy.
> 4) The scary part, which is vast dystopian databases of facial fingerprints is already being done with normal photographs. The depth data is not needed.
And yes ... after all, humans are quite capable of identifying people with high accuracy from 2D photographs. Depth maps are not required for there to be serious privacy issues with such databases.
It gives you about 15 fps of depth data
Given the small size of the laser projector, I imagine natural movement from the phone being hand-held would result in significant displacement of the projected dots over a 1s interval? Have you tried integrating the 15 frames to see what it looks like?
We submitted a game about 3 weeks ago using front-facing ARKit as its core game mechanic and it hasn't been approved by Apple yet.
I'm waiting to see if they're going to allow us to use the new technology in novel ways or not before I invest a lot more time in it.
[1] https://en.wikipedia.org/wiki/Simultaneous_localization_and_...
Yes and no. We’re mostly good at that (with exceptions — I have to see someone a lot before I remember their face), but we evolved for small groups and therere are now enough people that doppelgänger is a profession.
On the other hand, databases are still a problem because collections of timestamped photos can reveal far too much about us once an identity is properly confirmed.
Which would be true if they were actually storing this on the cloud in a form they could access. As far as we know, they are not. That's the point of the "Secure Enclave".
This is how most (all?) state of the art acquisition methods for standard objects (faces, hands, etc) work. By warping a high res template to fit the data in some optimal manner, you get guarantees on the output topology without having to do tons of messy cleanup.
I'd be curious to know if face unlock had problems with vision impairment, i.e. no gaze vector.
Weren't they specially created to make use of that 3D cam?
It’s possible to generate reasonable 3D models of faces from a single photograph. [1]
The highest resolution 3D scans I’ve seen are produced by aligning data from multiple high resolution photographies.
The big problem with that aproach is that it requires a lot of detail in the source material. Smooth surfaces, blurry images, or noise because of poor lighting makes it impossible for the algorithm to find features to align.
This is where the dot matrix projector comes in: by projecting a bunch of dots on your face, you get features that the algorithm can align, making the scan faster and more robust in low light.
Pretty neat stuff, though I've never found any actual artistic or practical use for it.
3.1) The lack of Animoji and front-facing ARKit seems to be a software limitation made for business reasons rather than a hardware limitation. See: Google's Pixel 2 portrait mode photos done using a single front-facing camera that have stacked up well against the ones from the iPhone X.
Does it also work in the dark without the depth camera?
I was really surprised how well it does even when covering up the IR sensor prior to opening the app.
I don't doubt that they are using the IR data to improve things. But it does "good enough" without.
> The reason for the misconception comes from the implementation: The IR system only (currently) fires periodically to create and update the depth mask. The RGB camera has to capture persistently to track movements and match expressions. In other words, cover the IR system and the depth mask will simply stop updating and likely, over time, degrade. Cover the RGB, and the tracking and matching stops dead.
"...likely, over time, degrade."
1) He doesn't know.
2) It's Animoji, so why would it matter if it did degrade? There is already a stock 3D image of the Poop. It simply needs the RGB camera to track where your facial features are.
The A11 chip has dedicated “neural engine” hardware which is used for Animoji and other facial recognition tasks.
How much could be done in the standard CPU on other devices I’m not sure.
Well, lets just say there's the kind of scary fact that nobody, trustworthy, has audited this thing.
Like, should a company that didn't run a "doesn't let root login on first-try" test be allowed to be making such wide-ranging decisions as face-scanning?
What if I don't want to have my face scanned, but nevertheless need to pick up somebody's lost-phone/detonation-device? Shall I just wear a mask?
The point is that we have moved beyond a zone where 'disagree/-agree' means anything, any more. Our data is out there.
Not so sure I want my face involved where, preferably, my hands should be..
Well...yeah. If you're out in public and your face is visible, you don't have a reasonable expectation of privacy.
Does that make it even the tiniest bit more acceptable, or does that mean it's really high time to stop it? Being an apologist kind of hinges on that, and no need to put anything in quotes.
https://qz.com/1131515/google-collects-android-users-locatio...
Many privacy-conscious settings that I've been in prohibit phones entirely.
Like, its a cool technology - sure. But has nobody thought of the militarisation of it? Sheesh.
Exactly what I've said from day one - that Apple's "FaceID is 50x more secure than Touch ID" claim, based on the False Acceptance Rate, was total bullshit. That only works if you're going to throw random data at the authentication mechanism.
But someone who's going to target you isn't going to do that. It's going to you a 3D profile of your face from your online photos or from CCTV cameras (to which not only the government has access, but hackers, too).
In practice, it's much more difficult to gain a clone of someone's fingerprint than it is to gain a clone of their 3D face.
Good thing I'm not Jason Bourne, and the most likely scenario for someone trying to get into my phone involves my sister's kids.
1) The ARKit "Face Mesh" seems to be a standard model that is scaled and skewed to fit your face (for example, it ignores glasses, still works if you put your hand in front of your face, etc). It is _not_ a 3D scan.
2) The "TrueDepth" data is not really all that granular. It seems similar to the depth map you get from the rear-facing cameras on the "plus" sized models. Here's what the sensor data spits out: https://twitter.com/braddwyer/status/930682879977361408
3) Apple is really good at marketing. It's been shown that, even if you cover the TrueDepth camera, features that "require" it still work fine (including Animoji and the apps that I've been developing using the front-facing ARKit APIs).
3.1) The lack of Animoji and front-facing ARKit seems to be a software limitation made for business reasons rather than a hardware limitation. See: Google's Pixel 2 portrait mode photos done using a single front-facing camera that have stacked up well against the ones from the iPhone X.
4) The scary part, which is vast dystopian databases of facial fingerprints is already being done with normal photographs. The depth data is not needed.
I agree with the author that the privacy implications of all-encompassing databases could be scary. But I disagree that this has anything to do with the iPhone X or its TrueDepth camera.