Yeah it sounds like a great option to place a camera invisibly in the middle top half of your screen so you can actually look a person in the eye when videoconferencing. No more weirdly looking down or to the side for everyone.

I'm sure it can be used for creepiness as well but I see the benefits too.

iPhones solve the camera angle by changing the position of the pupil in software instead.

It’s a bit uncanny valley though, right?

I’ve never noticed it, so apparently not.

Nvidia Broadcast has a similar feature but it's really bad and completely unusable if you wear glasses. I don't have an iPhone to compare.

It seems to work fine with glasses for me. Only it loses tracking if I look too far away (bottom of the monitor). Also it thinks I have olive green eyes for some reason.

Given that this will probably only produce a relatively low res, high SNR image, it might be the sweet spot is to use this tech to capture a true eye-contact perspective from the glass in front of a screen, then neural net combine it with images from offset cameras, to create a 'true' through-the-screen view.

why stop there? one rendering for each eyeball

Are you sure? I remember that being present briefly in a beta several years ago, but then it was removed. I don't remember it ever coming back.

Some searching says it was trialed in iOS 13 betas, but didn't make it into the final cut. It was released with iOS 14 though.

https://ios.gadgethacks.com/how-to/disable-facetimes-creepy-...

Oh, TIL.

Which versions of iphones do this?

Any model newer than iPhone X (XR/XS and up).

It's a Facetime exclusive, though - no API so other videoconferencing apps can't use that.

1986 solution vs cyberpunk solution

how did we even get here?

Having the screen be the camera 1) removes the need for the holepunch as well as magic island workaround 2) could clearly be used in tandem to improve the effect, even potentially reducing the required computation. Sure, Google also fixes images in post, but that's still not a substitute for better cameras. You're always information limited.

I believe Nvidia offers something similar with Nvidia Broadcast[0]. [0] https://www.nvidia.com/en-us/geforce/broadcasting/broadcast-...

This doesn't solve the issue, because it looks like the person is always making eye contact! You don't want that—you want it to look like the person is making eye contact only when they are actually making eye contact. Constant eye contact is exhausting.

It does what??? How have I never heard about this before? Is this deployed right now? Is it always enabled? I have so many questions, don't even know where to start. That sounds amazing.

But how will people know you've got the latest flagship phone if it doesn't have bigger and more numerous camera bumps than last year's??

I never understood bigger and more numerous camera bumps until I used optical zoom and took photos at night. Image quality with thick lenses is incomparably superior to that of thin lenses and 3 image sensors collect 3x as many photons as 1 so you don't have to hold the phone still for such a long time. Ideally the entire backside of my phone would be lenses

and your 1500€ phone still sucks compared to my 500€ DSLR, lol

Yes but I have my iPhone everywhere and the pictures the 15 pro max takes are beyond ‘good enough’.

Even the zoom is pretty good nowadays.

Often the choice isn’t iPhone photo or DSLR photo it’s iPhone or no photo at all.

It’s a right faff getting the DSLR out and carrying it around. I brought mine to NYC over Xmas and ended up using my phone mostly.

Also no one I know prints photos out, they look at them on their phone, iPad or maybe TV if they care enough or have a screensaver slideshow.

I’d love to use my DSLR more and every year I tell myself I’m going to but I just… don’t.

I tried to go on holidays with my dslr… what a pain. So heavy (unless you buy a $3000 mirrorless) for photos I never print and the times I print them no one ever take the time to look at them. They stay in a closet until thrown out. The quality from my iPhone is not that great as a 4K monitor wallpaper compared to my DSLR. But given the fact that it weights nothing and on holidays the iPhone allows me to text, use GPS and google maps, play, read articles, watch videos and allows an immediate editing of my photos in raw and created a shared album right away … it’s a no brainer anymore

You're right about the strengths of the phone camera, and I also take many more pictures with it. But printing large prints, framing and hanging pictures is absolutely worth doing, much more than printing 4×5s to flip through. I love having them and everyone who comes into my home takes some time to look at them.

Just buy a used mirrorless camera, it’ll set you back a couple hundred, not 3k.

New technology is fun and all, but even decade old digital cameras are more than good enough for good holiday snaps that you might want to crop and print.

Buy any compact camera with a 30x optical zoom and it will take better pictures than a smartphone. Maybe not all of them, but you can zoom on details from a distance, take pictures of animals that would flee if you try to get close, etc. The last one I bought about 10 years ago was about 300 Euro, maybe 200. It has wifi to backup pictures to my phone.

I regularly use my DSLR and print photos with my own canon photoprinter. i bought a 70mm-300mm lens for a bronyconvention (galacon) and it is simply impossible to get images of that quality with a phone.

and the 50mm f1.8 is really great considering the low price.

Yes and I manually sort through thousands of sunflower seeds every year to get the good ones for my breeding program but our obscure hobbies aren’t the norm.

Now I want to know more about your sunflower breeding program :)

..until you find out it's for bronycon snack packs

Your 500 euros DSLR sucks compared to my 1500 euros phone for gaming. Do you carry your DSLR in your pocket every time you leave the house? These is obviously a matter of specialized vs general use.

Your 500€ DSLR isn’t a €300 phone, a €300 camera, a €300 navigation device, a €300 Sonos Remote, a €300 PDA all in one.

For the everyday needs of the average non-enthusiast consumer, I'd argue that an iPhone is simply better than a DSLR or mirrorless - not just more portable or more convenient, but capable of producing reliably better images.

Sure, the DSLR is the obvious choice if you're a serious photographer, but most people aren't serious photographers. If they buy a DSLR or mirrorless camera, they're going to use the kit lens and leave the mode dial on auto. For people who just want to point and shoot, the iPhone's computational brilliance shines through.

The iPhone isn't so much a camera as a generative algorithm that happens to use image sensor data as a prompt. That's infuriating if you're a photographer who just wants full control over a big sensor, but it's tantamount to magic if you don't know what an f-stop is and have no inclination to learn.

I'd bet that if you gave my mother an iPhone and a DSLR, she'd get consistently better images from the iPhone, even if we gave her a one-day crash course on photography first. Sure, she might fluke the odd decent photo with the DSLR, but 90% of the time, the iPhone's algorithmic guesswork is going to beat better imaging hardware with dumber software.

So we should call those shots snapshots when made on a phone. Well, the only thing you need to know when shooting with dslrs is to move the dial to Auto/P. If you want to play with DOF, move it to Av. If you want to shoot the dog, move it to Tv and 1/500. If you want to shoot an airshow move it to 1/1200 or higher. Is that too much? :D

I'm getting serious "you can build Dropbox yourself quite trivially" vibes here. I mean, people don't even turn their flash off in broad daylight…

It's really that easy. Ppl like to complicate things A LOT. Just to look more important.

I'm going to tell your mother you said that.

Phone-cameras produce muddy trash. Ever tried to make a good portrait with difficult light? That was the reason i began photographing seriously, because the photos i made with my phone on the galacon a year prior were trash.

And yeah u have to learn how to properly use a dedicated camera which is why i only shoot with manual settings, for casuals phones are enough.

Phone cameras haven't stopped improving in the last couple of years. Your data might be outdated

What phone? I've never owned one that took worse photos than a DSLR in auto mode under normal lighting

Yeah I seriously doubt that. Not without a lens that costs 3x-4x times the camera. If you spent 500€ on a kit that makes iPhone or Pixel photos look bad... you bought it stolen, lol.

I bought it new from a camera-store lol. Its the Canon EOS 2000D with the Canon EF 50mm f1.8. (I don't count my other lenses, bought them later).

And yes, it is better than any phonecamera.

The best camera is the one you have in your pocket when you need to take a picture.

It's doing a whole hell of a lot more and is much smaller. I'll take the tradeoff as a casual camera user.

Quality is probably limited so this would be best for a front facing camera.

I'm pretty sure they meant for the front cameras. The camera bumps are here to stay, I'm afraid.

If I look at my camera lens when I'm talking and I look at the screen while you're talking, and then you do the same, then there's no issue. Am I the only one that does this? (I know the answer. I am. At least at my company, I am.)

This feels normal if you make videos but it's pretty unnatural.

Thanks for linking to that.

> coupling, decoupling and light guiding elements to divert incident light to a concealed sensor

So, there's a camera in the dash looking up at the windshield and focusing where it expects to see a face, thereby using the windshield as a reflector? And maybe there's some additional etching and deposited films in the windshield to support the angles required?

And perhaps you can put cameras elsewhere, and similarly subtly modify the windshield or other glass to look at other things as well?

It collects through the edge, and the glass can be (apparently) flat: https://www.youtube.com/watch?v=NORPeCcIXRQ

Wow. The original article was talking about doorbells and TV broadcast applications. That looks like the quality of the original moon landing broadcast.

>nd light guiding elements to redirect incident light to a concealed sensor,

So you can't just stick on any existing window or pane of glass? That's good.

I don't think so. From the article:

> The Holocam technology "uses holographic in-coupling, light guiding and de-coupling elements to redirect the incoming light of a transparent medium to a hidden image sensor."

That suggests, at least to me, that you'll need something more than just a simple sheet of glass. There's probably some engineering required to allow light to be guided and redirected towards what sounds like a typical camera sensor.

in-coupling and de-coupling of light, anyone care to explain?

I don't actually know what that specifically means in this case.

If I were to guess, I'd assume that "de-coupling" means "splitting the spectrum into RGB." RGB, because it's easy to reproduce for human viewing, but you could do this with other spectrums as well.

I imagine "in-coupling" refers to how they get the light to a sensor.

I used to worked for a company that would use lasers to "build" mirrors inside of blocks of glass. They could tune those mirrors to reflect just red, or just blue, or green, or any combo.

I can imagine using this sort of tech to build panels of glass that can route "de-coupled" RGB to travel spectrum specific sensors.

Or, it might be possible that they're just capturing the full spectrum of light, but using microscopic etchings or surface treatments to bounce light down a layer of glass, somehow bouncing "pixels" of light down the pane of glass in a way that isn't lost while capturing other "pixels".

I say "pixels", but what I mean is "a portion of the glass." If you think of the glass as the polished ends of fiber optic strands, all bundled together, each of them routed to a specific sensor pixel - that's sort of what I mean. If you can etch your prisms/combs/gates fine enough, you can have higher "pixel" density from your glass pane.

Then again, they're the lens experts and could be doing something else entirely. I'm really just guessing.

Yes I interpret it as guiding light inside the glass such that the sensor is on the lateral aspect of the glass embedded in a glass frame perpendicular to the target image.

Sounds like they embedded a light splitter with some sort of periscope style lense inside a glass pane.

So not actually bolted oblique to the glass as GP suggested.

Edit: theres an actually good demo video showing the real state of the tech rather than mockups: https://www.youtube.com/watch?v=NORPeCcIXRQ buried below in the comments so just surfacing higher. everything else is artists lying to you.

---

> Glass surfaces can also generate energy. The microoptical layer in the window pane absorbs incident sunlight and transmits it in concentrated form to a solar cell. This combines the advantages of conventional windows – natural light and an unrestricted view – with the additional benefit of efficient energy production.

what? holy shit?

That video is pretty cool. Truly an invisible camera.

Nah, this will only be good enough for sensor-level power (like 5W from a whole window). Only useful in very limited circumstances. It's not going to replace normal solar power.

5W is nothing to sneeze at.

but like, thats enough for the camera to power itself. pretty cool no? we can deploy this tech without plugging it in.

>deploy this tech without plugging it in

you mean because it can also power a wireless transmitter or a large memory array storage? cuz a hologram that's not "plugged into something" might turn out to not be that useful.

Since you can run a PI on that level of power it seems enough?

I guess it might be not enough if the day is not sunny enough

It is, but do you really want your windows to have integrated cameras? You can't just add this on to a window because the light is transported to the edge of the glass and that is inaccessible in normal windows.

5W is enough to power an ESP32

>ESA and NASA space missions have carried this trailblazing ZEISS technology on board for many years. It is also well established in the semiconductor and medical technology sectors.

Huh. Seems like it should be fairly easy to find info on then... though some googling around makes me think they might just be referring to their more general diffraction gratings and whatnot.

Zeiss multifunctional smart glass

Product page: https://www.zeiss.com/oem-solutions/products-solutions/multi...

NASA/ESA mention: https://www.linkedin.com/posts/zeiss_automotive-augmentedrea...

This looks really cool.

Could you put a polarizer on this and have that filler out the other side of the display, so a smart window is one way visible? That could make smart glasses and headsets much better.

I doubt you'd even need to. The fact that it's not a uniform grey blob means they're already controlling how the light leaves the device. I suspect it's already directional.

First, holy crap!

Second, #3 is always a red flag for me. It is sometimes code for "we can do this in the lab but we have no idea how one would manufacture it." A similar analogy is "we've got this great idea for a program you can license but no one here knows how to actually code it up."

Third, the impact of this going mainstream would be hard to underestimate. All those people working on transparent displays like they do in sci-fi movies? Yup they could do that. A video conference system with solid eye contact (mentioned in a couple of places) sure you could do that too. A mirror that could show you wearing different clothes? Yup I could see how that would be coded.

That #3 though. That is what tempers my enthusiasm. Did I miss any announcement that they had a display at CES? (or was it just an announcement) If the former I would seriously consider flying over to Vegas to check this out.

> Second, #3 is always a red flag for me.

Does Zeiss manufacture any public facing camera currently ? All their general public facing projects are collaborations with Sony or Vivo or other makers, and I think they only manufacture lenses, so nothing with a high quality chip in it.

It's probably another story on the medical side, but this doesn't fit a highly specialized, peofessional only niche with a PC doing the processor on the side.

If it was anyone but Zeiss I'd call it a red flag too, but they're basically Old Gods of Glass.

I don't disagree. One way to know if it can live up to the hype is that it will be the lighthouse feature of the next iPhone which will both not have a notch for the front facing camera, it will do "touch anywhere" touch ID and process all of the screen operating gestures visually rather than capacitively so that you can operate you phone while wearing ski gloves if you want too.

Here is a demo video from someone who was there: https://www.youtube.com/watch?v=NORPeCcIXRQ (from another hn comment)

Ooh thanks for that! Showing monochrome only. I wonder if three layers of glass would get you color? And fixed focus (not too surprising) but that answers two questions I had initially.

As they claim this can also do display (presumably to replace the camera element with a projection element) it would be interesting to see what level of opacity they can achieve in a brightly lit environment.