Oversharpening is a curse in the industry to be sure, unfortunately to most people an oversharpened image looks "better".
It seems that the software is doing a negative film photographer's trick of "expose for the shadows and the highlights will take care of themself" but rather it's "expose for the shadows and the HDR will take care of the highlights" - ending up in this overprocessed looking fake sky effect.
It's like there can't be any ambiguity anymore in a photo due to the exposure, the shadows and the highlights can't clip, we have to see everything. That seems a big change in mindset to me.
A painting of a room with a window would typically show both the room and the scene from the window "perfectly exposed", like the iPhone is doing with your skies.
This looks very strange to someone used to images from more traditional cameras, and the artefacts like halos stand out.
The iPhone offers strong post processing tools for photos and video that make it easier and more accessible.
I’d love a smartphone with an Arri Prime lens equivalent, but it would never fit in my pocket
As an aside, I’m a professional photographer and my greatest wish would be that people shop around for a photographer on a desktop or at least a tablet. Incredibly tiny effective image sizes can hide a lot.
I maintain that the 6S remains all that the majority of users need. Anything added after that was just to sell phones. And I’m a big Apple fan.
https://www.macstories.net/stories/iphone-11-pro-in-rome/
I have the SE 2016 (same camera as the 6s) and improved low light photos is the main feature I'm looking forward to when I replace it.
I'd say it's even worse with video. My phone can make 4K videos, but watching them on a 4K TV is painful. They'd probably look better if you would downscale them to 1080p and then re-upscale them to 4K. They'll be a bit more blurry, yes, but they won't have this artificial flat look everywhere.
Memorizing high-frequency patterns and then reproducing them is one of the very few tasks that AI is actually good at. As such, one can use the raw HDR exposure data from a phone and the noisy raw image data from the sensor and produce naturally-looking high-res images from it. The way that it works is that the AI has memorized the small-scale details from millions of good photos taken with large lenses and proper equipment, and that way it can replace the noisy phone sensor data with its noise-free counterpart.
However, such an AI currently needs 10GB of GPU RAM and it runs for 2 seconds on my 1080 TI for a 4K image. So we'll need another 10x in mobile GPU performance before it becomes feasible to integrate such technology into camera apps. And then of course another 20x for live video processing.
Also remember that the majority of academic work is done based on benchmarks that don't include inference time or hardware requirements. As these things start becoming useful without these restrictions industry will start tweaking the cost functions that are being minimised to make it useful in the real world too.
Which process is this? Got a github link? Are you talking about AI upressing? I haven't found it improves noise
But no, this is not AI upscaling.
Instead, it is representing the input image in a feature space trained to store natural images and then reconstructs the output from that intermediary representation. If chosen well, the intermediary will store details relevant for high quality images, but discard those specific to low quality images, like noise.
It's called a convolutional autoencoder with bottleneck.
But I believe for most users, it needs to be close to instant on the device. Nobody wants to wait 5 seconds to see if their selfie turned out OK.
I agree some HDR skies in your shots do look weird, but frankly that's probably just HDR. How do you find the phone's photos when you switch off the effects you don't want?
Then I read your description and if you hadn't mentioned the skies look added in I wouldn't think twice about it – now I can't stop seeing it. It really does touch on this feeling I've had towards my own photos for some time, thinking something is off but I don't have the expertise to understand what. I think you may have just nailed it for me – thanks!
But for anyone who is used to dealing with proper gear (cameras & lenses) and having the freedom to make very subjective artistic choices for things like white balance, local contrast, tone mapping, noise reduction, etc. what a phone does is always a compromise. And also it's a 1 size fits all solution optimized for mass appeal, which usually means "oooooh blue sky, pretty saturated colors, super sharp/contrasty photos", which as any postprocessing enthusiast knows are all variations of compressing dynamic range (i.e. super lossy steps).
I fall in the last category but I do appreciate the best effort my cheap Nokia 7 Plus camera app makes. I occasionally use open camera to get raw dngs for editing in Darktable but they actually do a decent job with the jpgs given the noisy original. My shiny new Fuji X-T30 produces much nicer raw files with tons of dynamic range and detail that simply is not there on a mobile phone. All of the AI stuff that happens in a phone you can also do with a decent camera but it requires lots of specialized tools and tweaking (e.g. hugin for hdr stuff).
But the best camera is the one you have with you and for most of us that's usually a smart phone.
I wonder if in a few years we’ll look back at the photos from this time and think they aged poorly.
People see phone photos all the time so they’re used to it and normally fine with it, but given an opportunity, they will notice and appreciate quality photos.
Too bad windows phone 10 was such a slow and buggy mess, and microsoft had by that point pretty effectively managed to kill the app ecosystem by rebooting it three times.
It does pull multiple full frames off the sensor in rapid exposure tho, and chooses whichever one it thinks best (blur, ML, whatever) for the initial key photo.
Picking a different frame doesn't impact visual quality on a mobile screen, but it was quite a noticeable IQ drop on a 4k/5k display.
Things might be different on newer iPhones and the latest iOS.
In a way the iPhone and recent HDR techniques on smartphones improve on this as it's now quite unusual for a phone to overexpose highlights. The tone mapping could be improved to lessen that 'fake HDR' effect you can see in your examples, but that doesn't seem like it would be too difficult.
Although this is really quite recent. It used to almost be a hallmark of my iPhone that, unless I took care, things like clouds would be blown out. I was somewhat prepared for this as I mostly shot slides pre-digital for a long time after I didn't have easy access to a good darkroom any longer. And slides don't handle over-exposure well at all.
So I don't have an iPhone, but I do currently have in my home office two pictures taken with my OnePlus 5T, both printed on A3 size paper and framed, hanging on my walls. They look absolutely perfectly sharp and I'm constantly impressed that something so good came out of a smartphone.
Here's one of the pictures itself: https://drive.google.com/open?id=1iVenWHP4TesmhTWAY2qMDQs21u...
Here's how it looks printed: https://drive.google.com/open?id=1h6P5ZbNjQ5FD9nkzdB_TnfsA4l...
Videos shot with iPhone SE 2 is indistinguishable with iPhone 11, which by itself was class leading. So, naturally people are going to take notice of better quality videos on these platforms.
This strategy is likely going to be successful in US, what I'm doubtful about is with > $550 price tag whether iPhone SE 2 would make it a viable choice in developing markets; even though TikTok, Instagram are quite big there.
That's pretty much the age we live in, the best Youtube videos, for example, have extremely bad videography and no audio processing from a professional PoV, but they get millions of views because they're funny to most people.
I mean, should I even try to mention the POTUS? (commence the downvotes heh)
I have no idea why people's standards have regressed, but they did.
If you're dealing with HDR artifacts, it sounds like you aren't shooting RAW. Shoot RAW, and you can process it however you want.
AFAIA the default camera app doesn't allow you to shoot in RAW, it doesn't even give you the fundamental manual expsure controls, and that's part of my point - the defaults are designed to make it easy and make the photos look good on the phone, which is what 99% of users want.
I'm absolutely fine with that, BTW, but it's rather frustrating that the out of the box camera app doesn't include the ability to control the three fundamental exposure factors. Not even hidden behind an "advanced" setting or anything.
I know Apple like to push the whole "shot on an iPhone" thing, and they want to make sure the default app is giving the "best" quality. So maybe that's the reason they don't include them?
And to take your analogy further - I have a Polaroid 195, it has full manual controls. Of course, it was aimed at pro photographers who wanted those controls, and sold at a premium.
Nitpick: smart phone cameras have very wide apertures (f̶2̶.̶2̶ f1.8 in the case of the iPhone SE).
Obviously, the format has implications for DoF, but those can be either positive or negative. When I shoot 4x5, I spend most of my time fiddling around with movements trying to get everything in focus :)
The iPhone's telephoto lens has a 6mm focal length, and so tends to produce an image with more in focus than a lens that's 25x longer.
(Why do phones use a 6mm lens and not a 150mm lens? Because they don't have 150mm to spare between the lens and the sensor.)
>(Why do phones use a 6mm lens and not a 150mm lens? Because they don't have 150mm to spare between the lens and the sensor.)
That is not true for today's lens formulas. You're probably thinking of old singlet designs from centuries back. For e.g. reverse-telephoto lens designs allows you to make lens much shorter than their focal length. Its all about getting the final magnification by positioning positive and negative lens elements, and ofcource other corrective lenses appropriately in the lens formula. This is why you can use "tele-converters" on lenses to increase their magnification (and thus their effective focal length)
Most people do none of these things though. Or do them so rarely that it’s all but pointless.
A film is continuous right, so no amount of digital encoding would truly encode a strip of film. Of course there is a point after which it doesn't matter, but I am more interested in the technical look and feel of film vs. digital. A few years ago I looked at a film enlargement to A4 in a scrapbook and I have to say it had a different feeling about it. There were also funny artifacts of the kind that make superstitious people believe in invisible spheres that ehm are not invisible on film...?
If you contrast this with a CD recording, the bit rate is limited even if it's an audio CD due to the way they were recorded. Vinyl is supposed to be limited to 48 bits, but perhaps you can engrave in an analog way?
Chemical film creates distortions of its own. Colour film often pushes the reds to make skin tones, lipstick, and sunsets warmer. You can choose how much grain you want and B&W is often shot on fast high-grain film to give it a gritty textured look. Landscapes need slow micro-grain film to look as lush and detailed as possible.
And so on. The point is chemical film isn't any more real than a digital CCD.
Digital consumer displays all have colour distortions and dynamic range limitations, and so does chemical processing. Even if you apply no deliberate processing to an image, by the time it ends up on paper or a web page it usually looks quite different to the scene you'd see with your eyes.
So yes - the "feel" is different. And that's a creative choice - or it can be, for the professionals. Polaroids look very different to medium format.
Amateur phonecam users get one-size-fits-all enhancement whether they want it or not, in addition to some basic editing options. If they didn't their lifestyle shots on IG would all look lacklustre, and they wouldn't be happy about that.
That said medium format film and larger would still be hard to beat.
As far as tonality, a modern sensor should also have greater dynamic range than almost all film. They do behave differently though - highlights are harder to overexposed on film but dark areas are easier to underexpose, compared to digital.
Essentially, a film grain or a cell on a CCD sensor is like a “bin” that collects and counts photons. As you make the area of the bins smaller, there are fewer photons collected in a given image, and statistical fluctuations from sampling become more relevant. You get more “noise” in the final image.
I understand that you are saying that you see the effects even if you are not a the "molecule" level as you say, but I would think that the hard theoretic limit of photons should be at a finer granularity than high end digital cameras.
I shoot wetplate collodion sometimes and that has an effective ISO of under 2 and the detail is astounding - especially for a process that’s 150 years old.
I mean, you scroll past images on Instagram, look at your own photos on the phone you took them with, look at your images on your Instagram feed on other people's phones or on the Instagram web app, and yeah they look pretty good.
Get them on something bigger though? Look a little closer? Use decent resolution monitor? A reasonable sized print? They look bad. I'm not even talking about pixel peeping here, which is bullshit. I mean they look overprocessed. Like the machine took a course in photoshop on YouTube and didn't finish it.
The examples in the article show this, they're full of little mistakes that don't happen when you have proper hardware - larger aperture lenses on larger sensors. Proper graduated neutral density filters to prevent halos. Sensors with large enough area/photosites to give better dynamic range or better high ISO performance. Real camera movements to tilt/shift/rise/fall/swing.
I mean, most of this is boring technicality stuff that doesn't really matter in the grand scheme of "is this an interesting photo" type considerations. It's more a case of computational photography is currently trying to climb out of its uncanny valley. It'll get there eventually.
For now we're still at the point of "Eventually, anyone that values photos will carry a standalone camera." I say this as someone who shoots mostly with an iPhone, but still carries around a 4x5 camera when it matters.
Edit: to expand on this a bit I've just uploaded some images I shot with the 11 pro a couple of weeks after I bought it back in October: https://leejo.github.io/2020/04/28/off_season_ibiza/ # the caveat is that I was still figuring the phone/camera out but wanted to see "what it could do out of the box" - and yeah, the images look good on the screen with very minor lens corrections and levels in lightroom, but look overprocessed even with my attempts to pull that back: the sky in the photos looks added in, there are halos around parts of the images where HDR is trying and failing, the sharpening is too aggresive (real life doesn't have "sharp" edges), and so on. When you view these at larger sizes than seen in the blog post they look much worse.