Your phone didn't recognize the object, you relied on a cloud service to do that for you.
When you use such services for free, you'll end up with all kinds of legal compromises that don't necessarily benefit you as an individual in the long run. Your personal convenience is subservient to other goals that don't necessarily align with public interests at large.
You could argue that the infrastructure will keep miniaturizing and one day you might not need those services. But that's not how things are currently evolving. Moreover, it will always take massive amounts of data to re-create the same models that are able to recognize patterns that are relevant to your specific context when you have a query.
At the end of the day, it's about what trade offs you are willing to accept. Cloud services based on machine learning do give you a good amount of convenience, but then you have to be willing to accept the hidden costs as well.
In iOS 13 (which I am running), it is indeed my phone that does this.
https://www.apple.com/ios/photos/pdf/Photos_Tech_Brief_Sept_... (page 4) says:
> Photos is enabled by powerful machine learning to deliver unique features like Memories, Search Suggestions, and For You. Photos analyzes every photo in a user’s photo library using on-device machine learning that delivers a personalized experience for each user. And this analysis is designed from the ground up with privacy in mind, with all of the processing done on device—and the results of this analysis are not shared with anyone, not even Apple.
> Photos uses on-device processing to analyze each photo and video in a number of ways, including:
> • Scene classification
> Identifies objects, like an airplane or a bike, and scenes, like a cityscape or a zoo, that visually appear in a photo, using a multilabel network with over a thousand classes.
> [...]
The training itself as you point out, happens not on the local device but on the servers that Apple own. So that part you are right about, but that is to be expected. Otherwise, manual tagging by each individual user (as well as significantly more processing power) would be required after all in order to train the models in the first place.
I disagree with the premise that we should spend time manually organizing and tagging our pictures all that much.
The metadata that the phone adds to pictures – time stamp and GPS coordinates – is already sufficient in a lot of cases for finding pictures that I look for.
And where that metadata is insufficient, improved search powered by machine learning will come to the rescue. And not just tomorrow but even today.
Just the other day, a few weeks back, I was standing in the kitchen that I share with two other people and I wondered to myself whether the kitchen knife in the dishwasher was mine (I’d bought a new one a few days prior but couldn’t remember what it looked like). I take a lot of picture of random stuff and mundane things, most of which I never bother to organize or tag or anything. I pull up my phone, search my photo library for “knife” and lo and behold, I did take a picture of it when I bought it and my phone has recognized the object in the photo to be a knife so it was able to find it for me.
Important files and photos I do organize. Specifically for three reasons:
1. Ease of access.
2. Grouping related data together.
3. Tying photos and other data to abstract concepts like ideas for possible games or products.
So I am not advocating no organization or tagging at all.
But I think a lot of people are unaware or at least haven’t really incorporated the distinction between information that is already present in the data, and information that must be manually added. So they spend a lot of time manually creating folder structures that encode information which could already be automatically derived from the data itself.
As for messages in closed apps, I just screenshot them. And I am relying on OCR technology to be or become good enough to refind those messages in the future. That way, if the platform itself is gone by then or the messages are not on the platform itself or hard to find on the platform itself for whatever reason.
So far I haven’t even needed to use OCR. Because if I look for a message I often have other memories of where I was, when it was or something else that happened around that time. So I just jump back in time in my photo stream and either find the screenshot right away or I find pictures near-by in time and spend a tiny amount of time looking forwards and/or backwards in time and I find the screenshot.
I do wish though, that iOS would automatically tag screenshots with the name of the app that the screenshot was taken in. And I think it would be cool if the screenshots were stored as SVG with pure text and vector shapes plus embedded bitmaps, so that the whole potentially needing robust OCR in the future thing could be side-stepped.