News for hackers. Stuff that matters.
Re:Folders with High File Counts by jakub_g on Thursday January 19, 2023 @09:07AM (#34440388)
As a consumer of Android devices, one thing that's super annoying is that all pictures you take with a camera are stored in a single massive folder.

When you connect to the device from your computer over USB (which is usually USB 2.0 even if USB-C, except in major 2021+ models), it will take forever to enumerate the files in that folder. Once you start copying, there's big chance it hangs, so you need to disconnect the device, and go through that painful process again.

(I know, I'm oldschool, I don't have automatic cloud backup enabled).

Re:Folders with High File Counts by totetsu on Thursday January 19, 2023 @09:31AM (#34440634)
try

adb-sync --reverse /sdcard/DCIM(?) ~/photos.go.here

Re:Folders with High File Counts by dotancohen on Thursday January 19, 2023 @12:17PM (#34442836)
adb-sync has been deprecated, use better-adb-sync instead:

https://github.com/SelfAdjointOperator/better-adb-sync

Re:Folders with High File Counts by totetsu on Thursday January 19, 2023 @12:34PM (#34443047)
I saw that but I don't think the command changes as it still says adb-sync in the readme file of that repo you just linked…
Re:Folders with High File Counts by londons_explore on Thursday January 19, 2023 @10:21AM (#34441229)
Isn't that mostly because android devices use the old school "picture transfer protocol" mode, which involves a round trip to the device for every single object in a directory, rather than being able to just retrieve a directory listing in all one request like typical filesystems.
Re:Folders with High File Counts by qa-tari on Thursday January 19, 2023 @11:51AM (#34442475)
These days it's exclusively MTP as an extension to PTP that can transfer more than pictures, but similar problems with slow directory listings still exist due to the required translation between names and object IDs. Also the protocol does not support multiplexing requests, so you can't copy files while a directory is getting iterated.
Re:Folders with High File Counts by anshorei on Thursday January 19, 2023 @11:45AM (#34442370)
I've always loved how shotwell does it: "YYYY/MM/DD/filename.ext"

The only downside has always been that merging your photo libraries when you did two different things can give some funny results. Shotwell will split the events based on the import, but if you browse the files without shotwell or reimport them later you lose that.

Re:Folders with High File Counts by sneak on Thursday January 19, 2023 @04:49PM (#34446131)
After 20 years of digital photography, this is the way:

    YYYY/YYYY-MM/YYYY-MM-DD/YYYY-MM-DD.SHASUM.IMG_NNNN.EXT
This lets you keep date metadata on individual files moved or copied out, detect bit rot, etc.
Re:Folders with High File Counts by nine_k on Friday January 20, 2023 @07:39AM (#34452291)
Interesting. Have you noticed any bit rot on your images?
Re:Folders with High File Counts by sneak on Friday January 20, 2023 @11:03AM (#34454927)
No, but it's a thing that happens. I keep mine on zfs raidz2 with weekly scrubs.
Re:Folders with High File Counts by rr888 on Thursday January 19, 2023 @12:13PM (#34442775)
iOS is terrible from windows too. I figured they really dont care as you aren't a good customer.
Re:Folders with High File Counts by MawKKe on Thursday January 19, 2023 @12:18PM (#34442852)
..and always with super helpful naming scheme like IMG_0001.JPG, IMG_0002.JPG, ... that restarts from the lowest unused number after cleaning up phone storage, ensuring you'll have naming collisions in your backup folder
Re:Folders with High File Counts by MaKey on Thursday January 19, 2023 @01:16PM (#34443581)
Which phones still do that? My Samsung Galaxy S21 uses date and time: YYYYMMDD_HHMMSS.jpg
Re:Folders with High File Counts by sidpatil on Thursday January 19, 2023 @01:35PM (#34443792)
I gave up on USB file transfers off of Android phones for that reason.

A faster option is to install an HTTP or FTP server app on the phone, then download the photos through the network.

Re:Folders with High File Counts by stevesimmons on Thursday January 19, 2023 @02:57PM (#34444821)
Or Syncthing
Re:Folders with High File Counts by nine_k on Friday January 20, 2023 @07:38AM (#34452280)
Getting the images via e.g. Syncthing running on the phone does not seem to encounter this problem. Likely not over USB though, but Wireguard or better Nebula are easy to set up on both your phone and your router / NAS / desktop.
Re:Folders with High File Counts by greatgib on Friday January 20, 2023 @09:39AM (#34453737)
Yes, this is might constant nightmare too! Around 10k photo files Android starts to be painful to use. like app file browser taking huge to show the file list or crashing, and as you said, terrible experience when trying to get files on a computer.

This is mostly because of the mtp protocol that is really shitty, supporting only a single channel at a time and having to send the whole list of all files in one time. But also because Android implementation of it is very buggy.

I was always stunned that for a dozen years, Google stupidly spend millions in reworking uselessly the UX twenty times but never solved real pmserious usability problems like this one!