News for hackers. Stuff that matters.
Re:Regex2fat: Turn your favorite regex into FAT32 by rwmj on Wednesday April 15, 2020 @02:27PM (#22881219)
If you think this is crazy, check out the VVFAT driver in qemu[1]. At first sight it seems simple enough - turn a host directory into a virtual FAT filesystem which is presented to the guest.

The clever/insane thing is it supports writes. It is able to "reverse" those block level operations from the guest to modify the source filesystem on the host.

It was written by the ever exceptional Fabrice Bellard. EDIT: No it wasn't, it was written by Johannes Schindelin, thanks for the clarification in replies.

[1] https://github.com/qemu/qemu/blob/master/block/vvfat.c

Re:Regex2fat: Turn your favorite regex into FAT32 by lifthrasiir on Wednesday April 15, 2020 @03:23PM (#22881921)
> It was written by the ever exceptional Fabrice Bellard.

I think it is rather by Johannes Schindelin [1].

[1] https://github.com/qemu/qemu/commit/de167e416fa3d6e4bbdcac90...

Re:Regex2fat: Turn your favorite regex into FAT32 by yellowapple on Wednesday April 15, 2020 @04:31PM (#22882670)
On the other hand, I don't think we've ever seen Fabrice Bellard and Johannes Schindelin in the same room together.
Re:Regex2fat: Turn your favorite regex into FAT32 by lifthrasiir on Wednesday April 15, 2020 @05:05PM (#22883072)
That is... compelling, I admit, as Bellard was known for his use of various pseudonyms (e.g. Gérard Lantau for FFmpeg). But I guess Schindelin now works for Microsoft while Bellard still works for Amarisoft [1], right?

[1] https://www.amarisoft.com/about-us/

Re:Regex2fat: Turn your favorite regex into FAT32 by giantrobot on Wednesday April 15, 2020 @05:31PM (#22883311)
But if you rearrange, remove, and then add some letters to Amerisoft it spells MicroSoft. The implication is clear.
Re:Regex2fat: Turn your favorite regex into FAT32 by zaat on Wednesday April 15, 2020 @07:42PM (#22884491)
And now you can even write the regex for it into FAT32
Re:Regex2fat: Turn your favorite regex into FAT32 by unwind on Thursday April 16, 2020 @03:15AM (#22886942)
Huh? Where do you get the 'c'? Typo?
Re:Regex2fat: Turn your favorite regex into FAT32 by himlion on Thursday April 16, 2020 @03:29AM (#22887021)
>and then add some letters
Re:Regex2fat: Turn your favorite regex into FAT32 by userbinator on Wednesday April 15, 2020 @07:49PM (#22884548)
That seems like the perfect solution to the problem of newer Android devices losing USB mass storage support because the internal data filesystem is no longer FAT32 and they combined the partitions together into a Linux-format one. Unfortunately a quick Google suggests no one has tried to take that file/concept and implement it so the Android device could present a virtual USB drive using that with a directory on the "host", but I guess that just makes it an idea for anyone else who has some time and would like an interesting challenge...
Re:Regex2fat: Turn your favorite regex into FAT32 by netsharc on Wednesday April 15, 2020 @08:56PM (#22885007)
I can imagine other interesting use cases. fat32 over the network, for example: so you can e.g. plug in a USB device into some legacy hardware, and when it does a directory read, the device lies to the host about what's inside, only retrieving the bytes over the air during a file read operation.

I think the hard part is that if the "disk" is mounted, the FAT would be in memory on Windows, so it's very hard to put in a file from "outside", because Windows will probably never re-read the FAT from the disk and will never see that file.

Re:Regex2fat: Turn your favorite regex into FAT32 by tomc1985 on Wednesday April 15, 2020 @09:20PM (#22885137)
Why can't they keep a fat32 driver around and mount it like they always have?
Re:Regex2fat: Turn your favorite regex into FAT32 by userbinator on Wednesday April 15, 2020 @09:46PM (#22885261)
The data partition used to be FAT32 and plugging the Android into USB with the mass storage option would unmount the partition and then the Android emulates a USB mass storage device to expose the block-level contents of the partition to the host computer. Note that this couldn't be done with the system partition, since the Android system itself is running on it; and even if it was unmountable, the filesystem would be a Linux one like ext3 or ext4, making it unreadable to a Windows host. In the newer versions, they combined the two partitions into one with a Linux filesystem, so there would no longer be a separate data partition to unmount from the Android and mount on the host.

The idea I'm proposing is to turn VVFAT or an equivalent idea into a kernel module that treats the data directory --- or indeed any --- directory on the Android, including the system one, as the "host" since it has a filesystem-application interface on one side that just uses the normal filesystem calls, and expose the virtual FAT filesystem over USB mass storage on the other side.

Re:Regex2fat: Turn your favorite regex into FAT32 by netsharc on Thursday April 16, 2020 @09:15AM (#22888991)
Without knowing how QEMU does it, I'm guessing writes onto the emulated fat32 partition inside QEMU get caught by their software, and QEMU talks to Windows and says "Hello Windows, I would like to store this data as a file in this directory.". The directory can probably be on an NTFS disk or even Samba mount?

If a phone is "mounted" as FAT32, the writes from Windows should be intercepted by the driver and it would create Linux filesystem calls to create/write files. If an Android app decides to write onto that partition, there needs to be a program running on the Windows side to tell Windows' filesystem driver "Hey I'm going to create this file". Otherwise Windows will never see that file.

(I'm just guessing here, maybe there is a function to re-read the FAT?)

Re:Regex2fat: Turn your favorite regex into FAT32 by tomc1985 on Wednesday April 15, 2020 @09:54PM (#22885305)
Oh, I was thinking about this the wrong way -- I had read it as Android's ability to mount USB OTG mass storage devices plugged into the phone.

I agree with you, if the internal filesystem can't be mounted as a filesystem on most PCs then there should be some sort of virtual/emulation layer.

Re:Regex2fat: Turn your favorite regex into FAT32 by kazinator on Wednesday April 15, 2020 @11:21PM (#22885713)
The insane thing is supporting writes on the host side (are they?) which change the contents of the VFAT right under the guest OS. Writes in the other direction are a lot simpler because the guest is in control of the VFAT state.

Come on, any sort of caching scheme will wreak havoc with underhanded updates to the image.

Re:Regex2fat: Turn your favorite regex into FAT32 by LeoPanthera on Thursday April 16, 2020 @01:40AM (#22886464)
I've used this feature quite a lot and it's important to understand that it is crazy unstable. The fact that it works at all is impressive, but it does explode quite a lot.
Re:Regex2fat: Turn your favorite regex into FAT32 by rwmj on Thursday April 16, 2020 @07:36AM (#22888364)
We discussed VVFAT at the KVM Forum (2018 IIRC), and as a response I wrote: http://libguestfs.org/nbdkit-floppy-plugin.1.html (It does not support all the crazy write/host filesystem interaction because I'm nowhere near as smart as Johannes Schindelin.)
Re:Regex2fat: Turn your favorite regex into FAT32 by beholder1 on Wednesday April 15, 2020 @03:00PM (#22881656)
I see another name as author and no trace of Fabrice in history. Can you elaborate please?
Re:Regex2fat: Turn your favorite regex into FAT32 by loeg on Wednesday April 15, 2020 @03:32PM (#22882005)
QEMU itself came from Bellard originally, but I believe you are correct that the RO and RW vvfat support came from Johannes Schindelin:

https://github.com/qemu/qemu/commit/de167e416fa3d6e4bbdcac90...

https://github.com/qemu/qemu/commit/a046433a161a1f554be55df8...