News for hackers. Stuff that matters.
Re:Screenshots as the Universal API by carlosdp on Wednesday January 18, 2023 @08:53AM (#34426583)
I was skeptical about this concept when I first heard it, until I actually tried some of the latest image models.

Now, I’m fully bought in that this is correct.

There’s a model you can use on hugging face, for example, where you can feed it any PDF or image of a document, ask it a query (“What is the total of the invoice?”), and it just spits out the right answer.

Turns out decades of work trying to make universal data interoperability standards will most likely be replaced by screenshots and images! (Again, this is for inter-app data movement)

Re:Screenshots as the Universal API by citrin_ru on Wednesday January 18, 2023 @09:10AM (#34426737)
From an invoice in a standardized XML format (some countries require this by low) total amount can be extracted in milliseconds if not microseconds. How long it takes to do this from an image?
Re:Screenshots as the Universal API by lostmsu on Wednesday January 18, 2023 @01:53PM (#34431135)
How long would it take you to get your plumber to convert his handwriting into a standardized XML?
Re:Screenshots as the Universal API by stoned on Wednesday January 18, 2023 @08:57AM (#34426627)
Do you remember what the model was?
Re:Screenshots as the Universal API by dd36 on Wednesday January 18, 2023 @10:41AM (#34428024)