News for hackers. Stuff that matters.
Re:Demo of =GPT3() as a Spreadsheet Feature by cstross on Wednesday November 02, 2022 @02:31PM (#33440741)
Now wait for =deep_dream() or maybe =stable_diffusion() as a graph-generating function! (Graphs plotted with this function will of course zoom in infinitely but the further you go the more eyes and shiba dogs you'll notice in the corners ...)
Re:Demo of =GPT3() as a Spreadsheet Feature by bee_rider on Wednesday November 02, 2022 @03:31PM (#33441654)
Plots that devolve into madness as if you zoom in too close? Finally the programmers get to experience electronics class.
Re:Demo of =GPT3() as a Spreadsheet Feature by dj_mc_merlin on Wednesday November 02, 2022 @07:25PM (#33444267)
Sadly current models are bad at plotting graphs with any kind of accuracy. We're still quite far off from getting them to do a properly labeled, colored pie chart with a legend.
Re:Demo of =GPT3() as a Spreadsheet Feature by dplavery92 on Wednesday November 02, 2022 @08:33PM (#33445008)
For what it's worth, I'm also very bad at plotting graphs with any kind of accuracy, which is why I use plotting software instead of doing it by hand.

I get the feeling that my visual system and the language I use are respectively pretty bad at processing and conveying precise information from a plot, (beyond simple descriptors like "A is larger than B" or "f(x) has a maximum"). I guess I would find it mildly surprising if any Vision-Language model were able to perform those tasks very well, because the representations in question seem pretty poorly suited.

I get that popular diffusion models for image generation are doing a bad job composing concepts in a scene and keeping relationships constant over the image--even if Stable Diffusion could write in human script, it's a bad bet that the contents of a legend would match a pie chart that it drew. But other Vision-Language models, designed for image captioning or visual question answering, rather than generating diverse, stylistic images, are pretty good at that compositional information (up to, again, the "simple descriptions" level of granularity I mentioned before.)