Re:Philosophy of the GNU Project (2018)
by DonHopkins on Thursday December 17, 2020 @02:47PM
(#25459536)
Have you seen Morgan Dixon's "Prefab", and his ideas about pixel-based reverse engineering and modifiable user interfaces? One of his papers asks, "What if Every GUI Were Open-Source?"
Video: Prefab: What if We Could Modify Any Interface? Target aware pointing techniques, bubble cursor, sticky icons, adding advanced behaviors to existing interfaces, independent of the tools used to implement those interfaces, platform agnostic enhancements, same Prefab code works on Windows and Mac, and across remote desktops, widget state awareness, widget transition tracking, side views, parameter preview spectrums for multi-parameter space exploration, prefab implements parameter spectrum preview interfaces for both unmodified Gimp and Photoshop: http://www.youtube.com/watch?v=lju6IIteg9Q
Video: Content and Hierarchy in Prefab: What if anybody could modify any interface? Reverse engineering guis from their pixels, addresses hierarchy and content, identifying hierarchical tree structure, recognizing text, stencil based tutorials, adaptive gui visualization, ephemeral adaptation technique for arbitrary desktop interfaces, dynamic interface language translation, UI customization, re-rendering widgets, Skype favorite widgets tab: http://www.youtube.com/watch?v=w4S5ZtnaUKE
PDF: Prefab: What if Every GUI Were Open-Source? Morgan Dixon and James Fogarty. (2010). Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI '10. ACM, New York, NY, 851-854. https://web.archive.org/web/20141024012013/http://homes.cs.w...
>ABSTRACT:
Current methods for implementing graphical user interfaces
create fundamental challenges for HCI research and
practice. Researchers are often unable to demonstrate or
evaluate new techniques beyond small toy applications, and
practitioners are often unable to adopt methods from the
literature in new and existing applications. This position
statement examines a vision in which anybody can modify
any GUI of any application, similar to a scenario where
every GUI of every application is open-source. We are
currently working to enable this vision through our
development of Prefab, using pixel-based interpretation of
GUIs to enable modification of those GUIs without any
cooperation from the underlying application. We see
participation in the FLOSS HCI workshop as valuable in at
least two regards. First, fully realizing this vision will likely
require a community-based approach, so we are interested
in Prefab as a platform for collaboration between HCI
researchers and the FLOSS community. Second, enabling
arbitrary modification of any GUI would seem to blur many
current distinctions between open and closed applications,
introducing new research questions and further magnifying
the importance of the workshop’s focus.
Video: Prefab: What if We Could Modify Any Interface? Target aware pointing techniques, bubble cursor, sticky icons, adding advanced behaviors to existing interfaces, independent of the tools used to implement those interfaces, platform agnostic enhancements, same Prefab code works on Windows and Mac, and across remote desktops, widget state awareness, widget transition tracking, side views, parameter preview spectrums for multi-parameter space exploration, prefab implements parameter spectrum preview interfaces for both unmodified Gimp and Photoshop: http://www.youtube.com/watch?v=lju6IIteg9Q
Video: Content and Hierarchy in Prefab: What if anybody could modify any interface? Reverse engineering guis from their pixels, addresses hierarchy and content, identifying hierarchical tree structure, recognizing text, stencil based tutorials, adaptive gui visualization, ephemeral adaptation technique for arbitrary desktop interfaces, dynamic interface language translation, UI customization, re-rendering widgets, Skype favorite widgets tab: http://www.youtube.com/watch?v=w4S5ZtnaUKE
PDF: Prefab: What if Every GUI Were Open-Source? Morgan Dixon and James Fogarty. (2010). Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI '10. ACM, New York, NY, 851-854. https://web.archive.org/web/20141024012013/http://homes.cs.w...
>ABSTRACT: Current methods for implementing graphical user interfaces create fundamental challenges for HCI research and practice. Researchers are often unable to demonstrate or evaluate new techniques beyond small toy applications, and practitioners are often unable to adopt methods from the literature in new and existing applications. This position statement examines a vision in which anybody can modify any GUI of any application, similar to a scenario where every GUI of every application is open-source. We are currently working to enable this vision through our development of Prefab, using pixel-based interpretation of GUIs to enable modification of those GUIs without any cooperation from the underlying application. We see participation in the FLOSS HCI workshop as valuable in at least two regards. First, fully realizing this vision will likely require a community-based approach, so we are interested in Prefab as a platform for collaboration between HCI researchers and the FLOSS community. Second, enabling arbitrary modification of any GUI would seem to blur many current distinctions between open and closed applications, introducing new research questions and further magnifying the importance of the workshop’s focus.
I wrote more about it here:
https://news.ycombinator.com/item?id=11520967