The best way to use Photoshop without a mouse is to use a Wacom tablet. My Fitts parameter estimates for mouse interaction are terrible, but much better than any keystroke interface I've seen, much less command-line interfaces.

You were attacking interactive image editing by typing strings of text:

> in command line ImageMagick you say "x=100 y=200". This is not a trivial difference

But I don't think anyone really thinks that's a good idea.

I agree that direct manipulation is a great benefit, and I don't think there is any plausible alternative for designating positions and especially paths in pixel images the way incremental search provides a better alternative for designating positions in text. Vim in particular is distinguished from vi by supporting more direct-manipulation and noun-verb interaction flows, and vi differed from command-line editors similarly by supporting more immediate feedback and pseudo-direct-manipulation interaction: you would change a word by typing fscwbad ^[ rather than s/strange/bad/^M, for example. I think this is an improvement.

I'm continuously kind of appalled that we're carrying around these high-bandwidth multitouch devices all day and apparently the best way we've figured out to take advantage of this situation is one-finger scrolling of lists of pre-existing options, tap-selecting, and a simulated onscreen keyboard, plus the occasional pinch-zoom. I feel like even pictures under glass is about 10 years overdue for some new UI paradigms to take advantage of the new possibilities of the medium.

I wonder if there's a way to describe and interactively refine descriptions of areas of interest in a symbolic way—as Lisp trees if not Unix strings—maybe using existing neural networks. Certainly a speculative idea, of course.

> I wonder if there's a way to describe and interactively refine descriptions of areas of interest in a symbolic way—as Lisp trees if not Unix strings—maybe using existing neural networks. Certainly a speculative idea, of course.

I think the future is more like saying to the computer what you actually want. Like "delete the apple from the picture".

Photoshop already has a primitive form of this particular action, called Content Aware Fill, where you draw a rough selection around an object you want to delete, and it will fill a plausible background in its place. Works pretty well.

https://helpx.adobe.com/content/dam/help/en/photoshop/using/...

In a sense these are different uses of a computer for graphics: one is a medium of human visual expression, which is Photoshop’s forte, while “delete the apple” is more like delegating a functional task to an independent contractor. I don't think one is likely to entirely replace the other, just as Photoshop hasn't replaced netpbm, photography hasn't replaced painting (even if we do more painting in PS than with paint nowadays), and the city bus hasn't replaced dancing or skateboarding, even though all of them are ways to move your body.

There are certainly some uses of PS that things like caption-to-image GANs will replace. Analogously, last night I spent quite a while with the expressive, high-bandwidth, direct-manipulation interface of a box cutter cutting cardboard, for something I'd much rather be doing with a laser cutter cutting shapes generated from a constrained-optimization algorithm. At least this time I managed to only cut the cardboard.

Texture synthesis is a really interesting tool for both purposes. There are a lot of amazing papers in recent SIGGRAPHs with new algorithms for this kind of thing.