“GUIs are pretty but slow. Ok, they’re faster but just glued on to CLI programs. Ok, they’re native but less information dense. Ok, that’s a good thing, but…”
“The internet is nice but you still have to go to the store/bank. Ok, you can shop/bank online but you still have to call airlines. Ok you can buy airline tickets but you still have to go to the library to research. Ok, you can research but you still have to drive in to work. Ok…”
Maybe it’s a half empty / half full thing, but there is some portion of the populace that can’t acknowledge the promise of anything until it is perfect and comprehensive.
And, yes, there’s another portion that can’t acknowledge gaps or flaws of the promise is sufficiently huge. But it’s the former that grinds my gears.
Well written GUI apps will always have good support for keyboard-only use. This is doubly true for apps that care deeply about accessibility, since some people have disabilities that make it hard for them to use a mouse or trackpad.
On the other hand, TUIs will never support non-Latin scripts properly, or have good support for screen readers, or be easily discoverable without a manual, or have uniform automation support using native automation APIs. In my view, CLIs are strictly inferior to GUIs unless you’re putting together data pipelines.
> Maybe it’s a half empty / half full thing, but there is some portion of the populace that can’t acknowledge the promise of anything until it is perfect and comprehensive.
Probably because paradigm shifts or new ways of doing things come with a lot of trade-offs that can be seen only after the change has been absorbed more. Early adopters are usually blind to these issues (or decide to be blind at them), conservatives don't look at the improvements but think that there might be something not that good in the future by projecting past experiences.
More self-driving cars on the road -> more driving data for that company -> better self driving cars
But when we have that, it’s hard to believe that self driving won’t work.
The question is whether these are somehow inherent to the LLM approach or whether scaling up and continued improvements can eventually get rid of them.
They are the main barrier at this point between a very useful tool, but one that still needs to have all its output carefully checked by humans when it comes to anything important, and a true autonomous agent that can be given full tasks to do on its own.
I'm not at all trying to downplay the power or significance of LLMs, btw, in case that's why I'm getting downvoted... I'm using copilot/GPT4 every day and they are massive productivity boosters. But currently I see them as tools for producing rough drafts that need to be revised and checked over. If they can't solve hallucinations, LLMs will stay in this lane, which is still incredible, amazing, and useful, but won't necessarily get us to the AI endgame that the hype is predicting.
Also, I have no illusions about how language models work. I notice that the sloppier the code I'm editing, the worse suggestions (and dumber comments) it suggests. If I start writing a comment and copilot immediately suggests "TODO: delete this", that's a better indicator of poor code health than any metric I know about. It's not a magic wand, but it can be extremely useful if you use it sensibly.
Then again, you can just decide to forget how it works and ask it questions directly in comments:
// Hey GPT3, write me a poem
// ANSWER:
I (mostly) kid but some seniors I know struggle at those things.
Not quite, but a bit of plumbing can get you closer. Not human using a computer close, but interestingly closer nonetheless.
I've been trying to accomplish something akin to this by having a program monitor and alter another program within a virtual machine, using GPT-generated solutions to error traces to correct bugs in the sand boxed program.
It watches the program to see when an error occurs, feeds the error to GPT with pertinent code, then tries to splice in the solution.
It kind of works. I don't think we're going to see human-levels of success from this in the immediate future, but I was able to write a simple event-based system which alters a program to resolve simple bugs. It even does it on a different git branch, and there is some stubbed out code and prompts for generating tests. In my manual testing, this actually worked too. If the tests passed I was going to have it push the change set and create a PR explaining the changes, tests, etc.
I doubt I'll continue now that Copilot is doing this already. My point though is that with the right configuration, the right data and prompts, and a system orchestrating the start/stop/test patterns based on the state of the sandboxed program, you can begin to achieve something akin to an inexperienced person solving bugs.
Sometimes it does a terrible job and other times it kind of falls over itself. But we're already leaps and bounds ahead of previous systems, and I just cobbled this together with what's possible via OpenAI's API.
The crazy part is that there are so many possible layers. Like say we get our initial solution and we verify that it works. Well, now we can have a system which optimizes the implementation. Like a PR buddy that observes the implementation and determines: should this test be appended to an existing suite of tests? Can the test case simply be added to an existing table-driven test? How can we streamline this patch to avoid an endless stream of additional files and tests to maintain? I think that's actually tractable already. While the success rate won't be 100% today, it'll clearly only improve.