News for hackers. Stuff that matters.
Re:Act-1: Transformer for Actions by mrits on Wednesday September 14, 2022 @05:37PM (#32843726)
Natural language interfaces are very limited and certainly not the next generation of computing. Granularity of functionality and composable input will always be more efficient as long as the original source is a human. I think the natural language part of your product is the lease interesting and certainly not the most impressive.
Re:Act-1: Transformer for Actions by davepeck on Wednesday September 14, 2022 @06:14PM (#32844138)
> Natural language interfaces are [...] certainly not the next generation of computing.

I'm game to take the other side of that wager.

My instinct from the last 5 years of advances in ML language research is that we're right at the cusp of having radically better natural language interfaces.

I chose a half-decade horizon because "Attention Is All You Need", the paper that introduced the transformer model, was published in 2017. Two of its co-authors are co-founders of Adept.

Re:Act-1: Transformer for Actions by jacobr1 on Thursday September 15, 2022 @12:18PM (#32854274)
Agreed. Multiple paradigms and "right tool for the job" and all that. Technology is often additive rather than successive.

At this point ... think of google. How many tasks start with some vague statement of intent in a google search and then refinement by reviewing pages or better search queries: "How do I ..." Already many (most) internet users are using natural language to interface with the internet. Average don't go straight to wikipedia - they type a question and get a response where wikipedia is the first first or the knowledge graph except is sufficient. Or "plane ticket to $location" and get linked to airline sites (or the search engines interface).

We are much further along this path than many might suspect.

Re:Act-1: Transformer for Actions by aleksiy123 on Wednesday September 14, 2022 @08:34PM (#32845452)
Most of human interactions with each other are through natural language and it works very well for us. Most of our goals start with this kind of communication. It feels like to me its the natural way for us to interact.

We just didn't have the tech that was able to take our high level instructions and carry them out for us like a human can. I think that is the long term goal of human computer interaction. This product seems like a significant step towards that.

Re:Act-1: Transformer for Actions by rolisz on Thursday September 15, 2022 @02:47AM (#32848008)
> Most of human interactions with each other are through natural language and it works very well for us.

Non-verbal (body language) communication is a bigger part of our interaction than verbal communication (https://www.entrepreneur.com/leadership/you-dont-say-body-la...)

Re:Act-1: Transformer for Actions by aleksiy123 on Thursday September 15, 2022 @03:13AM (#32848204)
We are talking about natural language not just verbal. I don't see anyone writing down body language to communicate anything...
Re:Act-1: Transformer for Actions by rolisz on Friday September 16, 2022 @05:29AM (#32864012)
You said most human interaction - that includes more than natural language.
Re:Act-1: Transformer for Actions by mrits on Wednesday September 14, 2022 @09:35PM (#32845934)
Do you have any research backing up that human interaction "works well"? As far as I can tell it is absolutely horrific. So much we literally have games called "telephone" mocking how horrible it is.
Re:Act-1: Transformer for Actions by dreadlordbone on Wednesday September 14, 2022 @10:13PM (#32846193)
Gonna start with the last 10,000 years of human history.
Re:Act-1: Transformer for Actions by aleksiy123 on Thursday September 15, 2022 @03:15AM (#32848223)
Works well in comparison to anything else we know. Would love to hear any other methods you might have for communicating complex ideas.
Re:Act-1: Transformer for Actions by tluyben2 on Thursday September 15, 2022 @05:05AM (#32849129)
It works well-ish in general, but it is rife with ambivalence, noise, lack of recall and bias. I find it horribly inefficient for my work on it’s own and as someone who grew up with the teachings of older generation programmers (most extreme, Dijkstra), I notice that most attempts to move everything to verbal communications is laziness; people don’t want to write, draw, spec etc if they can do prevent it. I even think ‘agile’ is a lazy way to say you are not going to spec out anything but just have a chat and hope for the best.
Re:Act-1: Transformer for Actions by jacobr1 on Thursday September 15, 2022 @12:29PM (#32854450)
I think the key aspect of this is how you capture iterative refinement. One can capture a high level intent ("Build a webpage with widgets to accomplish foo", or "Buy me tickets to the playoff game") But upon inspection there are fractally increasing levels of details that need further specification. For the webpage - say authorization requirements or visual design details just as an example. Or for the tickets ... which seats? what cost is acceptable? if the next best seat is just slightly more than the state budget is the tradeoff worth it? What about the special bundle that includes some sort of promotional item, worth it? And if it is worth it this time, does that apply next time?

For code-technical decisions, design documents, or the code itself provide a good record for the ever increasing details and scope, but we certainly haven't found the best way to capture everything. Perennial debates about code documentation, or what should go in PRs, or literate-style programing and more show we haven't figured it out. But at the same time, I definitely find myself having non-productive slack conversations, or PR threads or whatever, when my colleague (or myself) prompts: "hangout?" or similar to have call and talk through the issue. Something about the real-time conversation is able to cut-through written mis-communication very well - which then can be captured back in text for the benefits of other colleagues.

Re:Act-1: Transformer for Actions by i_am_toaster on Thursday September 15, 2022 @09:20AM (#32851494)
I agree on this front in regards to language being a poor interface for technical tasks. NLP has some extremely useful applications, but language is inherently lossy and abstract. We use the same words to describe different things in different contexts, and it’s common to incorrectly use words in contexts we are unfamiliar with. I don’t think this is a solvable problem in a general domain. Given a more specific domain, I think it can become more accurate, but domain level understanding from the user is required to interface with the model and the scope of its utility will be dependent on how it is applied. Personally I don’t think this 1-1 interface to do a thing is an appropriate design for those situations, although perhaps IDEs will begin to use some NLP to learn when to recommend certain types of refactoring tools, sort of like linters for design choices.
Re:Act-1: Transformer for Actions by aleksiy123 on Thursday September 15, 2022 @01:06PM (#32855119)
Btw I want to clarify that I don't just mean verbal. Writing is natural language as well.

However, on your point I think natural language is the gateway to all those things. If you have a spec you will need to describe it in natural language first then use it.

Visual though is I think another way we should be able to communicate with a computer in the future and it seems like we are heading in that direction.

I guess how I imagine all this working is you provide all this high level instruction to the computer through writing/speaking as well as any diagrams and it figures it all out as well as queries you for any further clarification it needs. Just like interacting with your colleagues in day to day.

Re:Act-1: Transformer for Actions by amilios on Wednesday September 14, 2022 @05:39PM (#32843757)
On the flipside, natural language interfaces have the potential to be extremely easy to use for anyone, including non-experts. Anyone can type a message to the computer, without having to learn the specifics of an interface's custom controls. There are different types of efficiency. I'm assuming you're referring to something like 'operational efficiency', while NLI wins on 'ease of adoption' per se.
Re:Act-1: Transformer for Actions by angrais on Wednesday September 14, 2022 @05:49PM (#32843887)
Consider search engines and searching for content more generally: are you certain that natural language is most often used?

When I search for content, I use key terms to produce refined and better results. If you don't use such terms then what you're looking for may be difficult to find.

Re:Act-1: Transformer for Actions by dwrodri on Wednesday September 14, 2022 @06:11PM (#32844120)
So, I actually thought the opposite: the growth in userbase that most big tech companies have seen over the 2010s probably meant the "death" of people creating queries á la AltaVista. But it turns out, I think I'm wrong, Google's own research says the average query is between 2-3 words in size, but apparently it has in fact been going up over the years.

Here's my source: https://dl.acm.org/doi/pdf/10.1145/1753326.1753333?casa_toke...

Re:Act-1: Transformer for Actions by averylamp on Wednesday September 14, 2022 @11:43PM (#32846719)
I think this is true for everyone. Without ever knowing how google search works, somehow over time you figure out how to do prompt engineering and figure out that the results you want are more highly correlated with certain phrasing.

Wouldn’t something similar apply here though, where after using it for some time you get an inherent understanding of what it works well with and does not?

There’s always some sort of mix of things to learn and make available to the end user though, so that they understand how to use the tool to be successful.

Disclaimer: I also work at Adept

Re:Act-1: Transformer for Actions by jacobr1 on Thursday September 15, 2022 @12:38PM (#32854600)
But don't we do this with other humans too? The language one uses with close family, or friends, or colleagues all shifts to achieve the desired effect. Code-switching is pretty common. I think people underestimate how much communication is a feedback-driven process.
Re:Act-1: Transformer for Actions by mrits on Wednesday September 14, 2022 @09:38PM (#32845953)
Nobody can type a message to a computer without knowing how the interface works. I think you are confused on this topic. Natural language is just a translation layer. If you wanted to you could put AI on top of a UI that works on intent as well.
Re:Act-1: Transformer for Actions by pyinstallwoes on Thursday September 15, 2022 @12:12AM (#32846944)
Language is the ultimate interface.

The highest standard of UX is the genie that does as one wishes.

The best interface is no interface.

Re:Act-1: Transformer for Actions by woojoo666 on Thursday September 15, 2022 @02:05AM (#32847728)
Programming languages are languages as well though. Every language has tradeoffs, and there are multiple natural languages (english, chinese, spanish, etc). I'd be surprised if english was the best tool for every task
Re:Act-1: Transformer for Actions by jacobr1 on Thursday September 15, 2022 @12:35PM (#32854559)
> I'd be surprised if english was the best tool for every task

Certainly not - language models (including multi-lingual models) map words to some kind of (300ish?) multi-dimensional concept space. I wonder if we can translate that back to some sort of symbolic representation that is much more precise than human language. Some kind of IR where we could compile human into representing but also program against. I suspect the early prolog people were attempting something like this but were very wrong about framing reasoning as logical deduction rather than a stochastic process.

Re:Act-1: Transformer for Actions by jcims on Wednesday September 14, 2022 @05:45PM (#32843831)
You might not be the target audience though, eh?
Re:Act-1: Transformer for Actions by i_am_toaster on Thursday September 15, 2022 @12:19AM (#32846992)
I’d argue this product has no target audience.
Re:Act-1: Transformer for Actions by tveita on Thursday September 15, 2022 @07:42PM (#32859978)
Based on the example videos I think it could be an amazing accessibility aid.