News for hackers. Stuff that matters.
Re:Automating programming: AI is transforming the coding of ... by js8 on Friday July 09, 2021 @12:00PM (#27785072)
I think Copilot is as helpful to programmers as ELIZA is helpful to psychiatrists.

I don't think it really understands code, only very superficially (unlike GPT-3 which might actually genuinely understand parts of language, computer code requires more abstracted mental model). ELIZA was similar, some people also believed that it can understand what they are thinking. Turns out it was all just in our heads.

Re:Automating programming: AI is transforming the coding of ... by wongarsu on Friday July 09, 2021 @12:14PM (#27785251)
> unlike GPT-3 which might actually genuinely understand parts of language, computer code requires more abstracted mental model

Shouldn't computer code be simpler to understand and express than natural language? The amount of knowledge about the world is much smaller and mostly concerns artificial systems. And of course programming languages are laughably simple and well defined compared to the constantly evolving mess that is the English language.

Re:Automating programming: AI is transforming the coding of ... by js8 on Friday July 09, 2021 @02:13PM (#27786690)
In programming languages, it is much more common to define new words (symbols) than in natural language. The identifier (name) of the symbol (word) in such case is less informative about its nature than the actual definition, and I think this is the Achilles heel of the method.
Re:Automating programming: AI is transforming the coding of ... by lambdaba on Friday July 09, 2021 @04:42PM (#27788264)
Yes, programming languages are simple, programs are infinitely complex, they could express anything, and homoiconic languages can even define their own syntax.
Re:Automating programming: AI is transforming the coding of ... by js8 on Friday July 09, 2021 @07:19PM (#27789594)
Absolutely. These days, I am exploring a concatenative language, Factor. I feel like if there ever was a consistent syntactic pattern in this language, then it would make sense to define it as a new word, rather than have a system which would repeatedly suggest it based on partial match.

And that's what I am alluding to with my entropy comment. Factor's syntax has extremely high entropy (if we look at words as tokens), or rather, it is designed to allow extremely high entropy of code.

Ultimately, I think it comes down to preference, what the tools should be doing for us programmers? Should the program source code (language syntax) have low entropy, and the tools should exploit that when we write the code (as Copilot does), or should we strive for as high entropy of the source code as possible, and have the tools explain the context for us (for example, in Factor, it would be what the expected types on the stack are in the certain place in the code, etc.)? I am in favor of the 2nd approach, I think that is going to be more productive.

Re:Automating programming: AI is transforming the coding of ... by lambdaba on Friday July 09, 2021 @07:40PM (#27789700)
I agree, programs are and will always remain first for humans to read, which means they have to follow the shape of thought and not the other way around.
Re:Automating programming: AI is transforming the coding of ... by throwawaygh on Friday July 09, 2021 @09:23PM (#27790313)
> Shouldn't computer code be simpler to understand and express than natural language?

Yes, but the ideas that are being expressed in formal languages are much more complex and detailed than the ideas that are being expressed in natural language.

Writing some plausibly on-topic blog posts is way easier than writing even a very simple program, especially if you don't really know what you're talking about.

Re:Automating programming: AI is transforming the coding of ... by lambdaba on Friday July 09, 2021 @04:35PM (#27788202)
What would it mean to "understand" it. Imagine a program written by such a system, it's easy to imagine an immensely complex question about the system that sounds simple, for instance asking about design choices, code architecture, aesthetic choices like expressivity. Even if it could produce some code, we couldn't say it understood it if it couldn't answer those questions, and those are very difficult questions which would require human-level intelligence, and even more a human psyche.
Re:Automating programming: AI is transforming the coding of ... by qsort on Friday July 09, 2021 @12:12PM (#27785219)
I'm not even remotely worried for my job security, but I don't think the ELIZA comparison is apt.

ELIZA is clearly a toy, Copilot is something that could become integrated with the workflow of a professional developer. Maybe not copilot specifically, but if Intellisense is in our IDEs already, something like it is not a stretch.

> I don't think it really understands code

It clearly doesn't, but (a) nobody actually claimed that outside of the usual media circus, and (b) that's a very high bar. It doesn't need to understand your code in order to be useful.

Re:Automating programming: AI is transforming the coding of ... by js8 on Friday July 09, 2021 @02:09PM (#27786650)
ELIZA wasn't "clearly a toy" when it appeared. It actually confused people.

I don't deny that good suggestions are useful in coding. I am just not sure you can get these by mimicking other people's code without really understanding it.

I think the reason why code is different than natural language is that in code, the meaning of identifiers (words) is very dependent on the context of the specific codebase, and not nearly as universal as in natural language. Sure, there are words like personal names in natural language that change meaning depending on wider context, but they are not as common in the natural text.

So focusing on the names only, rather than for example understanding types of runtime objects/values, and how they change, can be actually actively harmful in trying to understand the code and making good suggestions. So I would believe that suggestions based on e.g. type inference would be more useful.

There is another aspect to this, text has inherently low entropy, it's somewhat easy to predict the next symbol. But if code has low entropy (i.e. there are universal patterns that can be applied to any codebase in a particular programming language), I would argue that this is inefficient from the programmer's point of view, because this entropy could probably be abstracted away to a more condensed representation (which could be then intelligently suggested based on semantics rather than syntax).

In other words, I think the approach assumes that the probability of the next word, based on syntax, is similar to probability of the next word, based on semantics, but I don't believe that's the case in programming languages.

Re:Automating programming: AI is transforming the coding of ... by ska on Friday July 09, 2021 @02:22PM (#27786791)
> ELIZA is clearly a toy,

That's a red herring. The "Eliza effect" has nothing to do with how toy-like ELIZA actual was, it is a lesson about how humans were very bad at evaluating its actual capability and insight. This continues to be true today, and is an effect with at least some impact on essentially any AI/ML system that people interact with.

Re:Automating programming: AI is transforming the coding of ... by feoren on Friday July 09, 2021 @02:36PM (#27786943)
Copilot is clearly a toy.
Re:Automating programming: AI is transforming the coding of ... by strken on Friday July 09, 2021 @10:36PM (#27790683)
It's closer to a search engine. It's not competing with developers, it's competing with opening up your browser and searching for "astar algorithm implementation."
Re:Automating programming: AI is transforming the coding of ... by falsaberN1 on Friday July 09, 2021 @04:18PM (#27788056)
I have played with GPT3 and similar lesser models for a while, and in my opinion it doesn't understand anything. It's good at correlation (it might figure out a pikachu is a pokemon or that The Wall is a music album) which is kinda amazing but ultimately seems to be really "useless"* and it's not deterministic.

I think the current "AI" models will never understand a thing unless they become able to update their own models and have some guiding logic to them. Sure it has the context tokens so it might give a little illusion of following along, but the randomness of it all also means it will get wrong stuff that it got right before (and vice versa) and will clearly break the illusion when prodded ("pikachu is a purple dragon", "The Wall was composed by Metallica"). It's unfortunate.

GPT3 and other text models have no sense of state. You can have the same instance work with 2 (or more) different persons and text provided by user A will have no bearing on anything user B does, since it's just trying to complete the provided text in form of tokens. As computationally intensive as it is, the whole thing is a lot simpler than it seems.

It's like we managed to get the "learning" part of intelligence more or less right, but nothing else yet. I always wanted to have an AI "trusted assistant" kinda thing, but even the biggest GPT3 models don't give me much in terms of trust. Wouldn't trust it to turn a lamp on and off as it is. (I know GPT3 is just a text model but you know what I mean). And if it happens, it won't be run locally because the hardware requisites for this stuff are obscene on a good day, which opens a few cans of worms (some of which have been opened already).

*useless as in getting anywhere close to intelligence. It can become decent entertainment and might have some interesting uses to correlate datum A to datum B/C/D, recognition and so, but so far it's just fancy ELIZA. An "educated guess engine" at best.