News for hackers. Stuff that matters.
Re:TLDR explains what a piece of code does by curl-up on Thursday September 22, 2022 @05:45PM (#32945023)
How come general developer audiences aren't more acquainted with GPT-3 (and Codex in particular) capabilities? People in the twitter thread all seem completely mind blown over an app that basically just passes your code to an existing API and prints the result.

I don't want to sound negative of course, and I expect many of these apps coming up, until Codex stops being free (if they put it on the same pricing as text DaVinci model, which Codex is a fine-tuned version of, it will cost a ~cent per query). I'm just wondering how come the information about this type of app reaches most people way before the information about "the existence of Codex" reaches them.

For all the publicity around Codex recently (and especially on HN), it still seems like the general IT audience is completely unaware of the (IMHO) most important thing going on in the field.

And to anyone saying "all these examples are cherrypicked, Codex is stupid", I urge you to try Copilot and try to look at its output with the ~2019 perspective. I find it hard to beileve that anything but amazement is a proper reaction. And still, more people are aware of the recent BTC price, than this.

Source: have been playing with Codex API for better part of every day for the last few weeks. Built an app that generates SQL for a custom schema, and have been using it in my daily work to boost ma productivity as a data scientis/engineer/analyst a lot.

Re:TLDR explains what a piece of code does by P5fRxh5kUvp2th on Thursday September 22, 2022 @06:05PM (#32945233)
MS has been trying to get AI into intellisense for years now and I always turn it off.

The lack of control over it just makes it annoying. In many ways it's faster to just type out the algorithm than it is to lay the algorithm out and spend the time trying to understand what's there so I can successfully convert the code to what I need.

Then there's the lack of stability. Yesterday it did something different from what it's doing today, so I can't even use muscle memory to interact with it anymore.

Intellisense has _always_ had that annoyance factor of getting in your way sometimes, forcing you to write code in a certain way to minimize that. All this just makes it more annoying and I don't believe anyone who claims it truly makes them more productive.

Re:TLDR explains what a piece of code does by ianbutler on Thursday September 22, 2022 @07:38PM (#32946058)
FWIW I’ve been using CO pilot now for a while and I have to do very little laying out, usually just from a name and context it will give me 80% of what I want and then it’s much quicker for me to just edit it into the correct form if need be. My productivity has very heavily increased because of the amount of rote boiler plate I can now just completely obviate.

I think you should be careful to realize that though it may not fit for you intellisense is very helpful for a lot of people and that it may be your tastes as to what you find annoying that do not generalize. I for one don’t even notice the things you’re saying bother you because the mental overhead to me is very little. Just quick glance, tab to auto complete if it’s useful otherwise keep typing.

Re:TLDR explains what a piece of code does by P5fRxh5kUvp2th on Friday September 23, 2022 @08:46AM (#32951150)
What you're describing in terms of boilerplate can be done via snippets.

Think of it like parenting.

I can either convince the little tike to clean up his room or do it myself in half the time. Only with babysitting I may take the time to teach them something, with things like autopilot I have no such motivation.

I don't even like autocompleting brackets and the like. It flat isn't consistent enough for me. It's ok-ish when writing fresh code, but completely gets in the way when editing code to the point that it's easier and faster for me to just disable it.

Re:TLDR explains what a piece of code does by curl-up on Thursday September 22, 2022 @06:13PM (#32945312)
What I am talking about has nothing to do with Intellisense or your workflow. What I am saying is that, if someone in 2019 told you that there is a "thing" that is able to take a very complex sentence and qith high accuracy (and awareness of the database details) generate 50 lines of SQL, using CTEs, complex JOINs, subqueries, string formatting, date manipulation, etc, you would have been amazed. That thing now exists, and it didn't exist before. It is a complete phase shift and it cannot simply be viewed as a incremental improvement. This is a whole different beast.

Using this beast as intellisense is just one application (called "Copilot") and it has all these annoyance factors sometimes. But I am not talking about that.

To me, this is like we found a way to transform iron to gold with low energy usage, and people are complaining that gold is not that useful. And most chemists not even hearing about the news. I'm constantly amazed by this, every single day, as I read threads like this one.

Re:TLDR explains what a piece of code does by SanderNL on Friday September 23, 2022 @04:26AM (#32949331)
Thank god, finally found someone with the same issue. This is incredibly frustrating.

I'm also wondering why the news coverage of this is so abysmal. In the Netherlands there is very little if any awareness of this, in my bubble at least. (We all seem to very aware of every goddamn bowel movement of every soccer player.)

Even my government seem to just very recently become aware that using data and generally working in a structured manner is preferable to just winging everything all the time. Some departments are even starting to use basic statistics which some even call AI. Nobody is quite sure what anything means and how to make sense of it and all these high-level decision makers studied history, administration or something legal. Absolutely nobody with a clue about anything beyond '80s tech - if even that. It's downright disturbing to see this immense gap and we are supposed be somewhat advanced. But I digress..

Re:TLDR explains what a piece of code does by wrikl on Thursday September 22, 2022 @06:23PM (#32945399)
I can only speak for myself, but it just hasn't been around long enough for me to properly trust any AI-driven tool to give me correct output for anything important.

I'll admit I haven't played with Copilot yet (since I don't think my employer would be happy for me to send off proprietary code to third-party servers, so I've effectively self-banned myself from using it at work*), but I'd feel that for anything non-trivial like your example of complex SQL queries I'd be reluctant to use the generated output without extra scrutiny (essentially a very fine-toothed code review, which is exhausting).

My opinion will probably change as the tools become more mature, but for now I'm treating them as toys primarily which limits the excitement.

Something like TLDR is less risky as it's not producing code, just summarising it, but I'd still feel wary to trust it since it's such a new field. Maybe this speaks more to my own paranoia than anything else!

EDIT: *and on this topic while I'm here: I'm actually a bit confused (and honestly... jealous?) on the topic of privacy for these kinds of external models. Is everyone who's using Copilot and tools like this working at non-Bigcos? Or just ignoring that it's sending off your source code to a third party server? Or am I missing something here?

It'd be against the rules to use external pastebins or other online tools that send off private source code to a server, so I'm kind of shocked how many devs are talking about how they use AI tools like this at work... is this just a case of "ask for forgiveness, not permission"?

Re:TLDR explains what a piece of code does by curl-up on Thursday September 22, 2022 @06:33PM (#32945489)
Check out the SQL example I posted below. If you're interested, I'd be happy to post more. To me, this is not about "is the machine accurate enough already". Maybe it isn't and it needs to mature. But the door has been opened now, and it's only up to "technical details" now.

And I'm not saying this can replace developers, as it clearly isn't capable of building complete codebases and reasoning about the system as a whole. But writing self-contained code snippets seems like a solved problem to me, and I think that's the biggest thing that happened in our field since a long time ago.

Re:TLDR explains what a piece of code does by lupire on Thursday September 22, 2022 @10:36PM (#32947343)
Code generators are old. Matching code to text is clever.

Generating correct code would be useful.

Re:TLDR explains what a piece of code does by curl-up on Friday September 23, 2022 @03:50AM (#32949126)
Please point me to an "old" model that was able to do something similar to my example, and was general enough to do that not only for any custom schema, but for basically all often used languages as well (both natural languages as input, and programming languages as output).
Re:TLDR explains what a piece of code does by visarga on Friday September 23, 2022 @12:58AM (#32948228)
It's possible to start with tests and then generate the code, using tests to ensure it is correct. Humans iterate many times on a piece of code - write, execute, compare with your desired output, AI should work step by step with executions and feedback too.
Re:TLDR explains what a piece of code does by Wherecombinator on Thursday September 22, 2022 @06:35PM (#32945509)
I’ve heard AI researchers describe this phenomenon before. As soon as something is discovered or invented it immediately becomes trivial and boring. The goalposts shift, and now they have to find the next amazing thing that will suffer the same fate.
Re:TLDR explains what a piece of code does by P5fRxh5kUvp2th on Thursday September 22, 2022 @06:22PM (#32945392)
I think starlink is a bigger deal by far.
Re:TLDR explains what a piece of code does by curl-up on Thursday September 22, 2022 @06:40PM (#32945544)
Can you elaborate? To me (but I know very little about it) it seems like part of the incremental progress in the internet availability. What am I missing?
Re:TLDR explains what a piece of code does by lupire on Thursday September 22, 2022 @10:37PM (#32947346)
It takes availability from 0 to 1 in many areas.
Re:TLDR explains what a piece of code does by Izkata on Friday September 23, 2022 @09:27AM (#32951653)
> Intellisense has _always_ had that annoyance factor of getting in your way sometimes, forcing you to write code in a certain way to minimize that. All this just makes it more annoying and I don't believe anyone who claims it truly makes them more productive.

I have the same issues with these tools, but the one situation I can imagine it being really useful is people who are good at reading and understanding code, but are slow typists. Or more particularly, people who have to think about typing, no matter what the speed is (though I think they're usually the slower ones). I believe it's only once you can type without thinking about typing, and have done it for a while, that these tools become an annoyance because you've gotten used to not interrupting your thoughts on the problem at hand.

Re:TLDR explains what a piece of code does by iudqnolq on Thursday September 22, 2022 @07:36PM (#32946040)
I have no use for it, and don't expect ever having a use for it. 95% accurate, 99% accurate, and 99.9% accurate are all aweful in this context.

It's something run repeatedly, so small chances will occur. Amoung it's failure states are being very, very wrong in ways that are hard for a skilled human to detect without more work that writing from scratch.

And no one in the space is discussing ways to eliminate categories of bugs, only ways to reduce the frequency. Most of those solutions have the side effect of making the less frequent bugs harder to detect. On balance, that's worse.

And, less importantly, it's only useful for writing boring code that should probably be generalized to an API. Sure, I write plenty of that, but it's not an exciting area to follow in my spare time.

Re:TLDR explains what a piece of code does by johnfn on Thursday September 22, 2022 @08:33PM (#32946475)
> 95% accurate, 99% accurate, and 99.9% accurate are all aweful in this context.

Really? What exactly is the bar then? I'd say most professionals I know hover somewhere between 95% and 99%.

Re:TLDR explains what a piece of code does by iudqnolq on Thursday September 22, 2022 @10:48PM (#32947440)
Assume it's per invocation, and each invocation generates a few dozen line function. How many such functions do you write when you get in the groove? If you multiply it out, you'll probably end up expecting a few bugs a day at 95%, similar to most humans might write.

Except you're pretty used to the sorts of bugs you write, and the AI isn't you. So these bugs will be harder to find.

So why is this better than writing by hand? Most of the hard work of programming is figuring out specs and debugging, not banging out well understood and specced implementations.

Re:TLDR explains what a piece of code does by 8n4vidtmkvmk on Friday September 23, 2022 @03:20AM (#32948943)
wat? who's using the results without reading it and writing tests? not many bugs should be slipping by
Re:TLDR explains what a piece of code does by lupire on Thursday September 22, 2022 @10:38PM (#32947353)
What is the denominator? Lines of code? Functions?
Re:TLDR explains what a piece of code does by MartinCron on Thursday September 22, 2022 @08:35PM (#32946496)
It's something run repeatedly, so small chances will occur. Amoung it's failure states are being very, very wrong in ways that are hard for a skilled human to detect without more work that writing from scratch.

Also makes for a great plot summary for the original Jurassic Park

Re:TLDR explains what a piece of code does by vxNsr on Thursday September 22, 2022 @06:03PM (#32945220)
I was aware of and use copilot but I didn’t realize it was built on top of codex. And wasn’t even aware codex existed until you commmented.

I read hn pretty regularly but unless you’re really excited about the AI space a lot of this news washes over you and you mostly ignore it.

Re:TLDR explains what a piece of code does by curl-up on Thursday September 22, 2022 @06:20PM (#32945380)
This is what amazes me, since it seems like such big news, and people in the field are just not aware of it. Just for reference of what I am talking about, here is a piece of code that was generated, without any cherry picking at all (you just have to trust me on this, sorry) by allowing Codex to be aware of the database with some smart prompting (this is on a DB with music store data):

Q: Best selling artist per country

A: https://pastebin.com/qBVu2mvc

Needless to say, this query works and returns the data I wanted. Whether this is useful or not is up for discussion. But I cannot understand how it's not amazing.

Re:TLDR explains what a piece of code does by lupire on Thursday September 22, 2022 @10:43PM (#32947403)
What happens if an Invoice has more than one Line? Looks like it will count the Invoice total for each line inatead of each likne's amount.

If each invoice has 1 line, why does a line table exist?

Re:TLDR explains what a piece of code does by curl-up on Friday September 23, 2022 @04:08AM (#32949221)
Ha, do I feel stupid now! Just checked, and seems like I misconfigured my system (the part that builds the Codex prompt from the schema info) - Invoice and InvoiceLine tabls got confused. After fixing it now, it works as expected (using IL.Total).

Here are some more examples if someone's interested: https://pastebin.com/5Vr08N7z

Re:TLDR explains what a piece of code does by Izkata on Friday September 23, 2022 @09:34AM (#32951740)
This non-cherry-picked example is exactly one of the problems people are talking about though: It easily creates something that looks correct, and the users sees what they expect, so things get through that wouldn't have if they thought through the problem themselves.
Re:TLDR explains what a piece of code does by curl-up on Friday September 23, 2022 @12:46PM (#32954251)
Yeah, agree - building and verifying apps built on this models is very hard as they are hard to test extensively. But so are most other ML-based apps. The error in that example was on my side, not on the model. I mean, to me this seems simillar to saying that high-abstraction programming languages are a problem because their compilers are hard to develop and probably loaded with bugs.
Re:TLDR explains what a piece of code does by benmmurphy on Thursday September 22, 2022 @07:58PM (#32946207)
so it was smart enough to know that the total for the invoice could be safely used as the total sales for the artist because an invoice will only ever contain a single line item? no double counting because of multiple line items or misattribution because of different artists. that's a pretty crazy level of reasoning.
Re:TLDR explains what a piece of code does by FeepingCreature on Friday September 23, 2022 @02:25AM (#32948693)
I'd love to try Codex if I could run it on a local GPU and finetune it for my own code. I'd even push to use it at work. But as we're writing in a niche language and our code is heavily problem domain dependent, I don't feel like making my workflow vulnerable to an external supplier, even aside the IP concerns.

Call me when I can download and finetune the weights, like I can with Stable Diffusion.

Re:TLDR explains what a piece of code does by kamban on Thursday September 22, 2022 @09:17PM (#32946819)
The problem is that they don't understand the practical ways it can be used. Even tech savvy people don’t yet get it. Even my CEO, a kind of technical person, did not understand the full potential until I explained some use cases.

In one scenario, I took the slow running long MySQL query and rewrote that with Codex in 2 mins.

But I think people have started to realize the potential now.

Pitch: My app https://Elephas.app brings GPT-3 and Codex to all applications in MacOS. Many business professionals are using it.