News for hackers. Stuff that matters.
Re:Gitlab AI is going head to head with GitHub Copilot by wizzwizz4 on Thursday June 29, 2023 @03:35PM (#36525611)
If someone learns to program by memorising and then re-using blocks of code, they're absolutely doing it wrong – but, also, they should be able to attribute their code.

Most of the techniques I used, I've invented myself (or learnt from the standard documentation). When I use a technique that I haven't invented myself, I look up where it came from. Half the time, my version is actually radically different (and my attribution is mistaken); the other half, I've remembered an inferior version, so I steal the better version and then attribute it appropriately.

That's one way it's different. There are others. Really, though, we should be asking the question “in what way is this the same as humans learning?”, expecting answers that would convince an education specialist.

Re:Gitlab AI is going head to head with GitHub Copilot by olddustytrail on Thursday June 29, 2023 @03:47PM (#36525758)
LLMs don't memorize and reuse. They don't really have a memory of any kind. I think the problem is they're more different from human learning than you think.
Re:Gitlab AI is going head to head with GitHub Copilot by mitthrowaway2 on Thursday June 29, 2023 @04:15PM (#36526080)
The LLM's trained parameters are a lossy memory of their training data.
Re:Gitlab AI is going head to head with GitHub Copilot by olddustytrail on Thursday June 29, 2023 @05:55PM (#36527309)
No, they aren't.
Re:Gitlab AI is going head to head with GitHub Copilot by mitthrowaway2 on Thursday June 29, 2023 @11:49PM (#36530304)
Care to explain why not?

An LLM's working memory is just its context window, but the LLM also has embedded data in its parameters, which are set during training to minimize loss. This is effectively a memory of the training data, just as much as a digital photo of my face is a memory of the photons reflecting from my face.

Re:Gitlab AI is going head to head with GitHub Copilot by Der_Einzige on Friday June 30, 2023 @03:41AM (#36531732)
Why would you go on HN and say blatantly wrong information

AI is literally 1-1 equivalent to compression. If you don't believe me, you should check out this demo of GPT-2 as a (for awhile SOTA) text compressor.

Oh its down now, but here's the HN thread to prove this exists: https://news.ycombinator.com/item?id=23618465

Re:Gitlab AI is going head to head with GitHub Copilot by colordrops on Thursday June 29, 2023 @06:05PM (#36527429)
> Most of the techniques I used, I've invented myself

This is a very strong statement considering most code is just rearranging existing patterns. Unless you are doing cutting edge academic research, I'm very skeptical of your claim.

Re:Gitlab AI is going head to head with GitHub Copilot by wizzwizz4 on Thursday June 29, 2023 @06:20PM (#36527611)
You missed a bit.

> (or learnt from the standard documentation)

This means my Python code is mostly Pythonic. (The rest is kinda idiosyncratic, but I don't often get complaints.) But also:

> considering most code is just rearranging existing patterns.

I am an outspoken critic of "design patterns". They have their place, if you're working with legacy tooling like C++, Java or Rust, but if most of your work is rearranging existing patterns, you have long outgrown your tooling and you need a better programming language. (Or you're copy-paste programming and need to learn your tooling first.)

I am doing academic research, but that's besides the point. So far, I've learned that it's rather hard to be cutting-edge if you don't look at other people's work: you end up re-inventing all the wheels, and any insight you may have brought is lost in the noise.