News for hackers. Stuff that matters.

A comment on GitHub Copi­lot inves­ti­ga­tion. Parent comment

Re:GitHub Copi­lot inves­ti­ga­tion by Siira on Tuesday October 18, 2022 @01:11AM (#33243202)
> a very optimized stitching of code it previously saw.

I have seen this line repeated many times, but I never saw it actually explained. A lookup table is dumb and easy to understand/interpret. Deep models are not that. They are also not a linear interpolation of … something. What exactly is the claim being made here? Yes, deep models don’t generalize too well on ood data. How does this make them a “very optimized” lookup table?

[ Parent | on HN ]
Re:GitHub Copi­lot inves­ti­ga­tion by nightski on Tuesday October 18, 2022 @01:36AM (#33243343)
I mean an NN is, almost by definition, a weighted lookup table. The process to generate that table is not relevant.
[ Parent | on HN ]
Re:GitHub Copi­lot inves­ti­ga­tion by Siira on Tuesday October 18, 2022 @08:49AM (#33246362)
Not at all. That would only be true if there weren't nonlinearities between the layers.
[ Parent | on HN ]