News for hackers. Stuff that matters.

A comment on Google CEO says more than a quarter of the company's new code is created by AI. Parent comment

Re:Google CEO says more than a quarter of the company's new ... by Scene_Cast2 on Tuesday October 29, 2024 @10:21PM (#41991354)
This actually sparked an idea for me. Could code complexity be measured as cumulative entropy as measured by running LLM token predictions on a codebase? Notably, verbose boilerplate would be pretty low entropy, and straightforward code should be decently low as well.
[ Parent | on HN ]
Re:Google CEO says more than a quarter of the company's new ... by jeffparsons on Tuesday October 29, 2024 @10:33PM (#41991427)
Not quite, I think. Some kinds of redundancy are good, and some are bad. Good redundancy tends to reduce mistakes rather than introduce them. E.g. there's lots of redundancy in natural languages, and it helps resolve ambiguity and fill in blanks or corruption if you didn't hear something properly. Similarly, a lot of "entropy" in code could be reduced by shortening names, deleting types, etc., but all those things were helping to clarify intent to other humans, thereby reducing mistakes. But some is copy+paste of rules that should be enforce in one place. Teaching a computer to understand the difference is... hard.

Although, if we were to ignore all this for a second, you could also make similar estimates with, e.g., gzip: the higher the compression ratio attained, the more "verbose"/"fluffy" the code is.

Fun tangent: there are a lot of researchers who believe that compression and intelligence are equivalent or at least very tightly linked.

[ Parent | on HN ]
Re:Google CEO says more than a quarter of the company's new ... by 8note on Tuesday October 29, 2024 @11:29PM (#41991723)
Interpreting this comment, it would predict low complexity for code copied unnecessarily.

I'm not sure though. If it's copied a bunch of times, and it actually doesn't matter because each usecase of the copying is linearly independent, does it matter that it was copied?

Over time, you'd still see copies being changed by themselves show up as increased entropy

[ Parent | on HN ]
Re:Google CEO says more than a quarter of the company's new ... by malfist on Wednesday October 30, 2024 @11:58AM (#41996609)
Code complexity can already be measured deterministically with cyclomatic complexity. No need to use an AI fuzzy logic at this. Especially when they're bad at math.
[ Parent | on HN ]
Re:Google CEO says more than a quarter of the company's new ... by contravariant on Wednesday October 30, 2024 @05:50PM (#42000799)
There's nothing fuzzy about letting an LLM determine the probability of a particular piece of text.

In fact it's the one thing they are explicitly designed to do, the rest is more or less a side-effect.

[ Parent | on HN ]
Re:Google CEO says more than a quarter of the company's new ... by david-gpu on Wednesday October 30, 2024 @04:05PM (#41999610)
> Could code complexity be measured as cumulative entropy as measured by running LLM token predictions on a codebase? Notably, verbose boilerplate would be pretty low entropy, and straightforward code should be decently low as well.

WinRAR can do that for you quite effectively.

[ Parent | on HN ]