News for hackers. Stuff that matters.
Re:Size is the best predictor of code quality by cpeterso on Monday September 26, 2011 @12:58AM (#3037716)
I just pulled Steve McConnell's (must read!) CODE COMPELTE: A Practical Handbook of Software Construction off my bookshelf. The section How Long Can a Rountine Be? references some surprising (but perhaps dated) studies that suggest the evidence in favor of short routines is "very thin" and the evidence in favor of longer routines is "compelling". These studies are probably biased to desktop and corporate software written in C during the 1980s.

The consensus is that routines should have fewer than 200 LOC, but that routines shorter than ~30 LOC are not correlated with lower cost, fault rate, or programmer comprehension. btw, the longest function I've seen in commercial software I've worked on was 12,000 LOC! I will not name names. :)

* A study by Basili and Perricone found that routine size was inversely correlated with errors; as the size of routines increased (up to 200 LOC), the number of errors per LOC decreased (1984).

* Another study found that routine size was not correlated with errors, even though structural complexity and amount of data were correlated with errors (Shen et al. 1985)

* A 1986 study found that small routines (32 LOC or fewer) were not correlated with lower cost or fault rate (Card, Church, and Agresti 1986; Card and Glass 1990). The evidence suggested that larger routines (65 LOC or more) were cheaper to develop per LOC.

* An empirical study of 450 routines found that small routines (those with fewer than 143 source statements, including comments) had 23% more errors per LOC than larger routines (Selby and Basili 1991).

* A study of upper-level computer-science students found that students' comprehension of a program that was super-modularized into routines about 10 lines long was no better than their comprehension of a program that had no routines at all (Conte, Dunsmore, and Shen 1986). When the program was broken into routines of moderate length (about 25 lines), however, students scored 65% better on a test of comprehension.

* A recent [sic!] study found that code needed to be changed least when routines averaged 100 to 150 LOC (Lind and Vairavan 1989).

* In a study of the code for IBM's OS/360 operating system and other systems, the most error-prone routines were those that were larger than 500 LOC. Beyond 500 lines, the error rate tended to be proportional to the size of the routine (Jones 1986a).

* An empirical study of a 148 KLOC program found that routines with fewer than 143 source statements were 2.4 times less expensive to fix than larger routines (Selby and Basili 1991).

Re:Size is the best predictor of code quality by fhars on Monday September 26, 2011 @01:54AM (#3037805)
Judging from my experiences, several of these studies may be missing a major confounding factor: the complexity of the problem the code solves. All my longest methods are rather stupid output formatting stuff or if/else cascades that handle tedious by mostly trivial distinctions. But for code that solves hard problems I often write many small functions for independent steps of the solution.

So the studies that measure method length/bug count correlation within a single code base or in code written within a single organization might only measure the fact that code that requires no thinking contains fewer bugs than code that does. Paging Captain Obvious. Some of the other studies address that (e.g. Shen 1985 and the code comprehension studies), but as it is so often the case in quantitative studies of things related to programmer productivity, we lack repeated measurments where the only variable is the independent factor whose influence is studied.

Re:Size is the best predictor of code quality by gruseom on Monday September 26, 2011 @04:23AM (#3038064)
That's an interesting point. There's certainly a cap on the complexity of code that can be put into a single long function. (Unless, I suppose, it has inner functions that call one another, like how people do OO in Javascript; such things can be just as complex as whole programs.) Usually it's implementing some conceptually unified thing. Even if that thing is a complex algorithm, it's still cohesive enough to be able to say what it is. And implementing even a very complex algorithm is not particularly complex at a system level.
Re:Size is the best predictor of code quality by gruseom on Monday September 26, 2011 @01:27AM (#3037766)
I've mentioned those findings here before. It's great to have them listed. It's fascinating that, even with all their limitations and age, they fall so completely on one side of the question - the opposite side to what a lot of sophisticated programmers believe.

The last time we debated this on HN, there was a disagreement about how much complexity the interactions between functions add to a program. To me, complex call graphs are even worse than complex code inside a function. I was surprised to learn that an opposing view even existed.

Re:Size is the best predictor of code quality by berntb on Monday September 26, 2011 @03:34AM (#3037957)
>>complex call graphs are even worse than complex code inside a function

Hear hear.

I have a problem specific to that, here. Some programmers follow the "no documentation should be needed, the code is obvious" dogma in a non-typed scripting language -- and you have to look many levels up in the call graph before you even find out about the damn function parameter's types... :-(

Re:Size is the best predictor of code quality by dman on Monday September 26, 2011 @05:25AM (#3038172)
Ideally you have tests which let you know how every function can be (ab)used.
Re:Size is the best predictor of code quality by berntb on Monday September 26, 2011 @08:06AM (#3038479)
If I wrote 2-20 line functions, then I wouldn't document/test every one, either.
Re:Size is the best predictor of code quality by ZeroGravitas on Monday September 26, 2011 @08:53AM (#3038607)
I think the idea is that the short functions call other (short) functions at a lower conceptual level to create a large amount of functionality. By testing that function, you're testing how they work together.

This is different from short functions being short because they don't do much.

Re:Size is the best predictor of code quality by 0x12 on Monday September 26, 2011 @06:47AM (#3038308)
One of the very hard to quantify effects here are how many of the bugs found in 'short' routines would have gone unnoticed if the routines were longer. In other words, were all the bugs that were there (both in short and in long functions) actually found?

In trivial flow (short routine) bugs tend to stand out whereas in complex flow (longer routine with multiple levels of nesting) bugs can be much harder to spot.

It may actually be good that more errors were found in shorter routines after all that is exactly what it is about, finding errors, not making errors.

Re:Size is the best predictor of code quality by ghostganz on Monday September 26, 2011 @03:26AM (#3037940)
I wonder how these 1980's results relate to object oriented code and other things that have happened since then. Even 25 LOC would be a huge method in Smalltalk-style OO for example.

Other than OO, we also have much better tools for navigating code now. That may have changed how we approach and understand unknown code.

Re:Size is the best predictor of code quality by Confusion on Monday September 26, 2011 @05:06AM (#3038144)
Another reason (in addition to those already mentioned) why these numbers are not easy to interpret:

  small routines [..] had 23% more errors per LOC
If the smaller routines had, on average, 100/1.23 = 81% or fewer of the lines of the larger routines, then they still had fewer errors per routine.
Re:Size is the best predictor of code quality by jacques_chester on Monday September 26, 2011 @01:30AM (#3037769)
Good writeup. McConnell is right in that there's a vast abyss between the researchers and practitioners in our trade.

Another good book by McConnell, which also discusses size with summaries of studies, is Software Estimation: Demystifying the Black Art.

Re:Size is the best predictor of code quality by lurker19 on Monday September 26, 2011 @10:12AM (#3038987)
Smaller functions tend to be more reusable leading to fewer LoCc per codebase, which could overcome the bugs/LoC penalty.

Also, considering the date, this research could be influenced by the bug-prone-ness of parameter passing in C as a non-memory-managed language.