News for hackers. Stuff that matters.
Re:How many lines of code is Candy Japan? by Retric on Friday September 30, 2016 @09:33AM (#12612308)
This may just be me, but I find well maintained projects tend to get sticky around 10k lines of code. Much smaller than that and every new thing is just that a new thing. Much larger and it's easer for new things to kind of get tacked on at the edges. But, around 10k the project is small enough to understand and most new code tends to result in some refactoring for shared functionality. 10k lines is also fairly close to novel length.

Anyone else notice this?

Re:How many lines of code is Candy Japan? by wheels on Friday September 30, 2016 @12:14PM (#12613345)
Above 10k lines is where statically typed languages really begin to justify their overhead. I work primarily in Ruby, C++ and Java, and while for smaller projects, Ruby is far more of a joy to use, for large projects (I've worked on C++ projects well over a million lines of code) static typing is a blessing.

Refactoring Ruby is painful in a large project, even with decent test coverage (and a large portion of the tests basically enforce what you'd get automatically with static typing). C++ is doable in that you'll get (increasingly humane with clang and recent GCCs) compiler errors when you've forgotten some detail. But refactoring Java is positively germane with all of the tools that are enabled by such a consistent VM-model. (It really is neat to be able to rename a method in an object on 100k LOC project simply by renaming it in one place or to change a signature with helpers along the way.)

Some of that makes those languages less fun (and quick) to work with initially, but for projects that are likely to eventually grow quite large, there can be an eventual pay-off.

Re:How many lines of code is Candy Japan? by adamnemecek on Friday September 30, 2016 @07:09PM (#12615779)
What kills me about it refactoring c++ is dealing with headers. A lot of times, something should really be a separate file (or should me merged into another file) but it would be a pain in the ass to fix the headers.
Re:How many lines of code is Candy Japan? by webkike on Friday September 30, 2016 @09:44PM (#12616208)
This is a core problem with C++. Until we have packages, you might want to consider another language. Rust is usually a good choice.
Re:How many lines of code is Candy Japan? by sytse on Friday September 30, 2016 @06:40PM (#12615677)
GitLab is 30k+ lines of Ruby code http://gitlab-org.gitlab.io/gitlab-ce/coverage-ruby/#_AllFil...

We're not missing static typing as far as I know. We still are able to ship a lot of changes every month. I'm not sure if it is our structure (idiomatic Rails), test coverage (over 85%), something else, or maybe we should miss it but don't realize it.

One disadvantage is that static typing causes more lines of code than dynamic code. And code size is the best predictor of code quality http://blog.vivekhaldar.com/post/10669678292/size-is-the-bes...

Re:How many lines of code is Candy Japan? by adamnemecek on Friday September 30, 2016 @07:10PM (#12615784)
> One disadvantage is that static typing causes more lines of code than dynamic code.

Not if the statically typed language uses inference.

Re:How many lines of code is Candy Japan? by sytse on Friday September 30, 2016 @07:18PM (#12615798)
That would be the best of both worlds indeed. Hope we get this for Ruby https://codon.com/consider-static-typing
Re:How many lines of code is Candy Japan? by saurik on Saturday October 01, 2016 @02:12PM (#12619124)
The seminal languages with static typing had this feature, so I feel it does the field a disservice to say "the best of both worlds": there are a few annoyingly popular statically typed languages which don't have this property, and they should be considered deficient to the extent to which it is not for theoretical type depenence reasons.
Re:How many lines of code is Candy Japan? by jerf on Friday September 30, 2016 @10:04AM (#12612477)
One of the most interesting videos on programming I've watched in a while addresses that idea: https://vimeo.com/108441214 The video can be summed in a lot of ways as "What if we actually took that seriously and started designing around it?"

I haven't had enough chance to do it, but I'm at least intrigued.

Re:How many lines of code is Candy Japan? by lj3 on Friday September 30, 2016 @10:20AM (#12612585)
Isn't that the Unix philosophy in a nutshell? Lots of small programs that only do one thing, but can be strung together? They can also be rewritten without worrying about affecting anything else.
Re:How many lines of code is Candy Japan? by lmm on Friday September 30, 2016 @09:39AM (#12612330)
When working in Python I noticed that 20kloc was the point at which I stopped being able to hold a project in my head, at which point I became unable to safely refactor, which in turn meant refactoring basically stopped being worth it. Any codebase larger than that, the core ossified, and new functionality was only neally possible to add at the edges.

I think I can do a lot better than that now that I'm using Scala. Certainly I don't feel like I've hit the limit yet.

Re:How many lines of code is Candy Japan? by anewhnaccount on Friday September 30, 2016 @10:23AM (#12612611)
This is where you need tests. Leaning on the type system can take you further, but at some point you're going to need tests with Scala too.
Re:How many lines of code is Candy Japan? by lmm on Friday September 30, 2016 @10:37AM (#12612691)
Disagree. Effective use of types has a much better cost/benefit than tests, and it's possible to go all the way down to 0% defect rate using just the type system (by encoding proof of correctness into the type system) if you really need to.
Re:How many lines of code is Candy Japan? by exi2 on Friday September 30, 2016 @12:28PM (#12613461)
I highly doubt that you can prove correctness with types alone.

How would you verify the correctness of this function with types alone:

Integer add(Integer a, Integer b) { return a + b; }

How would types verify for me that the result is correct and not a+b+1?

Formal proofs work for this but to make them work you need more than just types.

Re:How many lines of code is Candy Japan? by omaranto on Friday September 30, 2016 @02:27PM (#12614268)
The return type of your function is too broad: It should be the "type of integers which can be proven to be the sum of a and b".

That sounds like a joke but it's what going overboard with dependently typed programming languages is like.

Re:How many lines of code is Candy Japan? by vertex-four on Friday September 30, 2016 @11:09AM (#12612897)
Only insofar as your types essentially become the bones of the program... which you'll then want to refactor at some point.

Types are awesome. Languages like Rust give them to me without having to learn a ton of theory. However, eventually, I realise that my types have boxed me into a certain design - and I need to refactor heavily, which involves rewriting a large chunk of my types which I was relying on to show me that my code was correct.

Re:How many lines of code is Candy Japan? by lmm on Friday September 30, 2016 @11:16AM (#12612953)
Of course you sometimes need to refactor your types. That's fine; the type checker will guarantee that you can do so safely. You probably want to be cautious about changing your axioms, and to that end it might be worth distinguishing them explicitly, but that's easily done (e.g. you can put them in a separate module).
Re:How many lines of code is Candy Japan? by vertex-four on Friday September 30, 2016 @12:57PM (#12613637)
It'll tell you your program compiles. Unless you've gone seriously overboard and typed every piece of information down to primitives, and built your types so there's no possible way to convert them out of those types at the wrong time, you've still got a chance of having bugs.

Functionality testing will tell you when you've failed to encode an invariant in your type, and there's a good chance that the reason you failed to is just that it's more effort than it's worth. Else we'd all be writing proofs for every program in Coq.

Re:How many lines of code is Candy Japan? by paulddraper on Friday September 30, 2016 @03:46PM (#12614792)
Do your tests also contain every single piece of information?
Re:How many lines of code is Candy Japan? by vertex-four on Friday September 30, 2016 @05:55PM (#12615502)
You can at least add a new test easily when someone reports a bug. Tightening up your types, otoh, is a trade-off against brittleness.

In practice, as I alluded to in my previous post, you really want types combined with broad functionality testing across a range of scenarios. You really don't want to be relying on "it compiles, I assume it works" unless you're able to back out of a deployment in seconds when it doesn't.

Re:How many lines of code is Candy Japan? by lmm on Saturday October 01, 2016 @03:32AM (#12616915)
Tests are much more brittle than types; automated tooling doesn't know what the tests are saying, and there's no way to make code "test-generic".

Some things are probably not worth typing, but if they're not worth typing they're not worth testing either. Whatever your target defect rate, the most efficient way to hit that target is using types.

Re:How many lines of code is Candy Japan? by vertex-four on Saturday October 01, 2016 @04:47AM (#12617079)
> there's no way to make code "test-generic"

Sure there is. Stop testing random bits of internal functionality, start testing huge modules with well-known (semi-public) APIs, or your entire program, and combine that with a ~reasonable~ type system. You want full-system testing. You don't need a whole lot of tests to be sure "it's basically working", and it's a lot easier to turn repeatable bug reports into tests.

When you inevitably break a piece of functionality despite your types, you might not know precisely how it broke, but at least you know it's broken without a customer calling you up. And every time a customer does call you up, you can add a full-functionality test - "this is what the user did, and this is what I expect from that".

Every other engineering branch backs up good practices and solid math with lots of testing. So should we, types or no types.

Re:How many lines of code is Candy Japan? by sushisource on Friday September 30, 2016 @12:13PM (#12613326)
Been using Scala a lot at work lately and I've definitely helped myself out with using types to good effect, but I still feel like I could be doing better... the one lesson I've definitely learned is if you're going to be serializing things pick a library that makes upgrading those serializations easy or you'll live in fear of changing types.
Re:How many lines of code is Candy Japan? by lmm on Saturday October 01, 2016 @03:34AM (#12616919)
Yeah, I'd recommend Thrift or Protocol Buffers for serialization. Having distinct serialized objects feels like overhead to start with, but it's well worth it if you ever want to change the code without risking incompatibilities.
Re:How many lines of code is Candy Japan? by rexpop on Friday September 30, 2016 @11:48AM (#12613158)
What can I read to be convinced of your viewpoint?
Re:How many lines of code is Candy Japan? by lmm on Friday September 30, 2016 @11:58AM (#12613240)
I don't know - I reached it through direct experience (a number of years programming professionally in a language with a good type system) rather than anything I read.

https://spin.atomicobject.com/2014/12/09/typed-language-tdd-... makes the case that many tests can be replaced more effectively by types (though the use of Java makes the point less clear than it should be, and means that some properties are so difficult to express through types that the author prefers to use tests). That one can prove correctness with types is well-known, and it stands to reason that there is no need for tests for formally verified code. These two things together suggest that there's never a point on the defect rate/cost curve that's easier to reach with tests than with types, but they don't rule it out entirely.

Re:How many lines of code is Candy Japan? by owenconti on Friday September 30, 2016 @03:10PM (#12614571)
How would your static typing save you from:

  model.setTitle( dto.getDescription() );
Re:How many lines of code is Candy Japan? by ultramancool on Friday September 30, 2016 @03:17PM (#12614613)
(assuming the bug here is a description being assigned to a title)

getDescription simply needs to return a Description object while getTitle returns a Title object. Duh.

Only in the final rendering should it get turned into a string and there it should only accept a Title type for doing so.

Okay, I'm kind of joking, but this is seriously the solution if you really want to get down to it - primitive overuse is the problem, so wrap it in a type that makes it explicit. You need cheap and easy types to take full advantage of this sort of thing.

EDIT: And even with those reading code developed in such a way may go overboard at some point. I'm not sure as I haven't really seen it done to the fullest extent possible. Typically even when taking a fairly strong advantage of such a type system, unit tests and stuff are still used, but theoretically you could enforce their parameters entirely in the type system. Of course, that enforcement can also be incorrect, but it does make it clearer in many cases.

Re:How many lines of code is Candy Japan? by owenconti on Friday September 30, 2016 @03:49PM (#12614821)
I can see your argument in some cases, but if you are accepting input, at some point you'll have to accept a string (or any primitive) and convert it to your custom type.

How does static typing ensure you're mapping the correct JSON input fields to your custom types?

Re:How many lines of code is Candy Japan? by lmm on Friday September 30, 2016 @06:50PM (#12615711)
> How does static typing ensure you're mapping the correct JSON input fields to your custom types?

Ideally you push it all the way out. Rather than JSON you use Thrift or similar where the messages are strongly typed.

But if you do use JSON, how much benefit does unit testing your JSON mapping really give you? The mapping in code is just a list of field:field pairs, the test is just the same thing, is there really any value in repeating it twice? I'd sooner rely on careful code review than a test.

Re:How many lines of code is Candy Japan? by retro64 on Monday October 03, 2016 @11:01AM (#12627851)
That's...horrible. This is not a comment on you or your experience, but is this typical for Python? I’m used to working in C++ code bases with several million lines of code. Does Python really begin to fall down that early? I’ve been looking at using it in an upcoming small project, but I expect it to be larger than 20K. Now I’m reconsidering.
Re:How many lines of code is Candy Japan? by shanwang on Friday September 30, 2016 @09:46AM (#12612372)
I agree for dynamic typed language like python and javascript 10-20k is a good limit. With a static typed language like C++, if the abstraction is well designed people should be able to handle more.
Re:How many lines of code is Candy Japan? by Cthulhu_ on Friday September 30, 2016 @10:35AM (#12612677)
There will be a lot less mental overhead in statically typed languages, thanks to sane / functional autocompletion and a lot of information being abstracted away into types, so on that I agree. The other part may be that more loosely typed languages have a tendency to be less strictly structured, but that depends on the project.

The project I'm working in atm is neatly structured and at ±60K, but you definitely see certain team members knowing more about one part than the other. We've also started a POC to start moving to Typescript for the more critical areas of the application (services, data models).