News for hackers. Stuff that matters.
Re:Smaller Code, Better Code by libeclipse on Sunday February 05, 2017 @06:20AM (#13571870)
There's a lot of people jumping on the bandwagon of down-voting anyone who dares to criticise the code, but I'm going to give it a go anyway.

There's a reason why readable and beautiful code is favoured: it's so that anyone else that opens the source and tries to understand it doesn't have a difficult time, and therefore, anyone that tries to contribute doesn't have a difficult time either.

Looking at the project's Github page, I can see that there's no contributor even coming anywhere close to the project owner. Whether that's because of the obscurity of the codebase or for another reason, I can't comment. However, it does stand that the project owner is the only real contributor, and so the minimum that he himself has to consider is if he can understand the code.

Having said that, looking at the code does make me cringe. I'm sorry if that offends anyone but it is what it is: the code is not very nice to look at. It seems as though it has been engineered to be as obfuscated and shrunken as possible, without any regard for readability. I mean just the file names themselves: was there really any need for single-letters?

Now the author claims that (and a lot of other people agree with him on this) it is not for the purpose of what I outlined above, but rather, as mentioned before, so that he can understand it all easily and rapidly modify it. Whether or not that's the case I do invite you to consider the fact that the post that we are all replying to is somewhat bragging about the extremely small size of the codebase.

Personally, I think this sort of code would fit in rather well on a code-golfing forum or something similar, not on a production system. Then again, it is a personal project so ¯\_(ツ)_/¯

Re:Smaller Code, Better Code by arcfide on Sunday February 05, 2017 @07:41PM (#13575664)
Don't worry about offending me with that comment. I have a pretty strong belief in why I'm coding this way, so I'm glad to have the opportunity to work with people like you who find the code scary and disgusting and see if I can't either change your mind or change the code to be better.

Firstly, about this being a personal project, it's actually a bit more than that. It serves as a research platform for a research agenda around the usability of programming languages and the HCI and pedagogy of computation, yes. However, it's also a commercially funded compiler that is commercially licensed and distributed. The compiler is still in early stages, so it's large a boutique offering at this point, but that's scheduled to change this year or maybe the next. And yes, the development team that works with me on this compiler has read the code, and while they are not as fluent in it as I am, they understand how to work with it and we can talk about the compiler and work through issues in the compiler that comes up.

Indeed, the fact that the compiler is so easy to track through at a macro level has allowed us to avoid needing extra documentation throughout, because when we have a question about some level of architecture design, we can usually pull up a page of the compiler and work through it without needing any other documentation.

And, the point of the post above was that small code is a useful metric for pushing for simplicity. There is a difference between obfuscation and small code, but my code is not obfuscated to those who need to work with it. It is obfuscatory for anyone who expects to read it like a normal program.

At the heart of this is the meaning of readability. You've implicitly defined readability as being a state for code bases that allows anyone else to read and understand the code. That's a high bar. If I write a standard proof of the uncountability of the real numbers, it's a rather high bar to say that everyone should be able to read that proof.

Also, if you look at the way that the Clang codebase is engineered, for instance, if you take any one snippet of 50 lines or so of code, it's all nice, neat, and readable. But when it comes to understanding the entire compiler as a whole, the codebase is completely unreadable. It requires external documentation to understand almost any part of that code at a macro level.

But Clang uses best industry practices and is, on the whole, what most people would consider very cleanly written code. And yet, it is essentially impenetrable from a macro level without other documentation.

Instead, I'd submit that readability is something that we should consider valuable for those who have the relevant pre-requisite understandings of key ideas and concepts when looking at a new code base.

Part of the problem is that this code base is introducing new, research level ideas into the coding space. There is a fundamental difference between it and other compilers in the approach that it is taking, and thus, you can't just look for the same patterns.

I've already touched on malleability elsewhere. SICP has a classic quote about the importance of malleability in code (amoeba vs. pyramid programming).

I'll give a simple description of the architecture. If you understand everything said in the following sentence, then you'll have no trouble understanding how the code is written, and if you don't, then learning these sub-domains of programming skillsets will go a long way in helping to clarify the design.

It's a three part dfns->C++ offline batch compiler overloading the standard Quad-Fix system function in APL interpreters to compile whole, closed namespace scripts built of pure functional dfns on the Dyalog 15.0 primitive vocabulary sans-guards through a PEG parser to a core compiler written in a Nanopass compiler architecture over a linearized Quad-XML style matrix AST representation where each pass is written as a data-flow, data-parallel function train leading to a single dispatch code generator with a runtime library header prepended to each output file containing implementations of each implemented APL primitive.

Those would be the basic set of techniques and skills that are being put to use in the compiler. If you already understand Nanopass, PEG parsers, the Quad-XML tree linearization format, function trains, and so forth, then the structure and format and design of the compiler is obvious and easy to work with after about 5 minutes of orientation. If you don't have that background, then understanding that part of the compiler is rather a difficult one. In addition to this, there are new techniques being used and applied to solve problems in this compiler itself, and those are being documented through the papers that I'm publishing on these techniques:

https://github.com/arcfide/Co-dfns#publications

Most people don't have a strong data-parallel, array style programming background, which makes the micro-level code the hardest part to understand for them. However, if you are experienced in that background, then working with the compiler passes is not difficult, provided that you take the time to understand the core idioms in play.

So, in summary, I'd say that you're right that the code looks horrendous, because your heuristics are designed for code that is completely, almost assuredly, fundamentally different than this code. However, like I said, come to the live session and see me explicate the architecture of the compiler. I'll explain a lot of the ideas I mention in the above sentence enough to allow you to walk through the code easily. If you still think it's scary, okay. I'd appreciate some ways to make it easier to work with it on a day to day basis.

Re:Smaller Code, Better Code by libeclipse on Monday February 06, 2017 @03:16AM (#13577529)
Thanks for the reply, and let me just say that you take criticism really well.

I still disagree with the premise that the code is clean, beautiful or readable, but I could concede that this may be due to me not having an in-depth understanding of it.

You speak about a small code base being a metric for a simple code base, and while this is true sometimes, it starts to fail as a metric when the code is more obfuscated than concise. This is the kind of code I'd see in a webshell, not a compiler.

I'll try and make it to your live session if I can; I look forward to it.

P.S. I don't know if this was intended or not, but you really came across as trying to make everything seem more complicated than it actually is in the "sentence" where you went over the architecture. Dumping a load of complex-sounding words in a giant sentence just sounds as if you're trying to justify your design decisions by further obscuring understanding of the project. I'm fairly sure this isn't your intention since you are hosting a live session, but it's just how it looks. :P

Re:Smaller Code, Better Code by arcfide on Tuesday February 07, 2017 @06:31PM (#13594087)
I look forward to convincing you of the simplicity of the code base. :-)

The sentence was a bit of a tongue-in-cheek sort of rhetoric. In particular, if you look up most of those words in the relevant domains, they're all standard practice ideas that are well understood around their various parts. Importanntly, none of the words or anything said in the sentence is really complicated, and if you were familiar with all of those ideas, then it would be an easy sentence. However, I wrote it in a way so that it appears to be obtuse and a bit ridiculous on the face of it. Basically, attempting a bit to mirror the code itself. The sentence very concisely and neatly describes the architecture of the compiler, but only if you know what you're reading.

One of the biggest issues with reading the compiler is that most people will have a "part" of the picture based on their backgrounds. If you have written compilers before, the overall compiler design and the strategies at a macro-level used for it will make perfect sense, but you'll balk at the data-parallel programming style. If you are an APL programmer you'll be very familiar with the basic tricks being used in the compiler and you'll easily be able to see at a micro level what's happening with the code, but not having the background in Programming Language Design (such as you might receive at Indiana University's PL course path), the overall design and the intent of the whole system won't be intuitive to you. This means that likely for anyone new coming into the code, there will be parts of the code base which feel very foreign to them. Of course, I don't know of a way of doing something new without making people learn a thing or two to understand it.

Fortunately I'm not the only one in the world that has experience in all of these areas. And even better, the code is straightforward enough that once you do learn the basic skills, it's easy to work with. But there are precious few APL/Array oriented implementers out there, and probably less than a handful of compiler writers out there that specialize in this area of languages. I hope that will change and that I can convince people that this approach is actually a very neat and easy one to work with. But that's not easy to do.

Re:Smaller Code, Better Code by arcfide on Sunday February 05, 2017 @07:51PM (#13575709)
Sorry! I just realized that I forgot to answer the question about file names. The filenames themselves are a bit of a cultural homage to historical APL development. They are a little bit of a part of my push to stay small, because if I go beyond 26 or so files, I'm in trouble. But it's also a little bit of a "self documenting" element. There's a famous example of the style of C coding that I'm doing here from the author Arthur Whitney, the K developer. He famously whipped up a little J interpreter prototype that was about a page of code and Kenneth Iverson spent some time studying that code to understand its structure and layout and found it interesting. Whitney famously tended to write software in a very ascetical style and just used single letter names for his files.

The use of single letter names in the files here is a bit of an inside joke, referencing back the style of programming of Arthur Whitney, signaling a bit of a historical "stylistic" or artistic connection, while at the same time being the first "alert" to the programmer that they are likely to see something along the lines of Whitney style C code inside of the files. It serves both as a chuckle to the APL community as well as a documentation of how you might want to prepare your mind before reading the code.

Re:Smaller Code, Better Code by RodgerTheGreat on Sunday February 05, 2017 @08:13PM (#13575810)
For those who haven't seen it before, the J Incunabulum:

http://code.jsoftware.com/wiki/Essays/Incunabulum

Re:Smaller Code, Better Code by arcfide on Sunday February 05, 2017 @07:42PM (#13575673)
Oh, and on another note, I've found that it's mostly programmers and computer scientists who struggle the most with the code. I've tried this style of programming out with high school students with little to no programming background, and they were able to pick it up and use it to do more in 12 hours than most students in an entry-level undergraduate course did in the first half of their semester.
Re:Smaller Code, Better Code by dang on Sunday February 05, 2017 @01:12PM (#13573613)
> There's a lot of people jumping on the bandwagon of down-voting anyone who dares to criticise the code

Please omit such offensive/defensive rhetoric from your posts to HN. It adds no information and is bad for conversation.

The problem here isn't "daring" to criticize, it's rejecting the unfamiliar. This is like traveling to a new country and complaining because they cook everything wrong and say everything wrong. Unfamiliarity is relative—it's not a property of the thing you're reacting to. Same with readability: it's relative to the reader.

In some contexts this is obvious. If you don't know German, you wouldn't reject a German text as unreadable or poorly written. But in other contexts, when we unconsciously assume or were taught that there's only one valid way to do something, we react with shock and distaste at work that violates known conventions. Such work may in fact be organized around different conventions for reasons we don't yet see. Good conversation across such boundaries requires a bit of distance from our own assumptions.

Programming is like the world of art this way. There are countless examples in art history of sharp departures from convention provoking shock and distaste, and people saying things like "There's a reason why readable and beautiful [art] is favoured". Riot police famously had to be called to the early shows of the Impressionists, yet the beauty of their paintings is obvious to us now.

Re:Smaller Code, Better Code by dmitriid on Monday February 06, 2017 @09:57AM (#13579926)
> This is like traveling to a new country and complaining because they cook everything wrong and say everything wrong. > If you don't know German > Programming is like the world of art

Argument from analogy?

Please omit such rhetoric from HN posts

Re:Smaller Code, Better Code by libeclipse on Sunday February 05, 2017 @03:54PM (#13574583)
Sorry if that offended you, it wasn't my intention; I tried to keep my comment fair to both sides.

Let me however just say that what you're doing in your reply is attacking the straw man. You're setting up a version of my argument and then attacking that, instead of responding to my argument directly.

I'll respond to your comment anyways. Code is art, just as you alluded to in your response. And from a solely artistic point of view, there's a certain glee and wonder at seeing short and smart code. When I browse codegolf on SE, I never fail to be amazed at the frankly fucking brilliant solutions some people come up with.

Having said that, my main point was that code like that, in my opinion, does not belong in a proper project. I get the point of it being practical for a single person, but that codebase is above and beyond what is reasonable. It's simply not nice code.

You're saying that I should not think that the code is not nice because I don't understand it, but you seem to be missing the main point in your flowery metaphors and analogies of art: art is subjective. You might find that code to be beautiful in its own way, and I'm sure that's justified to you, but I do not.