Reading is not the issue as well because most code is terrible and it doesn't matter how easy you make it to read the code I'm still going to waste my time reading it. So I'd say that the problem again lies elsewhere. Few people have the aesthetic sense to write elegant code. It doesn't matter how many tools and abstractions you throw at the problem there are still going to be people writing terrible code.
Writing and reading code or doing anything else with it is in many ways like art. It is also like science and craft and a bunch of other things that require creativity. Even with the accessibility of paints and all the accompanying technology we still don't have the likes of Michaelangelo and Picasso being any more prevalent than they were around the time those guys were alive. Literature is another good example. We teach everyone to read and write but it doesn't matter how much money or technology you throw at it (ebooks, libraries, etc.) we still don't have any more great writers than we did a century ago.
This is not a technology problem. It is a culture and human problem for which you are not likely to find a technical solution.
Most reasonably skilled programmers can read code. They choose not to. Cultural problem not technical one.
I've been in the industry a long time and the best codebases I've seen had a handful of things in common:
* They were written by highly competent programmers who all had an interest in doing a good job. * The code was neat, well-document, and sensibly structured based on agreed upon standards. * The programmers writing them weren't forced to inject changes faster than they could compensate for them.
That's it. Most of the disasters in industry are the result of extrinsic demands. People not caring or people. People documenting nothing. People under extreme deadline pressure hammering in something that then has massive, long-term design ripple effects on the rest of the codebase. Do this several dozen times and you're almost certain to produce a disaster and some point.
Therefore, most software problems have to do with people in over their heads or making changes haphazardly to meet deadlines. These cultural issues seem largely to stem from people thinking that writing software is a deterministic process - it isn't, and has more in common with chaotic systems than linear deterministic systems. Hell, I've written virtually the exact same piece of software twice at one company and each time produced both identical bugs and completely new bugs.
We would probably benefit more from understanding what software actually is and educating people about that than attempt to technologize away our problematic understanding of technology.</screed>
Elegant code does not necessarily mean readily understandable code (or "readable" code). For example, some Haskell programmers like writing extremely elegant code -- so elegant that you can base a whole new (elegant) branch of mathematics on it -- yet it is no more readable than, say, many early BASIC programs.
But why pick on Haskell? I've never seen Microsoft Word's code, but let's imagine it were the paragon of OOP design. Then, I'd like to add a feature similar to the spellchecker, that tests whether consecutive sentences rhyme. Now, I could probably find the spell-checking code rather easily, only to learn that it is attached to the main program via the most beautifully intricate plugin system -- with lifecycle management, runtime loading and unloading and whatnot -- that it takes me a few days only to learn that bit.
The point is that software is complicated, and very non-standardized. Code readability rests only in part on its structure, and a lot on how many "advanced" language features are used, the number of libraries used and their familiarity. You'll probably find "terrible" code that uses a couple of popular, mature libraries, that you're familiar with, much easier to understand than the most "elegant" polyglot codebase (written in both Python and Haskell, because, you know, the best tools were picked for each job) that makes use of 10 of the newest, shiniest libraries you've read a lot about on HN and always wanted to learn but never had the time to.
I want programming to be like reading, with most people able to skim most article-length pieces of prose -- even if it's poorly written -- and get a sense for its global organization. That doesn't require teaching everyone to write like Shakespeare.
To reiterate, you're responding to things I didn't say. Code shouldn't have to be perfectly designed to be readable, and nobody should have to wade through utter crap either. Very often code starts out nice when it has one author or three, and gradually turns to crap as more cooks are added. I want to eliminate that dependency on author churn, to have it be beautiful or ugly based on the capabilities of the programmers involved, not on the difficulty they had understanding those who came before. To make progress on this project, I find it most valuable to utterly ignore aesthetics.
That article you linked to (http://alistair.cockburn.us/ASD+book+extract%3A+%22Naur,+Ehn...) connects the dots really well. Programming is really about building theories and then implementing them with computational building blocks and then transferring the understanding of those theories. Short of developing mind reading I think there is an irreducible complexity in that endeavor that is impossible to skirt around.
In light of that article I'd like to amend my comment about aesthetically pleasant code. Some programmers are good at structuring things so that the overarching theory is present throughout all code level structures. That kind of code is both aesthetically pleasant and easy to read. I don't know if this is some kind of special talent of if it can be learned but given that most software is a confused mess I'm willing to bet there is a large talent component.
I started out thinking you couldn't solve social problems with technical solutions. Now I think social problems arise in the context of configurations of technical energy barriers. Making something easier can make good behavior more or less likely to arise. So it behooves technologists to think hard about what they make easier. But this is getting abstract, and I need to show examples of what I'm trying, what I'm keeping and what I'm discarding. If you send me an email I'll show you what I have.
Not necessarily. It can also be indicative of the circumstances in which the code was written. Even great programmers can write terrible code if they are stressed out or overworked. So you can't just look at the code they have written and jump to conclusions about their programming ability.
This is what we're trying to address at Sourcegraph (https://sourcegraph.com/). 80% of programming is about reading and understanding code, not writing code.
Why don't existing tools focus more on helping people read and understand code--and, more broadly, collaborate on a development team? Things like:
* seeing everywhere a function or class is used, in context (like https://sourcegraph.com/github.com/joyent/node/.CommonJSPack... on the right side)
* seeing who at your company knows the most about an area of your codebase
* seeing the history of changes, in terms of functions/modules added, not just lines (like https://sourcegraph.com/github.com/fsouza/go-dockerclient/.c...)
* having a long-lived discussion about a module/class/function that doesn't vanish after a commit is merged or the file's lines shift around
So far, most of the innovation in programming tools has been in editors or frameworks, not in collaboration and search tools for programmers.
While there are great editors and frameworks, the lopsidedness is unfortunate because making it easier for programmers to learn and reuse existing code and techniques, and to collaborate on projects, can have a much bigger impact than those other kinds of tools. That's because, in my experience, the limiting factors on a solo developer's productivity are the editors and frameworks she uses, but the limiting factor on a development team's productivity is communication (misunderstanding how to use things, reinventing the wheel, creating conflicting systems, not syncing release timelines, etc.).
...but I can't give you money without Java/Scala support. Roadmap? Pleeeeease? =)
Find All Usages and Jump To Definitions are the reason I spend 400 EUR on Resharper licenses, it makes reading code so much easier..
It lets you cross reference and document any language with gigabytes of code, e.g. Android platform code. Jelly bean ~9 gb of code.
Try it out: http://www.srcmap.org/s/sl.htm/p=android-4.2.2_r1
Sample Documents created from that app are here:
http://www.srcmap.org/p/1/4af293f91271/Android_Init_startup_...
http://www.srcmap.org/p/1/80c16319d25e/Docker.html
http://www.srcmap.org/p/6/4a88339ceef3/mtpd_source_code_stud...
http://www.srcmap.org/p/1/212c9a285186/Google_IO_2014____sch...
The ux is ok. A few gigabytes of code db running in a cheapest $5 instance of Digital Ocean.
The app is very simple ~8 mb of standalone binary, no external dependency do any other app/db.
I like to sell it as team/site license for large dev team in the future. The traction doesn't seem to be there yet.
Because my burn rate is < $150 per year, I have ~1500 users / month come to the site from pure google search alone.
I am just slowly experiment with different features/msg/channels.
Perhaps the biggest gain yet to be realized in the programming environment is the use of integrated database systems to keep track of the myriads of details that must be recalled accurately by the individual programmer and kept current in a group of collaborators on a single system.
Surely this work is worthwhile, and surely it will bear some fruit in both productivity and reliability. But by its very nature, the return from now on must be marginal.
Tools will not solve the essential difficulties of software engineering.
This tells me it's not that understanding how that code works is intellectually difficult; rather it's discovering how it works is time-consuming.
It would be amazing if you could "interact" with the code about its structure and intent the same way you might interact with its author.
"There are so many times where, if I could sit down with the original program author for 5 - 10 minutes, I could understand more from that interactive back-and-forth than several hours of reading code in solitude."
Peter Naur wrote a paper in 1985 where he conjectured that this seemed to be a universal law. No matter how much documentation authors provided, new programmers still needed to talk to them. http://alistair.cockburn.us/ASD+book+extract%3A+%22Naur,+Ehn...
So you could choose to hear what the author was saying right when he was working on a certain area of the code. (You'd normally turn it off, but if you're really stuck it might be a good last resort.)
Has helped me figure out what the hell I'm doing many a time.
FWIW I think you are exactly right, it made me sad that Google expected people to be 'useless' for anywhere from 6 to 9 months of their early employment as they tried to get their head around the code base and tools. Having the ability to eliminate that spin up time would probably triple their productivity numbers.
The biggest difficulty with current languages is their reliance on absolute determinacy. Programmers must express in exact form what computations ought be performs, and the languages possesses no intelligence in themselves. I think this needs to change. Inference should have applications outside type declaration, and the vast knowledge base that is the Internet should be taken into account.
At this point, there are two promising pursuits that I've seen. First is Wolfram Language, which you've all heard of. The other is Escher, which enables "programming in analogies". It's on early stages right now, but I'd encourage you to check it out at https://github.com/gocircuit/escher.
In the end, an analogy-based approach seems inevitable though, especially once natural language-based inexact. grow popular. Everyday speech is littered with ambiguities, and programming "languages" need to handle these. They can't throw compilation errors upon each inexact command.
The software fulfillment process has a lot of issues.
We are stuck in a local maxima and jumping the chasm to the next maxima with ideas like "what comes after programming languages" is really really hard.
Sadly, one of the biggest hurdles is self preservation because software developers don't want "software development accessible to the widest part of our society". I hate to say it, but there are Luddites among us.
So, I tend to agree that a startup is the wrong vehicle for this.
> I think we have to start with reading.
I agree with this point. People understand things differently. There should be multiple ways for people to read and understand software programs. Unfortunately, there is only one major way now and that is with source code.
I do think that tools like IFTTT are moving us towards more accessibility but it is a small step.
I like what you are working. Keep the conversation going.
It's a strange form of Luddism that seeks to limit opportunities to integrate the existing skills of the labor force into the structure of production. We don't have much in the way of unions or guilds, so perhaps the only effective way to restrict the labor supply is to keep the tools user-unfriendly.
But are you sure the RFS actually disagrees with you? What you're working on covers 2/3 of what the RFS deems worth focusing on. Collaboration and understanding.
†I think it's clear what comes after programming languages. A return to their roots. Programming a computer today is like moving by telling every single muscle what to do. To make writing code easier we want to be able to do more with less, to have the code figure out from context what the best thing to do is. We want more intelligence in our compilers. It is telling how powerful the notion of AI is when so many world changing ideas (functional programming, OOP, search, Databases [influence goes: prolog -> datalog -> SQL]) are compromises, failed attempt and detritus of AI projects.
So I'll stand by my statement that the RFS's phrasing is -- not wrong, but rather -- misguided as stated. Or maybe "misleading" would be better, but that carries connotations of malice.
1 http://nipy.sourceforge.net/nipype/ 2 http://nipy.org/nipype/0.4.1/_images/graphviz-01bb1b88b33f7c...
Software engineering won't exist as a profession until it really is turtles all the way down, and not a pig on a fish on a tiger etc.
Is this a reference to something, or should I point here if I want to reference this idea?
Both those things literally only came after reading so much code! No shortcuts from the Dev environment (idea, docs) aside from navigation and search!
from Bret Victor:
http://worrydream.com/LearnableProgramming/
and intentional programming from Charles Simonyi:
Once Apple and other companies showed what the world looks like when a phone is a giant touch screen + giant battery, phones with buttons become a niche, not the norm.
I don't know what the programming paradigm is that completely changes that, but I do know that I haven't seen it yet and everything we've done so far seems to completely miss the mark.
Someone is going to come along with some other radical assumption about what designing/implementing software really is about.
I know that it is not about static vs dynamic, functional vs. imperative, etc. like we've traditionally thought. Maybe it will be more like using Excel or Gmail. Maybe it will be more like flowcharts, or maybe just like sketching out a structure and the machine knows how to wire it together for you. Maybe humans won't be involved at all.
Speculation is endless, but nobody has had the iPhone unveil of new programming paradigms yet.
I'll suggest an alternative approach that might stimulate your neurons. Look back at the ways that your life is improved reading code today compared to twenty years ago. When I look back, the highlights for me are:
a) version control, and
b) automated tests
Forget all their benefits to the people writing code. For me, the guy with one afternoon to make a tweak to an alien codebase, it has been invaluable to have not just the current snapshot of a codebase but a story of its evolution so that I can see what the skeleton of the program used to look like in a simpler time, and gradually play enhancements forward atop it[1]. It has also been invaluable to be in situations where I can go, hmm why do I need _, why don't I just rewrite it as _, try it, and find a failing test with an enlightening name.
What's common to these two ideas is that they are additive. You don't have to give up programming languages. You just need new ways to share information between programmers besides just the code that runs "in production". My current favorite candidate for the next augmentation for codebases is logs. More info: http://akkartik.name/post/tracing-tests. If that whets your appetite feel free to get in touch. My email is in my profile.
[1] More details of how this helps: http://akkartik.name/post/wart-layers
Programming as it is now is a 2 way communication mechanism, we are writing code for other humans to understand, but for a machine to translate into what we intend for the machine to execute.
So, your point of making code a better human to human communication mechanism is absolutely correct.
What I really wonder about is what set of tools could exist that could solve whole sets of low level problems that we are specifying now that we shouldn't. Sort of like, right now we have blueprints and so on, and then humans look at those and build a house. Yet, with 3-D printed houses, you feed a set of materials and specifications into a machine, and you get a house built for you.
What is the programming equivalent that would allow us to "3-D print" software, so to speak? For physical things, it seems to be the combination of 3-D CAD systems, 3-D printers, and maybe some thought about how things need to be designed to fit this approach.
I have no idea exactly, but I could see a future where instead of paying people to hand craft all the code, that a series of features, modules, structure, etc. are specified, and the software is just "built". There would be more effort in the specification, but less on the build.
I think "naked objects" or "apache isis" are the best examples we've got for that for now , but they are rooted in somewhat complex java code, instead of being rooted in an easy to use tool, such that lets the business analyst who had some course , sit with the client and fully define a running system , step by step. In some cases the auto-generated UI would be good as it is , and the system will be used as-is.
In other cases , we might need easy tools to customize the UI.
When you think about it, the job of a designer is often quite systematic. You have some entities/data you need to communicate to the user, through whatever interface/device that's available to him. A touchscreen, a keyboard, knobs, LEDs, microphone, speaker, paper, etc.
When your "user" is a computer, JSON (while not perfect) seems to do the job as an interface. In the case of humans, JSON does a poor job at efficiently communicating information.
For humans:
- A calendar is better than "2012-11-05"
- A color is better than "#FF3300"
- An image is better than "http://example.com/image.png"
- A clickable link is better than "http://example.com/document.html"
The list goes on. We can easily generate a basic UI based on complex entities, and map specific types to custom/reusable templates if needed.
Now that your UI can automatically be generated from data, you can build an app (business model) once, and make it usable (and actually look and feel good/native) on any device. A smartwatch, a smartphone, a smarttv, etc.
Basically, the core of software development should be knowledge representation. Describe the world semantically (with RDF or similar technologies), and let the UI-compiler generate a UI for any given target platform, language, culture, user preferences. That's what responsive design should be all about.
It seems that there's something missing from all the existing implementations of your suggestion. Perhaps it is that the UI and data flow primitives that we have aren't flexible enough to express a lot of business models, thus requiring custom implementations.
Regardless, that was kind of my point - all these systems are only good as long as you stay within their constraints. The unfortunate thing is that the constraints always end up being way too tight in practice.
Would love to get in touch with you.
I've been working on this for several years, though a startup seems the wrong vehicle for it. I think the description in the RFS is misguided:
"We’re interested in helping developers create better software, faster. This includes new ways to write, understand, and collaborate on code, and the next generation of tools and infrastructure for delivering software continuously and reliably."
There's a blind spot in prose like this that gets repeated all over the place in our community: it emphasizes writing over reading. I think we have to start with reading. My hypothesis is that we need to reform the representation of programs to address this use case: I download the sources for a tool I use, wanting to make a tweak. How can I orient myself and make a quick change in just one afternoon? This is hard today; it can take weeks or months to figure out the global organization of a codebase.
You can't "deliver software continuously and reliably" until you rethink its underpinnings. Before the delivery problem there's a literacy problem: we programmers prefer to write our own, or to wrap abstraction layers over the code of others, rather than first understanding what has come before.
More on my approach: http://akkartik.name/about