Computer Science mixed with human factors looks similar to psychology in the early 20th century (we clearly have mommy problems). The statement "there are only 2 hard problems" is cute and pithy but doesn't really match my experience over the last 15 years.
Most of my painful memories relate to: multi-threading, timezones (bloody timezones), migration a system from A to B tech (i.e. SQL to NoSQL), ensuring a distributed system does something once--and only once, refactoring a god object, delegating a problem to an uneven strength team, building a system where the team organization doesn't match the desired architecture, getting the right requirements, dependency management, and deployment. There are probably more but that's beside the point.
There is research to support some of my observations but it's hidden behind paywalls or has thin supporting evidence. Comprehensive studies are hard and a lot of the research uses only students as convenience subjects. There really isn't a strong connection between the research community and practice in order to provide the right measurements.
What I would like to see is adding support to GitHub for community research studies. There is wealth of pure code observations. It would be interesting to allow integration of survey tools and marry that to the code observations. It'll answer some of the problems but not others (i.e. the pain of maintaining a live service). And finally as part of the GitHub EULA this research must be publicly available.
> the software development industry should stop doing such a disservice to
> itself, and start funding more scientific studies in software
> development.
Yes. > hypotheses like 'naming things is hard - we should stop doing it' are given
> way too much credibility with nothing to back them up.
Also yes, but let's separate generating hypotheses and discussing them from giving them credibility. It's a net win for people to share their shower-time hypotheses with each other. It's a net win for sites like Hacker News to feature them so that the "long tail of thought" can stretch ever longer.But as you say, we need to keep some perspective and grant that these are hypotheses. For example, a lot of times, people try turning a dial to eleven:
Long iterations are hard, what's the shortest iteration cycle? What if we try releasing every two weeks?
Code review is valuable, what if we review 24/7/365 by pairing or even mob programming?
Long methods are hard to understand, what if we make a rule that they can be no longer than five lines?
And so forth. And they get good n=1 anecdotal results. My hypothesis is that there's a Hawthorne Effect going on: When you turn a dial to eleven, you find yourself forced to observe and think about what's going on very thoroughly, and in the process of doing so, you get better results.
Then you keep doing that, and eventually the new way of doing things is internalized, and you stop being forced to think very hard about things, and you regress to your old level of hand-wavey productivity/quality/whatever.
This little pet model of mine explains why things like this keep popping up, why they are often valuable exercises, but also why, as you say, we should do some real measuring before pronouncing that we're DoingItWrong™.
For example, functional programming is terrible for beginners and has a high learning curve. People who use it in production swear by it. How do we test this?
Can we trust that what is true for a beginner is true for an expert? How do you afford 10 mid-level programmers for 6 months? Is that a large enough sample size?
The fundamental problem with studies, as I see it, is that we haven't figured out how to constrain the variability yet. We have a lot of work to get out of the hypothesis stage still.
Hm. I wonder if some of the larger companies are doing that sort of thing and keeping it internal-only or using it for internal coding standards without publishing it in an attempt to gain competitive advantage?
There's some evidence to suggest that the reason why studies like this so often produce seemingly conflicting results is that there aren't very many universal rules. The same practice can be helpful for one team and harmful for another based on any number of circumstances.
Overall, software development is looking more and more like a journey to me. And a journey in the polar sea is different from the journey in the sahara. Just don't leave jenkins at home in either case.
There's an entire field called Software Engineering. But, it's more fun to just do what feels quick and satisfying instead of binding yourself to processes other people dictated.
Unfortunately, I feel like that term has been very diluted in the last few years, to the point where I'm not sure it has much difference (at least colloquially) from "programming".
https://en.wikipedia.org/wiki/Software_engineering
Then you have Agile and XP and Scrum as either fast-flux Software Engineering (without all the "formal process/documentation" and instead use informal, but blessed, processes and immediate, undocumented, decision making) or anti-software-engineering depending on how you look at it.
Think: space shuttle or airliners, not social media. Sure, they still aren't perfect, but if you told me my airplane ran on Ruby I'd rather walk to my destination.
Agile is about building the right thing. We found out that gathering the requirements up front and developing an extended timeline and holding deviations to change control ended up building the wrong thing because adapting to change was too painful.
We learned that if we wrote software that documented its behaviors in code, it was sometimes more useful than snapshots of technical documentation. Sometimes that isn't true (public APIs and protocols are the first two that come to mind), and thus the appropriate documentation at the appropriate level is justified.
We learned that even the best engineers are very poor at how long a task will take, but they are relatively good at estimating how long a task will take relative to other tasks. We learned that we could measure how long these tasks take and get a better estimate of the length of time for a total project.
The problem is that the SDLC is a bad process for many classes of software that aren't in the embedded space. Agile is good for some problems (IE, where the requirements are poorly understood) and not others.
That's the true answer. There's no one-size-fits-all solution, but people think (and argue) in absolute terms, not situational nuance.
People routinely mistake knowing the end-goal requirement ("Make a healthcare website") with knowing every sub-task requirement ("uh... we just spent $500 million with the goal of making a website and nothing actually works!").
We're in a fast society. Nobody wants to spend three years in pseudocode and dependency evaluation and documentation and procedures before laying down real code, so the best we can do these days is augment our decision making as our understanding of the problems and goals evolve then resolve all conflicts and confusions in as real-time as possible.
The initial page for it on wikiepdia is interesting too (November 14, 2001): https://en.wikipedia.org/w/index.php?title=Software_engineer...
Especially: Methods have improved through the 1980s and 1990s, but the complexity in requirements have increased fast enough to consume any improvement.
There is also a constant eagerness to develop new tools and try the newest tools in each new project.
If the programmers are good, the system will work, and the customer will not fully appreciate how complex the task was.
I think the software development industry should stop doing such a disservice to itself, and start funding more scientific studies in software development.
Random, almost shower-thought hypotheses like 'naming things is hard - we should stop doing it' are given way too much credibility with nothing to back them up.