When people say 'corporate culture is the problem', that's what they're talking about. Of course better technology and processes can help. But if corporate culture actively prohibits anything new/unknown/non-standard, what is the true problem? That the tools aren't being used, or that the tools can't be used?
You're not actually disagreeing with anyone who points to corporate culture as the culprit. You're just not at the root cause yet.
Yes, yes, I know many many legacy systems are out there that are decades old. And every single one of them represents major lost opportunity.
Release early, release often isn't just for startups.
That's a reasonable argument when read as "What happens to our Erlang+Ruby crystal palace when management takes over and talented devs won't touch us with a ten foot pole and we are forced to hire Java monkeys who think lambda is something you barbecue on a skewer?"
One enterprise that I spent some time at has some systems that are very old and have been continuously running. Some are on the order of 40 years old. It wasn't until the 90's that they finally decreed that new assembler code was not allowed. And lots of COBOL.
So as functionality has been added to these systems, the word "Legacy" doesn't quite give the feel of the sheer size of them.
I would agree that management resists smaller more talented teams, and add that the obscure tag is supplied by gartner, which is about as risk adverse as one can imagine.
And as you point out, there is the whole change control issue. I remember two hour meetings weekly for a small division in which changes had to be fought for and coordinated.
I don't mean to sound pessimistic, but let's say that Sun or other vendor does exactly the right thing and offers the right kind of solvent for the enterprise monolith. The systems that exist are so interwoven into the fabric of the business that not only do the IT portions of the problem change, but also the structure of the business itself. Not clear that Sun or IBM or MS can add much here.
A useful thought experiment here is to consider the spreadsheet and how, as a significant piece of new technology and forward looking tool, did change the way that business is done. Did it reach down to the core of the business to alter the monolith?
I'm just curious, because I would have thought that this would work about equally well on software and mechanical engineering projects.
Yeah I think they work differently in general. From my experience they do not benefit from iterative development the same way we do. They do but they iterate when solving the problem, usually on paper. If they make a design mistake and it's in production or prototype phase it's much more costly to address. And because of this development cycles are longer and a lot more time is spent upfront.
I've watched and assisted mechanical engineers that spent days working out the math on the reliability of a class of lasers (and belts, motors, fans) subjected to sitting in a warehouse, followed by periodic motion and change in atmospheric conditions then launched (missile launch or space launch).
After the analysis was complete I could code up the math in a matter of minutes. Then expose it to all sorts of boundary cases and add it into a mathematical model of the entire system. Sometimes, at this point, it would go back to analysis to address some new found issues.
I have great respect for such engineers. Solving real life problems, like this, thoroughly is worlds away from what most programmers enjoy - creating.
EDITED: Clarifications ~
This is partly true in my experience and you will struggle to find a Mechanical Engineer that doesn't know what Kaizen is. With engineering iterating is expensive so iteration normally would happen at design stage. However, when it comes to Construction there are different levels of iteration, for example the Consultant will produce a set of specs and drawings but is up to the specialists to produce the detail studies and shop drawings.They will also spent months with mock-ups in order to confirm compliance with requirements.
However, I think every Engineering discipline (and I include Software Engineering here) can learn a lesson or two from Sanitary Engineers:) Systems produced by Drainage Engineers virtually outlive the buildings they are installed, large part of such systems are standardized (use your standard libraries). They run for almost ever and anywhere (ie, quality assurance brings reliability), they require almost no maintenenace (minimize bugs). They have very well defined objectives (no feature creep). If they don't perform well they stink so someone notices and does something about it! They employ 'set of primitives or principles' like gravity that always works, they are simple and construction can be delegated to cheaper labour (outsourcing). It is not a very glamorous field but civilization progressed when this field progressed (social responsibility). They are not expensive to design and produce. They use standard details (ie DRY principle) and I have seen .. a lot of these Sanitary Engineers debating if their discipline is ... art, engineering or science. Rings a bell?
Mechanical Engineering like IT - should be IE Information Engineering:) - has is own disasters and Management problems.
It is a very creative field and the only unfortunate thing is that it does not benefit from - open source. In this respect anyone in Software Engineering, should count themselves lucky!
Qualifier I hold a PhD in Mechanical Engineering
This I agree with. Most engineers would kill to have the kind of feedback cycle that we have. (Disclaimer: my experience, such as it is, is with electrical rather than mechanical engineering.)
Solving real life problems, like this, thoroughly is worlds away from what most programmers enjoy - creating.
This I disagree with. Of course (mechanical) engineers enjoy creating - for example, read Florman's Existential Pleasures of Engineering.
Yup, I've seen it too, from my experience. I was once writing some desktop engineering apps, to help mechanical engineers / designers do analysis on materials etc. They spend enormous amount of time doing iterations/studies on paper. It is worth it, because it usually saves lots of money and time down the road. They don't have the luxury that software developers have.
What I meant was it's possible to implement such a system in the framework of a dow30 type culture. The few times that I've seen it implemented, management supported it. But without someone just taking the initiative and doing it, it doesn't happen. The average newly graduated junior enterprise developer with an average lead enterprise developer probably can't pull it off. So it could be enterprise just has to catch up and they are just not there yet. Maybe that makes it a cultural problem, I'm not sure.
No doubt, management has yet to see the light. They want data to show that change in their process and new technology works before they commit. There should be a place for it when management finally realizes it's not a fad but a trend.
I've worked at big and medium size companies in the past decade. I worked at a startup that was acquired by a big company. I've implemented enterprise systems from design through support for governments and dow30 companies.
Those of you that think that these systems are symptom of the corporate culture are wrong. I'm not defending the poor corporate cultures that I saw. I am defending the argument that better technologies and processes could lead to better systems.
From a process standpoint the problem is it's counter intuitive to most management that smaller, higher paid teams using obscure* technology could out produce the highly polished cogs-in-the-machine process. For example, this usually isn't true for (most) mechanical engineering projects. We all know it as fact on HN but it doesn't apply to many enterprise industries. I remember times where as a team we were denied the ability to create scripts or tools to do work for us. Or we were denied the use of open source tools. Instead we spent days doing it manually. Then, when it needed to be done for the next project, we would again do it manually. We couldn't push a small but critical change without completing a multi-day code move process. When we did write scripts, IT had to approve them which meant you were at their mercy. There were many more of these sorts of poor process decisions. And there were smart capable programmers working there, often pushing to make the right decision.
When I run into old co-workers many still believe it's impossible to implement enterprise code in something like ruby, python, lisp, haskell, etc. It's Java or C#. Common reasoning - What happens if I need a new Clojure or Lisp programmer!? Or, where will ruby be in 20 years? We know today where Microsoft support will be in the future. Look at Microsoft's support plans. Etc.
I'll add a little bit of my own opinion in here too. It is my hope that eventually there will be a few elegant generic enterprise solutions where DSLs are implemented for business analysts. It's insane how much of this work is still done by programmers. How many more times does a programmer have to code a { main-menu customer organization admin ... } window or custom approval process, etc. (Yes, I know this has been attempted often without success, it's still a good idea. And I think in the end it'll happen. It just takes time to figure out what will work)
* obscure to management.
EDIT: Grammar