News for hackers. Stuff that matters.
Re:Banks scramble to fix old systems as IT 'cowboys' ride in... by repolfx on Thursday October 04, 2018 @05:16AM (#18138304)
Self made ... by both the banks and the programmers.

As the article notes, these developers don't seem to have done a great job of documentation. Perhaps deliberately, given the comment about personal vindication. "You can't survive without me" is not a great look for either employee or employer.

Perhaps they are cowboy developers in more ways than one.

Re:Banks scramble to fix old systems as IT 'cowboys' ride in... by Ensorceled on Thursday October 04, 2018 @05:31AM (#18138360)
Right. Those developers who were given days and months to document their work deliberately did nothing instead.

Wait, no, I've literally NEVER heard of management asking developers to delay the next release to document their work.

Re:Banks scramble to fix old systems as IT 'cowboys' ride in... by bitwize on Thursday October 04, 2018 @08:29AM (#18139254)
Couldn't they just find the time to document while they're developing?

Oh, but they can't increase the estimate. The customer signed off on that estimate, so we're committed to deliver by that deadline.

Re:Banks scramble to fix old systems as IT 'cowboys' ride in... by bigmanwalter on Thursday October 04, 2018 @09:41AM (#18139763)
Not really. Code needs to solidify before you document it. Documentation is best saved for after a major release.
Re:Banks scramble to fix old systems as IT 'cowboys' ride in... by Ensorceled on Thursday October 04, 2018 @10:11AM (#18140016)
Exactly. I'm always surprised when people think that it's easy to find time for documentation and that you only skipped that step because you are lazy and/or unprofessional.
Re:Banks scramble to fix old systems as IT 'cowboys' ride in... by repolfx on Friday October 05, 2018 @03:56AM (#18146457)
Honestly, I know that's the conventional wisdom, but I've never once heard an engineer go to their manager and say, "I'd like to spend some time documenting this important process you asked me to implement" and being told "no, we don't have time to document! just keep it all in your head!"

I'm sure this literally never happens.

What I do see quite frequently is developers who don't like writing documentation finding ways to not do it or justify doing a poor job.

Re:Banks scramble to fix old systems as IT 'cowboys' ride in... by arethuza on Thursday October 04, 2018 @12:13PM (#18141199)
Only time I've encountered a desperate fixation by a CIO on documenting applications was when the relevant development team was about to be laid off.

Fortunately I had already resigned at that point.

Re:Banks scramble to fix old systems as IT 'cowboys' ride in... by oldandtired on Thursday October 04, 2018 @07:18AM (#18138863)
You, obviously, have no idea how hard it is to write decent documentation for any system. Especially when it has been in existence for many years.

Good documentation is hard and anyone who thinks it is not does not have a clue as to what is needed for the purposes of maintaining any piece of software.

It is a very rare event to see documentation of a quality that explains all the interrelationships, reasons for the choice, etc, etc, etc.

I have in the past been required to provide all the documentation for projects I have worked on, functional and technical specifications, user guides and cheat sheets, detailed design documents and all of these take considerable time to write well. I can't say I am expert in this area because I am not. But I have done a lot better than most of the projects that I have had to interface to over the decades.

Documentation is a low priority for many management teams, they do not see the use in the kind of details that are needed. They do not give it sufficient support to do the job well.

I have seen and worked with people who have the skill to write the kind of documentation that is actually needed and they are a rare breed, well worth any money you pay them.

When companies complain about "cowboys", they often do not see that it is they who are the "cowboys".

Re:Banks scramble to fix old systems as IT 'cowboys' ride in... by repolfx on Thursday October 04, 2018 @10:36AM (#18140238)
You, obviously, have no idea how hard it is to write decent documentation for any system. Especially when it has been in existence for many years.

I've been programming for decades and my current project is frequently praised by other developers who work with it (it's a framework) for the quality of its documentation.

So, in fact, I do have such an idea. It's hard work. We do it anyway because we're professionals. The fact that people frequently don't do it doesn't change this.

One technique I use is to keep documentation in the same repository as the code, and force people to write docs via the code review process. That is, a proposed change doesn't pass review until it also updates the documentation files. People want to get their code merged, so they know they'll have to take docs seriously or else it'll get picked up in review.

Now maybe back in the mainframe days there was no culture of code review, but these are banks we're talking about. They excel at process. So it seems kind of unlikely they'd not want things documented, if management had understood what needed to be documented in the first place. But ultimately it was only the employees who knew that.

Re:Banks scramble to fix old systems as IT 'cowboys' ride in... by oldandtired on Friday October 05, 2018 @05:43AM (#18146841)
Your current project is praised for its documentation, that's actually good. However, if you are taken out by a bus today in some permanent fashion, how easily or quickly can someone come up to speed in supporting your project and not have to try an puzzle out why you did things the way you did? What assumptions are there that you have not documented? What relationship to other pieces of code or system functionality is there that you have not covered? What changes cannot be done because of the "brittleness" of interrelationships between different sections of the code? What are the expected business rules that are applicable to the software? What constraints do they put on others who want to use the software? What are the meanings of all the magic numbers that are littered throughout your code? Keep in mind all those other questions that they will have along the way.

It doesn't matter how "professional" you might be, you will not have covered all your bases. All you can do is spend lots of time in building up this documentation and if it is not being paid for in some way or time constraints get in the way, then none of it will get done properly.

It is good to see that you have some sort of process in place to manage the problem, but code reviews are limited in what they will capture.

I am documenting a piece of my own code at the moment for the purposes of submitting it to a specific project. It provides an additional functionality relating to something that the main system doesn't (at this point) handle. It is intended to be a simple workaround until the main system can supply this functionality at the VM and compiler level.

I have just looked at one method that simply converts a hexadecimal number to a potentially multi-byte sequence. The detailed reasoning for the code (the assumptions, choices of numeric values used, etc) amount to 10 times the number of lines that is actual code and I still don't think it is complete. I think that it will take at least two, three or maybe even four more revisions in the documentation to get it to the place where someone else could make changes without causing problems elsewhere. The code itself is really simple. But, there is an additional purpose to this documentation, in that, it will provide a baseline against which the final system changes can be compared.

I do this because, as a somewhat retired "professional" programmer, I want to see some serious documentation for this system and can afford to put in the time to do so.

So back to the main discussions, unless full and complete documentation is a requirement put in place and strongly enforced, what documentation there is will not be sufficient for the future. It is a rare professional who will instigate this and make sure it happens. From my experience, such professionals that do this, have not come from a programming background but have come from other industries (usually engineering based) where such documentation is considered normal or is mandatory.

I have seen few management teams that have given more than lip service to creation of the required documentation and the time frames they put on projects are such that there is little or no time for the production of quality documentation that is needed. Nor are they interested in paying for such.

Banks have a focus of making money. They are not interested in spending it. The only process orientation they have is increasing the bottom line in whatever way they can. If it means they cut back on programmer and computing costs, they will do this. Ultimately, it is the bank management that holds responsibility to understand that the documentation is required. If they did not understand this, they should not have been in the position of management. It is irrelevant if only the employees (those grunts at the coal-face) knew this.

Re:Banks scramble to fix old systems as IT 'cowboys' ride in... by vetinari on Thursday October 04, 2018 @05:32AM (#18138365)
If you want to blame someone, you can always say that he didn't document his stuff enough.

The problem is, that there is no standard, what's "enough", so it is easy to move the goal posts. Combine that with fast paced environment and putting out fires constantly, and you have a recipe for disaster.

Re:Banks scramble to fix old systems as IT 'cowboys' ride in... by lawlessone on Thursday October 04, 2018 @09:14AM (#18139556)
>If you want to blame someone, you can always say that he didn't document his stuff enough.

This, he could have documented it perfectly well in a time when everyone knew COBOL.

But that documentation would all be useless to me today with my knowledge of Java and Kotlin, but understandable to someone else from is generation that works with COBOL.

Re:Banks scramble to fix old systems as IT 'cowboys' ride in... by setr on Thursday October 04, 2018 @05:33AM (#18138369)
Tbf, it only really makes sense for the individual programmer to properly document code if there’s a decent chance they’ll forget (eg some obscure part of the codebase), or if you expect a reasonable chance that someone will have to read the code without you. Otherwise, its useful, but not necessary: you just go to the guru (perhaps yourself); his living body is the documentation.

If you’re thinking that only you’ll ever read it, or you’ll be working on this codebase for the next 30 years, it doesn’t really matter. Peoper Documentation is a cultural activity required for teams, especially those with shifting workers, as is common now. And should really be enforced by the managers, not the programmers thenselves.

Its significantly less necessary when you can expect to spend most of your career on a single codebase, a workstyle these cobol programmers presumably came from.

Re:Banks scramble to fix old systems as IT 'cowboys' ride in... by tcbawo on Thursday October 04, 2018 @08:17AM (#18139173)
I try to design and name interfaces clearly and comment code for myself. If it's been a few weeks since I've looked at something it helps me snap back. Also, I would never approach a codebase as if I will be working on it for 30 years. Job security comes from marketability, not obscurity.
Re:Banks scramble to fix old systems as IT 'cowboys' ride in... by lovich on Thursday October 04, 2018 @09:12AM (#18139546)
Was that true when these systems were first made? Many of them are decades old, and the culture of large institutions is usually resistant to change, and that comes from management on down. If the system didn't start with those values in mind, what would have prompted them to change?
Re:Banks scramble to fix old systems as IT 'cowboys' ride in... by tcbawo on Thursday October 04, 2018 @02:18PM (#18142247)
It's not that I don't expect the codebase to age or design it with long-term thinking, I just don't expect to be working on it for that long. Someone will replace me, and I'd like to leave them with well-designed, readable code that they will be able to understand.
Re:Banks scramble to fix old systems as IT 'cowboys' ride in... by lovich on Thursday October 04, 2018 @06:38PM (#18144298)
I'm talking about that as well. The modern era of jumping ship every 2-3 years on average in software is not how its always been. Jobs used to be very long term
Re:Banks scramble to fix old systems as IT 'cowboys' ride in... by tcbawo on Friday October 05, 2018 @07:26AM (#18147238)
It's not always about jumping ship either. In many companies, you will move between areas/needs or get promoted to other work after a couple years. Also, companies will outsource maintenance to lower-cost teams. It seems there is better project lifecycle management, less waterfall development, and fewer cushy open-ended government contracts. But, you're right. The expectation that you'll be able to do the same thing indefinitely for you career is gone.
Re:Banks scramble to fix old systems as IT 'cowboys' ride in... by magduf on Thursday October 04, 2018 @10:37AM (#18140256)
This is a terrible way to maintain a system. You never know when the guru is going to die or quit; you have to have a way of getting other people to maintain the system. This is why humans invented written language: to transmit information to each other and to subsequent generations.

However, writing useful documentation isn't trivial, and is best done with a technical writer, and takes time away from development. If management isn't willing to invest the time and resources necessary to produce useful documentation, then they're going to have problems in the future as the gurus leave.

Re:Banks scramble to fix old systems as IT 'cowboys' ride in... by TomMarius on Thursday October 04, 2018 @05:44AM (#18138414)
What happens when a bus hits the guru?
Re:Banks scramble to fix old systems as IT 'cowboys' ride in... by zoggenhoff on Thursday October 04, 2018 @06:54AM (#18138734)
Guru meditation error.
Re:Banks scramble to fix old systems as IT 'cowboys' ride in... by sekh60 on Thursday October 04, 2018 @09:12AM (#18139543)
Your DR planning better include a necromancer.
Re:Banks scramble to fix old systems as IT 'cowboys' ride in... by llcoolv on Thursday October 04, 2018 @06:49AM (#18138713)
yeah. this is how it had been done before the invention of writing. time has proven it not to be a very efficient method.
Re:Banks scramble to fix old systems as IT 'cowboys' ride in... by toast0 on Thursday October 04, 2018 @09:30AM (#18139676)
The only way to have really quality documentation is to have a lot of people reading it and providing feedback, over time.

It also doesn't hurt when a quality technical writer actually writes or edits the documentation in consultation with the people who designed or implemented the system.

Re:Banks scramble to fix old systems as IT 'cowboys' ride in... by sharemywin on Thursday October 04, 2018 @09:13AM (#18139552)
How many new devs are willing to even touch a COBOL program. documentation or not.
Re:Banks scramble to fix old systems as IT 'cowboys' ride in... by ams6110 on Thursday October 04, 2018 @09:50AM (#18139843)
For enough money, I'd say many. COBOL isn't sexy, but it's not difficult either. And it's demonstrated a lot more staying power than most other languages.
Re:Banks scramble to fix old systems as IT 'cowboys' ride in... by diggernet on Thursday October 04, 2018 @03:32PM (#18142933)
Darn right! I'd cheerfully learn Cobol for a job with the right parameters. I once took an RPG course in order to apply for one (though sadly did not get it).