News for hackers. Stuff that matters.
Re:With trillions at play, Pentagon fails audit for sixth ye... by MrMetlHed on Friday November 17, 2023 @12:31PM (#38306854)
Worked with a guy ten years ago that would obsessively read these reports from the Pentagon. Here's a couple of his stories about the matter: https://www.reuters.com/investigates/pentagon/ and https://www.reuters.com/investigates/special-report/usa-mari... (very old and I apologize for my horrific javascript at the time.)

It sounds like nothing has changed. Relevant to this crowd, maybe: "Most of the Cobol code the Pentagon uses for payroll and accounting was written in the 1960s, according to 2006 congressional testimony by Zack Gaddy, director of DFAS from May 2004 to September 2008."

"Wallace, the Army assistant deputy chief of staff, says the system has "seven million lines of Cobol code that hasn't been updated" in more than a dozen years, and significant parts of the code have been "corrupted." The older it gets, the harder it is to maintain. As DFAS itself said: "As time passes, the pool of Cobol expertise dwindles.""

Re:With trillions at play, Pentagon fails audit for sixth ye... by bogota on Friday November 17, 2023 @12:35PM (#38306912)
Start paying 250k a year for cobol developers and the problem fixes itself really fast. That or start migrating the system. Or more likely they keep using it until it completely breaks and then run to the government for a big handout to fix it.

Easy to be irresponsible with other peoples money.

Re:With trillions at play, Pentagon fails audit for sixth ye... by nickpeterson on Friday November 17, 2023 @12:38PM (#38306975)
Part of the problem is, you tend to need cobol experts to actually have any hope of successfully transitioning to another technology, and if we had enough cobol experts would look at the cost of migrating and say, “who needs to, we have a bunch of cobol developers?”
Re:With trillions at play, Pentagon fails audit for sixth ye... by adamc on Friday November 17, 2023 @01:04PM (#38307409)
Hire experts and have it be reported as "waste" and hear complaints in Congress about government waste.

We want government to be cheap. You get what you pay for.

Re:With trillions at play, Pentagon fails audit for sixth ye... by lancepioch on Friday November 17, 2023 @12:37PM (#38306955)
Don't they have caps and levels for federal employee salaries? I agree that raising the salary would decrease the issues. I think that they'd need exceptions to account for these increases though.
Re:With trillions at play, Pentagon fails audit for sixth ye... by Jtsummers on Friday November 17, 2023 @12:38PM (#38306987)
Yes, but not for contractors. Most software sustainment ultimately ends up in the hands of contractors with government supervision in the form of program offices. Not all, though, many programs are also "organic", primarily or fully staffed by government employees and managed by government employees.
Re:With trillions at play, Pentagon fails audit for sixth ye... by randmeerkat on Friday November 17, 2023 @01:25PM (#38307712)
> Yes, but not for contractors.

Sure, but it’s not like the consulting firms are paying their “contractors” more, they just siphon up the difference for their shareholders.

Re:With trillions at play, Pentagon fails audit for sixth ye... by ska on Friday November 17, 2023 @01:33PM (#38307853)
That's not really how it works. They are more than happy to have "market forces" drive up the hourly rates, so long as they get to keep their overhead % fixed, and that's probably how the contract is structured. Win-win (and perhaps -lose for taxpayers).
Re:With trillions at play, Pentagon fails audit for sixth ye... by randmeerkat on Friday November 17, 2023 @02:51PM (#38309074)
> That's not really how it works. They are more than happy to have "market forces" drive up the hourly rates

That’s exactly how it works, look at contract government salaries compared to anything in the private sector. They charge the government more as rates “go up”, but that certainly isn’t passed along. If large contracting companies really offered value to the government _and_ kept up with market rates for their employees, the state of federal software wouldn’t be what it is today.

Re:With trillions at play, Pentagon fails audit for sixth ye... by ska on Friday November 17, 2023 @03:02PM (#38309215)
I didn't say that your rate at a govt focussed consultancy would be identical to your rate elsewhere. I probably should have been more explicit.

I rejected the idea that the consultancy would get a rate increase based on "market rates being higher" and then just capture it all - in my (admittedly somewhat limited and path dependent) experience that just isn't how it works. It's more like we pay randmeerkat $X, we bill them out at $X * factor + overhead. "Market forces" mean we have to go Y > X in renewal or we will lose randmeerkat & friends, so now they get $Y and we bill $Y * factor + overhead. It's of course usually more complicated in general, and overhead especially likely isn't that simple.

Nowhere in there is the assertion that X or Y is what randmeerkat would get on the open market. Importantly, their market isn't really "programmers", but "programmers that work in govt + contracting halo". Which is part of why the idea: I could get $N more in SF tech may be compelling for you, but isn't compelling for them (unless too many people actually make that change, instead of just talk about it).

Also there are many other ways they can raise there rates, but if the claim is that it is due to market on the developer salaries that are going to be a line item, there is going to be at least first pass look at a) is that true (find some market data and wave hands) b) did it actually get spent that way (may come up in an audit).

So the real answer seems to be not "They get more and I get nothing" but "I get a bit more, and they get more scaled by what I get", i.e. "win-win".

There is lots wrong with the system of contracting, but I don't think criticizing a cartoony straw-man of it gets anywhere useful.

Re:With trillions at play, Pentagon fails audit for sixth ye... by toasted-subs on Friday November 17, 2023 @12:43PM (#38307074)
Starting pay for cobol in finance was around 350k last time I glanced. Nobody wants to rewrite a working system which will likely be much more expensive than finding one person to maintain the project.
Re:With trillions at play, Pentagon fails audit for sixth ye... by saiya-jin on Friday November 17, 2023 @01:27PM (#38307752)
With rewrite comes uncertainty and risk too, how many new bugs compared to code that works for decades?
Re:With trillions at play, Pentagon fails audit for sixth ye... by MrMetlHed on Friday November 17, 2023 @12:43PM (#38307077)
In the last part of the series from 2013 we have a chart that shows the Pentagon had spent over $10b modernizing their accounting systems (that was through 2013, no idea how much that number has grown.) I can't imagine anyone would miss a few million of that going to COBOL developers. The money scale for Pentagon projects is mind-boggling to me.
Re:With trillions at play, Pentagon fails audit for sixth ye... by thelastgallon on Friday November 17, 2023 @01:15PM (#38307570)
Incentives only work on low-skilled labor that is quantifiable. Eg. more money for flipping more burgers. It is not a settled science paying people a lot more will automagically solve problems. People crave Autonomy, Mastery & Purpose: https://www.youtube.com/watch?v=rbR2V1UeB_A

What is definitely going to happen is you will attract the cowboys who have convinced themselves and can convince you to pay top dollar. They will come up with lets do kubernetes + service-mesh + multi-cloud hybrid + blockchain + crypto + AI/MI on this COBOL ... They will take top dollar, make a bigger pile of mess, sell it as a success story and move on make bigger messes.

Re:With trillions at play, Pentagon fails audit for sixth ye... by whatshisface on Friday November 17, 2023 @01:17PM (#38307592)
250k is not top dollar, it is average for experienced engineers.
Re:With trillions at play, Pentagon fails audit for sixth ye... by ramilefu on Friday November 17, 2023 @01:31PM (#38307824)
Depends on where you live. An “experienced engineer” is going to make a lot less if they live in Mississippi than if they were in California. 250k for the former is likely “top dollar”.
Re:With trillions at play, Pentagon fails audit for sixth ye... by 1323portloo on Friday November 17, 2023 @02:33PM (#38308894)
If you are a small shop, sure. If you are a major employer you will either meet market rate and get top candidates (willing to live in your area) or you will get whatever candidates decided to live in Mississippi.
Re:With trillions at play, Pentagon fails audit for sixth ye... by bluefirebrand on Friday November 17, 2023 @02:39PM (#38308977)
People on HN are convinced if you don't live in California maximizing your salary, you're not capable of doing so.

Highlighted in the sibling comment: "you will get whatever candidates decided to live in Mississippi"

As if top talent never decides to live in Mississippi and accept 250k/year as their salary ceiling...

Re:With trillions at play, Pentagon fails audit for sixth ye... by 1323portloo on Friday November 17, 2023 @03:01PM (#38309197)
I didn't say that top talent doesn't live in Mississippi, I just said you will get what candidates DO decide to live in Mississippi. If you want top talent, you have to advertise nationally, and be willing to pay the national rate. I live in a city that is quite poor for an employer that is quite large. They pay slightly better than average for the area except for their engineers and upper management, that goes to market rate because they are willing to hire the best available.
Re:With trillions at play, Pentagon fails audit for sixth ye... by johnnyanmac on Saturday November 18, 2023 @02:21AM (#38316491)
People also seem to forget that there are "cost of living adjustments" for a reason. Many companies will fight and suggest that you don't need salary X if you don't live in place Y.

It's not impossible and $250k is nowhere near the ceiling. But you're not going to find those jobs on a causal LinkedIn search like you would in California.

Re:With trillions at play, Pentagon fails audit for sixth ye... by Miner49er on Friday November 17, 2023 @01:40PM (#38307972)
No, that is not average, maybe in high COL areas, but that obviously would then not be average.
Re:With trillions at play, Pentagon fails audit for sixth ye... by Kranar on Friday November 17, 2023 @01:30PM (#38307797)
250k is not top dollar for programmers, but it is top dollar in absolute terms. The point is that you don't incentivize better work by paying top dollar. Taking a mediocre developer and paying them more doesn't make them a better developer or even motivate them to become better.

It can attract people who are already good software developers absolutely, but the second point being made is that you also attract a lot of charlatans as well and so just throwing more money at the problem isn't exactly a great solution.

You need a solid culture to go with the money.

Re:With trillions at play, Pentagon fails audit for sixth ye... by rumdz on Friday November 17, 2023 @01:33PM (#38307848)
Outside of CA?
Re:With trillions at play, Pentagon fails audit for sixth ye... by metabagel on Friday November 17, 2023 @01:47PM (#38308077)
Even in Southern California, $250K is a very high salary for developers, at least in embedded systems, which is the space I’m familiar with.
Re:With trillions at play, Pentagon fails audit for sixth ye... by winrid on Friday November 17, 2023 @06:34PM (#38312264)
embedded system engineers have always had low salaries... I don't think I've ever seen an embedded C or C++ job pay more than $130k. I made more than that writing JS almost ten years ago..
Re:With trillions at play, Pentagon fails audit for sixth ye... by metabagel on Friday November 17, 2023 @01:45PM (#38308056)
Outside the Bay Area, $250K is top dollar.
Re:With trillions at play, Pentagon fails audit for sixth ye... by cebert on Friday November 17, 2023 @07:28PM (#38312970)
Not really, even in the Midwest, such as the Metro Detroit region, 250k is definitely attainable.
Re:With trillions at play, Pentagon fails audit for sixth ye... by myko on Friday November 17, 2023 @10:04PM (#38314492)
Base or with bonuses?

I find it hard to get roles in the midwest (large metro in Ohio) clearing 200k, though sometimes bonuses get you up there.

Re:With trillions at play, Pentagon fails audit for sixth ye... by cebert on Saturday November 18, 2023 @12:06PM (#38321708)
TC, my base isn’t that high.
Re:With trillions at play, Pentagon fails audit for sixth ye... by omginternets on Friday November 17, 2023 @08:39PM (#38313779)
It definitely is not.
Re:With trillions at play, Pentagon fails audit for sixth ye... by jongjong on Friday November 17, 2023 @09:16PM (#38314136)
WTF. I worked on some high exposure projects as a senior dev and never a final offer above $130k USD. Even after haggling hard at a company I had been at for 3 years and they told me I was their top, highest paid engineer and it led me to quit. Then I got approached by Facebook to work in London HQ and the offer they were dangling was £100GBP which was about $130k USD. Highest offer I ever got before I even got to the final round of interviews was $160k USD from a well known crypto company but for some reason the head of engineering didn't show up to the final interview at the last minute after I flew through all the tech tests as those devs who interviewed admitted. It was weird.

I always assumed the reported 250k plus salaries were fake news or only reserved for children of powerful people.

Re:With trillions at play, Pentagon fails audit for sixth ye... by sneed_chucker on Friday November 17, 2023 @11:32PM (#38315215)
Why would you assume that? Look up levels.fyi

$250k total comp isn't even particularly high at one of the big companies.

Re:With trillions at play, Pentagon fails audit for sixth ye... by johnnyanmac on Saturday November 18, 2023 @02:18AM (#38316472)
gotta keep in mind that this sounds like pre-COVID times, and even big companies weren't going to pay San Fran Salaries to low CoL areas.

It definitely isn't easy to find those numbers if your CoL doesn't require them. You'd need to move to those areas.

Re:With trillions at play, Pentagon fails audit for sixth ye... by datavirtue on Saturday November 18, 2023 @07:17AM (#38318679)
I just quit a $180k job for a $125k job. I did it to join a team that I want to work with, and people that I can trust. I used to waddle out of bed around 9am and quit at 2-3pm after hours or boredom. Now I jump out of bed at 5am and don't quit until 4 pm--sometimes I spend all day traveling. I about cry at night because I love my job so much.
Re:With trillions at play, Pentagon fails audit for sixth ye... by omginternets on Saturday November 18, 2023 @10:00PM (#38328194)
To be fair, I think the 250k mark usually implies some kind of specialization.
Re:With trillions at play, Pentagon fails audit for sixth ye... by rybosworld on Friday November 17, 2023 @02:34PM (#38308901)
This is a lie that is commonly spread here and on reddit.
Re:With trillions at play, Pentagon fails audit for sixth ye... by bogota on Friday November 17, 2023 @11:53PM (#38315418)
250k with government perks and retirement is pretty nice. Also for 250k cash you will find lots of people willing to learn cobal.
Re:With trillions at play, Pentagon fails audit for sixth ye... by VoodooJuJu on Friday November 17, 2023 @02:28PM (#38308787)
>People crave Autonomy, Mastery & Purpose

I'm not a sack of chemicals whose desires can be reduced down to some pithy pseudo-scientific Maslowian quip. The only thing I crave from my job is cash so I can take care of me and my family. 250k/yr for a COBOL position would suit that end just fine. You can keep your autonomy, mastery, purpose, and all the other foofoo, just hand me the check so I can be on my way.

Re:With trillions at play, Pentagon fails audit for sixth ye... by wolverine876 on Friday November 17, 2023 @05:55PM (#38311775)
> I'm not a sack of chemicals whose desires can be reduced down to some pithy pseudo-scientific Maslowian quip. The only thing I crave from my job is cash

I thought you were against reductionism?

Re:With trillions at play, Pentagon fails audit for sixth ye... by VoodooJuJu on Friday November 17, 2023 @06:11PM (#38311959)
Re:With trillions at play, Pentagon fails audit for sixth ye... by Cyphase on Friday November 17, 2023 @10:41PM (#38314811)
They were replying to someone who was saying something about "people"; they only said something about "I" (themselves).
Re:With trillions at play, Pentagon fails audit for sixth ye... by johnnyanmac on Saturday November 18, 2023 @02:23AM (#38316506)
>The only thing I crave from my job is cash so I can take care of me and my family

sounds like purpose, of which you need mastery of some skillset to earn your autonomy to furfill.

That said, I'm sure there's much easier ways to earn 250k than untangling a governement codebase. Not that they are offering that to begin with.

>just hand me the check so I can be on my way.

Yeah, you probably wouldn't pass clearance with that attitude if we're being frank.

Re:With trillions at play, Pentagon fails audit for sixth ye... by curiousllama on Friday November 17, 2023 @01:27PM (#38307751)
Cobol cowboys are a thing. They make substantially more than 250k. They’re competing with Wall St though, so hard to just pay more.

Fed govt migrations are ALWAYS shitshows. The army alone has like 20 individual email services. The pentagon is to org complexity what big tech is to technical complexity. It’s turtles all the way down.

This is literally one of the largest bureaucratic challenges on earth. There’s no simple fixes

Re:With trillions at play, Pentagon fails audit for sixth ye... by mschuster91 on Friday November 17, 2023 @02:08PM (#38308417)
The solution is simple: you need someone with absolute authority over everyone including the generals to lead the effort.

There's a time and place for democracy - but large IT projects are not. Do a thorough need analysis, compare with what's reasonably possible using COTS software, and adapt or discard what is not possible.

Re:With trillions at play, Pentagon fails audit for sixth ye... by count on Friday November 17, 2023 @02:39PM (#38308980)
I can't tell if you're being sarcastic, or earnest.

This is the funniest possible response. The 'thorough need analysis' is the reason we are where we are now!

The thoroughness is so thorough that the needs change faster than the analysis can be completed, obviating the analysis in the first place.

Re:With trillions at play, Pentagon fails audit for sixth ye... by vajrabum on Friday November 17, 2023 @11:53PM (#38315420)
That's exactly why DoD kept bringing Grace Hopper out of retirement first in 1967. They eventually gave her a star in the 80s, and along the way she was as responsible as anyone for the standardization efforts for COBOL and Fortran including the design of standardized compiler validation suites. Testing and interoperability before testing or interoperability were a thing! One of the later major projects she worked on was modernizing the Navy payroll system. Even junior sailors at the time knew that she and her team had made sure they got their paychecks. She retired from the Navy at age 79 in 1986. Nobody messed with Grace Hopper.
Re:With trillions at play, Pentagon fails audit for sixth ye... by neurotech1 on Saturday November 18, 2023 @01:26AM (#38316159)
Adm. Grace Hopper wasn't the only senior officer who was recalled because of their expertise. One is a retired JAG lawyer (a friend actually) who continues to advise at the Pentagon on several areas of expertise. She actually learned COBOL from Adm. Hopper herself, and always had high amount of respect for Adm. Hopper and how she disliked bureaucracy. There is a lesson there.

Ironically, my friend has programmed COBOL on DoD mainframes, and still does as a consultant, having interacted with them for decades. Back in the 1970s she had a surplus IBM minicomputer that her father brought home for business use, and learned COBOL on that, and used the computer for furthering her legal research.

Re:With trillions at play, Pentagon fails audit for sixth ye... by ska on Friday November 17, 2023 @01:30PM (#38307812)
> Start paying 250k a year for cobol developers and the problem fixes itself really fast.

Do you think they aren't? I don't know any now, but the last one I did was making more than that 15 years ago.

Old crufty legacy systems in a big bureaucracy are many peoples idea of hell. Empirically speaking, good pay and job security aren't enough to have people knocking down your door for that kind of work if they have other decent options.

Re:With trillions at play, Pentagon fails audit for sixth ye... by wubrr on Friday November 17, 2023 @05:37PM (#38311512)
That could work if the existing code is actually up-to-date, correct, and if you have knowledgeable users of the systems backed by this code. The part about 'code corruption' makes me thing this is not the case.

Worked at a big bank migrating some old code to new stack one time:

  - Original developers are gone.
  - Existing users don't understand the systems outside of their own narrow use-cases.
  - Code which was committed to repositories didn't match the actual running code, because people would go into production servers, manually edit the code there, in place, not document or commit the changes to repo. There were like 20 years of adhoc changes made in this way.
We had some business analysts which were supposed to talk to the users and gather requirements - they very quickly simply gave up and asked the developers to 'reverse engineer' everything.

Anyway, it was a hilarious shit-show, but kinda fun at the same time (if you're able to ignore incompetent leadership pressure - a skill I mastered during this time).

Re:With trillions at play, Pentagon fails audit for sixth ye... by obviouslynotme on Friday November 17, 2023 @06:54PM (#38312509)
>Code which was committed to repositories didn't match the actual running code

I remember the first time I saw this at a small company. The head programmer was the only one with the actual production source, despite a working source control system. He complained about how people didn't appreciate how hard it is to make builds work. I should have left when we lost three man months of code to losing email. I only left after screaming at the head programmer for refusing to stop making the repository unbuildable at all.

Re:With trillions at play, Pentagon fails audit for sixth ye... by ravenstine on Friday November 17, 2023 @11:23PM (#38315136)
This is a problem not just in software. Today, everyone wants a senior <insert job role> for the price of a junior <insert job role>. So many problems would be solved orders of magnitude more soon if people were properly compensated. Instead, most things in the United States are running on economic fumes.
Re:With trillions at play, Pentagon fails audit for sixth ye... by uticus on Friday November 17, 2023 @12:36PM (#38306942)
Of course, devil's advocate would point out COBOL in 60s was written with a lot more focus on lasting in general. They didn't largely have the same general attitude of 'we iterate quickly with our script kiddies' as is prevalent today.

Not that all software from then was wonderful. Or that COBOL magically guarantees great code.

But, on the flip side, you can't say the code base isn't battle-hardened (to various literal degrees).

* edit: corrected sp

Re:With trillions at play, Pentagon fails audit for sixth ye... by SteveNuts on Friday November 17, 2023 @01:17PM (#38307594)
It'd be battle hardened to 1960s standards, payroll has changed a _lot_ since then. And if they can't accurately describe exactly how certain calculations are made (or have to do some of those manually outside of the program after it runs), that would be a huge problem in an audit.

This is why big companies use Oracle or SAP for their ERP, auditors are very familiar with it and the way it functions is well-known and understood.

Re:With trillions at play, Pentagon fails audit for sixth ye... by monocasa on Friday November 17, 2023 @01:28PM (#38307776)
Also throwing out there that COBOL was a language designed by the DOD specifically for managing it's bureaucracy.

It was a naval research project headed by Admiral Grace Hopper to program with words rather than formulas like earlier "high-level" "autocoders", enabling secretaries to transfer their business logic into computers. It actually intentionally has a lot of the same structure as a recipe, hoping that this would allow easier uptake by secretaries.

Re:With trillions at play, Pentagon fails audit for sixth ye... by ebiester on Friday November 17, 2023 @01:33PM (#38307851)
I can build you a team using today's (quality: average B2B startup) programmers and today's methodologies that will write software for 40 years.

It will just cost twice as much as the team that iterates quickly. It will cost more because I will spend more money on business analysis up front. I will spend more money writing automated tests. Most importantly, I will maintain my own toolchain and libraries or pay someone to do so. Every piece of software deployed will have someone accountable to my business to maintain it.

It turns out that this is expensive. It's no more expensive than maintaining the COBOL code, mind you, and would likely give you better results than the equivalent COBOL code. Most companies will take their chances with less expensive software because that's what their competitors do.

About a decade ago, I saw one of the major systems you talk about up close. While they spent that amount of money for mission-critical systems, they didn't do nearly as good of a job on the average software in the system. I'm not convinced that any business is ready to go back to the battle hardened days.

Re:With trillions at play, Pentagon fails audit for sixth ye... by docandrew on Friday November 17, 2023 @02:11PM (#38308479)
I wonder though whether the up-front costs incurred by your team would eventually be outweighed by the long-term costs of the quick team.
Re:With trillions at play, Pentagon fails audit for sixth ye... by ebiester on Friday November 17, 2023 @03:19PM (#38309462)
I am not convinced that it would. The quick team will get to learn three times as much and adapt to the actual customer need. The cost of change in a system built with stability as a quality attribute will be higher. (As you can see from a system like the old mainframes!)
Re:With trillions at play, Pentagon fails audit for sixth ye... by giantg2 on Friday November 17, 2023 @08:16PM (#38313545)
Mainframes are expensive because they're niche and basically a duopoly. The companies supplying them know their days are numbered and charge high prices before the market goes dry.

In my experience, the quick teams only learn the surface content. A lack of depth tends to result in more errors down the road and poor architecture. But I guess it doesn't matter since half your team will jump to the next project company in 2-3 years anyways and take that knowledge with them.

Re:With trillions at play, Pentagon fails audit for sixth ye... by Spooky23 on Friday November 17, 2023 @11:49PM (#38315379)
The advantage that the original cobol people had is that they had functional manual processes to model.

Today, the payroll supervisor has no idea how many processes work.

Re:With trillions at play, Pentagon fails audit for sixth ye... by dustingetz on Friday November 17, 2023 @02:13PM (#38308500)
ha! you wouldn’t even be permitted to bid the project!
Re:With trillions at play, Pentagon fails audit for sixth ye... by quickthrower2 on Friday November 17, 2023 @02:27PM (#38308772)
The trick is to use JS. (Not anything that compiles to it)

It will be guaranteed to run in 100 years time.

Any other language can die or have incompatibility issues. Over that timespan.

Re:With trillions at play, Pentagon fails audit for sixth ye... by ebiester on Friday November 17, 2023 @03:10PM (#38309320)
You could use Java or Python and have the same guarantee.

I am not convinced Node will be here in 100 years.

Re:With trillions at play, Pentagon fails audit for sixth ye... by ak39 on Sunday November 19, 2023 @02:47AM (#38330202)
Node might not, but Javascript (even on the server) will mostly be.*

* assuming that the 100 years hyperbole is accepted

Re:With trillions at play, Pentagon fails audit for sixth ye... by klyrs on Friday November 17, 2023 @03:26PM (#38309571)
Brainfuck is the only language we need. It's feature complete and practically guaranteed to go unchanged in the next millenium.
Re:With trillions at play, Pentagon fails audit for sixth ye... by kube-system on Friday November 17, 2023 @04:06PM (#38310321)
I can't tell if I'm being Poe's Law'd. JS engines are a moving target and have been ever since it was called LiveScript.
Re:With trillions at play, Pentagon fails audit for sixth ye... by satvikpendem on Saturday November 18, 2023 @02:04AM (#38316385)
Just use LISP!
Re:With trillions at play, Pentagon fails audit for sixth ye... by satvikpendem on Saturday November 18, 2023 @02:04AM (#38316383)
> It will be guaranteed to run in 100 years time.

This is never guaranteed. How do you know that nothing will change in JS land in 10 years, much less 100? You don't, that's how.

Re:With trillions at play, Pentagon fails audit for sixth ye... by trealira on Saturday November 18, 2023 @10:54AM (#38320866)
Programming languages haven't even been around for 100 years.

There's no guarantee we'll even be running programs written in C in 2123, let alone ones written in JavaScript. 100 years from now is a long time.

Re:With trillions at play, Pentagon fails audit for sixth ye... by satvikpendem on Saturday November 18, 2023 @02:03AM (#38316373)
> using today's (quality: average B2B startup) programmers and today's methodologies that will write software for 40 years.

I'm not sure I would trust these people over those of the past, regardless of what they say they will accomplish, Chesterton's Fence, and the Lindy Effect, and all.

Re:With trillions at play, Pentagon fails audit for sixth ye... by johnnyanmac on Saturday November 18, 2023 @02:09AM (#38316412)
I imagine that is exactly what:

>spend more money on business analysis up front.

Entails. But sure, if the government doesn't want to pay for that quality, they can spend just as much or more resurrecting Cobol devs in cryo-stasis in 2099

Re:With trillions at play, Pentagon fails audit for sixth ye... by jrockway on Saturday November 18, 2023 @02:36AM (#38316588)
I don't think they have to keep COBOL devs in cryostasis. Some sufficiently motivated individual could likely learn it today. There are even programmers alive right now that would see a large legacy system and not immediately suggest a rewrite on their first day. It's rare but it can happen!
Re:With trillions at play, Pentagon fails audit for sixth ye... by bastawhiz on Friday November 17, 2023 @02:05PM (#38308372)
> you can't say the code base isn't battle-hardened

Your statement is undermined by the fact that the system has failed its sixth consecutive audit. If it's not hardened against failing an audit, I'm not really sure what it could possibly be hardened against.

Re:With trillions at play, Pentagon fails audit for sixth ye... by bumby on Friday November 17, 2023 @02:26PM (#38308739)
Part of Murphy's Laws of Combat: No combat ready unit ever passed inspection. No inspection ready unit ever passed combat.

Managers need to be careful of Goodhart's law. (in this case "passing inspection" may not be the best target)

Re:With trillions at play, Pentagon fails audit for sixth ye... by bastawhiz on Friday November 17, 2023 @02:36PM (#38308934)
And yet hundreds of thousands of companies pass the same audits. The call for audits came from ridiculous amounts of misreporting and waste. Accounting systems that can't do accounting and deal with trillions of taxpayer dollars should not have the goalposts moved. A 0.001% error in a trillion dollar system is ten million dollars. That's taxpayer money that could fix derelict bridges or pay for thousands of children to receive lunch at school.
Re:With trillions at play, Pentagon fails audit for sixth ye... by bumby on Friday November 17, 2023 @08:51PM (#38313912)
I agree with you, but I’d venture to guess most of those companies would use “combat” in that sense as an analogy.

The DoD does not.

Re:With trillions at play, Pentagon fails audit for sixth ye... by herewulf on Saturday November 18, 2023 @02:23AM (#38316503)
Yet the DoD's bureaucracy is neither combat nor inspection ready.
Re:With trillions at play, Pentagon fails audit for sixth ye... by bumby on Saturday November 18, 2023 @08:02AM (#38319050)
What makes you justify the latter? That’s a bold claim considering the current generation have more combat experience than practically any other generation. America has been in continued conflict for the entire lives since of many. I don’t think that’s a good thing, but it’s silly to claim they aren’t prepared for combat.
Re:With trillions at play, Pentagon fails audit for sixth ye... by bumby on Saturday November 18, 2023 @08:54AM (#38319616)
I tend to think the combat portion is a matter of scope. It’s hard to do everything, everywhere, all at once regardless if that’s on a politicians wishlist. The only force noted as “strong” by the Heritage Foundation readiness index is the Marine Corps and that’s a result of acknowledging they are a one-war organization of limited scope. That all goes to say the scope of the DoD is unlike any other organization on earth, so quips like “well, companies pass these inspections all the time!” is of little value.
Re:With trillions at play, Pentagon fails audit for sixth ye... by wmf on Friday November 17, 2023 @06:19PM (#38312053)
COBOL in 60s was written with a lot more focus on lasting in general

This is the same code that uses two-digit years and doesn't support lowercase letters?

Re:With trillions at play, Pentagon fails audit for sixth ye... by cwbriscoe on Friday November 17, 2023 @07:09PM (#38312696)
COBOL ONLY USES 2 CHARACTER YEARS IF THAT IS THE WAY IT WAS CODED. PEOPLE DIDN'T THINK THEIR 60'S - 90'S CODE WOULD STILL BE RUNNING IN 2000. COBOL DOES SUPPORT lowercase LETTERS. GET WITH THE TIMES!
Re:With trillions at play, Pentagon fails audit for sixth ye... by trealira on Saturday November 18, 2023 @11:01AM (#38320935)
> PEOPLE DIDN'T THINK THEIR 60'S - 90'S CODE WOULD STILL BE RUNNING IN 2000.

That serves as evidence against this point, though:

> COBOL in 60s was written with a lot more focus on lasting in general.

Re:With trillions at play, Pentagon fails audit for sixth ye... by cwbriscoe on Saturday November 18, 2023 @01:44PM (#38322929)
Yeah, that point is wrong in my opinion. I don't think programmers back then tried to build things to last any more than programmers today do.
Re:With trillions at play, Pentagon fails audit for sixth ye... by wredue on Friday November 17, 2023 @06:28PM (#38312182)
>Of course, devil's advocate would point out COBOL in 60s was written with a lot more focus on lasting in general. They didn't largely have the same general attitude of 'we iterate quickly with our script kiddies' as is prevalent today.

I worked on enterprise cobol for nearly 20 years and can generally attest that cobol code from the 60s is absolutely nothing at all what you’re describing.

It’s brittle and constantly breaks, requiring 24/7 on call support for people to deal with issues so it doesn’t hold up the next days work.

It was not written with any semblance of engineering prowess. Most cobol written in the 60s was written by business managers, not programmers.

The reason it stays running isn’t because it’s battle hardened. It’s because nobody in their right mind would ever pay to untangle the unfettered mess.

Re:With trillions at play, Pentagon fails audit for sixth ye... by giantg2 on Friday November 17, 2023 @08:11PM (#38313494)
Even when I started 12 years ago in Java/JSF/Oracle stack there seemed to be way more focus on durability and longevity. The system I started on was architected in a way that made it easy to modify and integrate with other system. It worked well. Eventually it's scope creeped, which would necessitate modernization. But it was very maintainable and lasted years.

Now the systems I work on are hot garbage because the tech is constantly needing upgraded versions, changes to the code are onerous because of the way the system was designed, we are constantly having data issues for various reasons, and people jump teams every 2 years so there's no continuity.

I'm convinced it's a leadership issue. Push for speed and you'll get it. It will cost you in other areas.

Re:With trillions at play, Pentagon fails audit for sixth ye... by psunavy03 on Friday November 17, 2023 @10:17PM (#38314600)
Having worked on a team responsible for maintaining COBOL code, if from the 80s and not the 60s, it may "last," but it's also a rats nest of dependencies that's a pain in the ass to test, and part of the problem of moving away from it is 40 years of undocumented business logic baked into it over the years.
Re:With trillions at play, Pentagon fails audit for sixth ye... by Spooky23 on Friday November 17, 2023 @11:47PM (#38315357)
If my experience with an unemployment system is a guide, the COBOL is fine. The scaffolding around it, different story.
Re:With trillions at play, Pentagon fails audit for sixth ye... by jrockway on Saturday November 18, 2023 @02:34AM (#38316572)
This just sounds like survivorship bias to me. I am sure there is a lot of junk COBOL that killed the companies that relied on it. Nobody cares today because the business is gone.

Something that's interesting to think about is that because the cost of changing COBOL is so high now, only the most important change requests actually get implemented. Compare that to a team that can develop and ship a change the same day; their efficiency leads to more requests that end up being bad ideas. The Pentagon has designed a system to ensure that only the most well-thought-out ideas make it into production, they just went about it in a very strange way.

Re:With trillions at play, Pentagon fails audit for sixth ye... by encoderer on Friday November 17, 2023 @12:36PM (#38306945)
I can imagine a future where new software is, mainly, written by humans while legacy software is maintained by ai.

It can take months for a new developer to understand a legacy code base but an LLM with a big enough context window would be instantly productive.

Re:With trillions at play, Pentagon fails audit for sixth ye... by Yahivin on Friday November 17, 2023 @12:40PM (#38307013)
I can imagine the opposite.
Re:With trillions at play, Pentagon fails audit for sixth ye... by encoderer on Friday November 17, 2023 @12:41PM (#38307034)
Cool tell me more.
Re:With trillions at play, Pentagon fails audit for sixth ye... by SteveNuts on Friday November 17, 2023 @01:19PM (#38307614)
Not OP but I could see some shops pushing AI generated code to production, then when changes need to be made, they can't get the AI to modify the existing code in just the way they need, so a human has to intervene.
Re:With trillions at play, Pentagon fails audit for sixth ye... by bastawhiz on Friday November 17, 2023 @02:07PM (#38308399)
I can't get Copilot to generate Python that adds numbers together correctly sometimes. Getting an LLM to generate correct, working code for a language that hardly anybody writes anymore is almost assuredly going to lead to failure.
Re:With trillions at play, Pentagon fails audit for sixth ye... by encoderer on Friday November 17, 2023 @03:02PM (#38309213)
yeah I agree but when you look at the slope not the y-intercept it’s getting obviously better.

one advantage the government would have is training/fine-tuning on a hundred million lines of domain specific cobol.

Re:With trillions at play, Pentagon fails audit for sixth ye... by bastawhiz on Friday November 17, 2023 @05:30PM (#38311417)
The slope doesn't really matter, because the target is "better than a human, and able to identify and fix its own errors". The slope will decrease as you approach this threshold.

It's also wildly bad to plan to train and fine tune on code that you know has bugs. Already we have Copilot generating code with trivial vulnerabilities because that's what it's trained on.

Re:With trillions at play, Pentagon fails audit for sixth ye... by encoderer on Friday November 17, 2023 @10:52PM (#38314898)
Roughly ~all code has bugs. We were all trained on trivial examples and buggy code - just like llms.

Honestly I think editing code will be easier than creating wholly new applications to precise spec.

Re:With trillions at play, Pentagon fails audit for sixth ye... by nradov on Friday November 17, 2023 @12:50PM (#38307187)
The hard part of accounting systems is doing the requirements analysis (including interviewing humans who give inaccurate or incomplete answers) and writing detailed functional specifications which account for every possible edge case. No LLM can do that.

Once you have the detailed specifications the coding is relatively easy. Some of that can be partially automated using AI CASE tools, but that only gives a marginal improvement to overall project productivity.

Re:With trillions at play, Pentagon fails audit for sixth ye... by charlie0 on Friday November 17, 2023 @01:35PM (#38307878)
Ding, ding, ding, we have a winner. This. I see issues at my small startup with not having clarity with certain billing items. This issue is not technical, it's the unknown requirements of why things were written in the way they current exist. I can't imagine what kind of bizatine rules they have in the Pentagon that were placed there decades ago, and no one can answer why. Codebase must be littered with Chesterton fences.
Re:With trillions at play, Pentagon fails audit for sixth ye... by dragonwriter on Friday November 17, 2023 @01:06PM (#38307445)
> The hard part of accounting systems is doing the requirements analysis (including interviewing humans who give inaccurate or incomplete answers) and writing detailed functional specifications which account for every possible edge case. No LLM can do that.

One of my first one afternoon (for prompt refinement and trying it out) toy GPT-3.5 “applications”—really just a prompt—was having it interview for reauirements and draft and progressively refine a requirements doc in a specified (by reference to an author in popular Agile literature, not by specific templates, so just relying on the model’s general training) format. Its pretty much what convinced me of the broad utility of the technology.

I absolutely think that with a bit of effort, GPL-4-128K, a custom RAG framework and/or the Assistants API, you could build something that handles a lot of this. I’m not really bullish near term on LLMs as full replacements, but I can see it as a big force multiplier.

Re:With trillions at play, Pentagon fails audit for sixth ye... by DougBTX on Friday November 17, 2023 @02:26PM (#38308736)
> The hard part of accounting systems is doing the requirements analysis (including interviewing humans who give inaccurate or incomplete answers) and writing detailed functional specifications which account for every possible edge case.

So lots of tedious repetitive manipulation of language to extract facts and transform them into a systematic format? My prediction: LLMs are going to eat that for breakfast.

Re:With trillions at play, Pentagon fails audit for sixth ye... by nradov on Friday November 17, 2023 @04:13PM (#38310420)
None of those customers have actual data sets to train against when it comes to ERP and accounting software requirements analysis. The data that exists at all is scattered across random Word documents, wiki pages, paper notebooks, and legacy requirements management systems. Most of it is summarized. It's not recordings of past interviews with SMEs that business analysts and product owners conducted to extract functional requirements. In order to train an LLM to conduct such interviews you would have to obtain actual transcripts of such interviews. Not impossible, but unlikely to happen anytime soon.
Re:With trillions at play, Pentagon fails audit for sixth ye... by wslh on Friday November 17, 2023 @12:32PM (#38306880)
I would love to perform static security code analysis to that code.
Re:With trillions at play, Pentagon fails audit for sixth ye... by tandr on Friday November 17, 2023 @12:47PM (#38307132)
Are there any SA tools, that would understand COBOL?
Re:With trillions at play, Pentagon fails audit for sixth ye... by Jtsummers on Friday November 17, 2023 @12:53PM (#38307224)
https://www.sonarsource.com/knowledge/languages/cobol/

Never used it (and hopefully never will), but yes.

Re:With trillions at play, Pentagon fails audit for sixth ye... by xeromal on Friday November 17, 2023 @02:04PM (#38308356)
Not all the RAM on earth would be enough to process that shit
Re:With trillions at play, Pentagon fails audit for sixth ye... by kylebenzle on Friday November 17, 2023 @02:09PM (#38308431)
They just print and throw more money at any problem, there is no limit.

In the Navy I worked on the "Zeus" project, it was the most expensive illegal operation in the world. We would drop $1 millon dollar hydrophones like they were candy, $1 billion "tails" we were a little more careful with but would drop and re-deploy those at least once a month.

The whole idea of audits for DOD is nonsense and at best is a distraction. I was a nobody and saw a $trillion wasted on what amounted to a "plausible deniability" project so they could appear to Congress to be doing their best in case NK or Russia ever fired a nuke.

The whole thing is a joke, from beginning to end and there was zero accountability.

Re:With trillions at play, Pentagon fails audit for sixth ye... by TurkishPoptart on Friday November 17, 2023 @03:40PM (#38309861)
Is the problem really related to COBOL, or is the fact that this institution is basically unaccountable to taxpayers?
Re:With trillions at play, Pentagon fails audit for sixth ye... by chatmasta on Friday November 17, 2023 @07:59PM (#38313361)
See also, "John Stewart Questions Defense Deputy Secretary on Budget:" https://youtube.com/watch?v=50MusF365U0
Re:With trillions at play, Pentagon fails audit for sixth ye... by samus on Wednesday November 22, 2023 @02:55AM (#38376281)
If it ain't broke, don't fix it, they say. But if it is already broken in significant ways, there is little point in not replacing it.