I know several people who were trained to do board level component replacement, but for most of the equipment they had, they were not authorized to do the repair, even though they were trained and equipped to troubleshoot and repair most circuit boards (including surface mount).

For design and creation of new things (eg: software) they used to train for that, but I don't think they do that anymore.

So some PFC just destroyed a $40,000 system because a Capt. was up his ass about fixing it right now, he has no ability to push back, nothing that can be done but to follow the order to the best of his abilities. If he destroys it, well if the capt. is a dick, he gets written up, and charged for it. If not, the equipment is still fried, and has to be replaced.

With a contractor, they are outside the chain of command, and can tell the capt. to pound sand, and tell them the equipment will need to be shipped and fixed properly. And reminds the capt. that there are several dummy switches that without the right tools, and training, the equipment will fry itself, and if this is discovered to be the case, the indecent will be turned over to CID (army detectives) for investigation and that he should make arrangements to source a replacement from supply ASAP.

Seeing the straight don't give a fuck of these guys could exercises, in relation to what is best for the equipment and not the commands ego was amazing, and made total sense why on the high end equipment this method was used.

there is your problem.

that does not mean that the fix is to have a 'nobody can touch this but the contractors'

The US Military - all branches - have a long and great tradition of making field fixes. This is critical out there.

Waiting 3 days for a 10min fix may be acceptable in a US base, but in a war zone it will get people killed. At the very least, they need to be able to get remote support for on-site personnel to actually implement the fix.

Not to say there haven't been great innovation, but realistically most of these repairs are being made by guys that barely got out of high school, and had maybe 20 weeks of training. (I was one of them, its not a dig, its reality)

And as much as it pissed me off when a piece of equipment was no longer available, because it got shipped off, I was also equally pissed off when my unit all got sick and was throwing up because our water purification system broke, and a commander told the cooks to add bleach to the water. That is something a soldier has to do, where as a contractor wouldn't be likely to.

In the military you spend a lot of your time pissed off because someone, somewhere is doing something stupid, and it will probably get you killed.

This very succinctly sums up almost every conversation I've had with ex-military.

I actually worked a bit designing military comms when I was just out of school, but left because the politics was too crazy. Military comms seems to be held back in the 60s where everything had to be built for purpose and it doesn't matter that this radio uses 15 bit bytes and runs on 22 volt power and that radio uses 13 bit bytes and runs on -48 volt power since they were never designed to communicate or exist in the same system. Until a "Grand Program" is started to get them to talk. Then contractors comes out of the woodwork and eventually they come up with a design where the two radios (positioned next to each other) pass "XML" (<bit position:0 value:0\>) over a custom designed third pair of radios and can communicate only a small subset of what either radio was able to express. Of course none of that uses Ethernet or IP or even some standard military connection protocol, because that would make things less efficient and suppliers want the biggest numbers possible so they are competitive.

If we are going to have a technology dependent military and they can't use COTS then they should at least make things modular so learning about one piece of hardware can be applied to other pieces of hardware. And hey, if it's modular then maybe military techs can at least swap modules to fix stuff.

We have time and available people who are educated, they can learn the some odd 40 systems in 27 weeks (excluding tacSat.)

So all those conveniences, only serve to make it easier for our enemies to understand. Additionally alot of the military communications equipment make use of proprietary hardware that perform functions standard equipment can't and in environmental conditions that would make it impossible for off the shelf equipment.

As for the network over FM they've had that since the late 90s, at the time because of the security and frequency hoping, and it being all done analog, the network could span roughly 15 miles and had a 64kbps backbone.

Just enough to send text file reports over.

Sure there's a time for LPI/LPJ, but short of a war with China I don't really see the need for most forces. Anywhere else we'd have air superiority and jammers are just loud, bright targets.

If you have a LPI/LPJ module you just put in the radio when you need it then it's much easier to keep out of enemy hands.

Then there is the cost of training for all the different radios. Yes, you could put one man or of a squad in training for 6 months ago he knows the basics about every radio there is. Or you could spend a couple weeks teaching everybody how to work a radio. Or you could have a real in depth training so you have a lot of fully qualified repair technicians.

I'm not exactly sure what your talking about for the radio waveform fielded in the late 90s. That's too late for SINGCARS and too early for JTRS waveforms like SRW and WNW. The capability description seems a bit like SRW (though that is frequency hopped, not FM), which was one of the few limited "successes" that came out of JTRS (the design goals were 100x as many members of the network, 10x the range, 10x the backbone capacity, and the ability for software upgrades to make it compatible with any future radio).

as far as COTS parts, I mean if you think you can build a radio of that size with its requirements,life span,etc someone probably would have.

they moved to slightly modded radios similar to law enforcement for squad level communications, but I dont see that happening for larger scale systems in the near future.

But the field operation guide doesn't tell you how to open the radio, that is beyond operator (RTO) and field services.

There were at least 11 builtin counter measures, to prevent someone from exploring the guts of the radio, and sure you could see what is in the radio, but once the eproms fried themselves, you would be hard pressed to figure out what was on them.

The manuals that explain those enter workings aren't issued to field units, hence why they have maint. Centers.

Not sure what you mean by too late for SINCGARS, the originals before the half sizes came out, I just as you could load the crypto and hopset from and ANCD, all you need to do was take a dog bone cable and splice and map the wires to a serial cable, and then configure telnet. Then set the proper broadcast mode on the radio.

you could probably find the specifics in a leaked field manual :)

Another big reason for the strange and overbearing standards is that there are so many different classified legacy systems that need to function, and no one is really allowed to alter, that newer systems need to be backwards compatible to work with them.

Sure, legacy systems are an issue and compatibility is necessary (especially for systems we need to talk with allies who spends less money upgrading equipment than we do). But that doesn't justify the new things not working together. Is there a reason that we got MADL with the F-35 and IFDL for the F-22 and needed a special program (with unmanned airborne relays) to allow the two to talk? Those are both new systems, there just doesn't seem to be any overall coordination. Heck, to some degree it seems like this inability to interoperate is intentional, as one of the F-35 requirements was that it could receive but not transmit on the existing legacy waveform, Link-16.

I don't mean to be so negative about it. The technology behind modern military radios is incredible and the radios effectively meet the requirements that the military specifies. This allows them to do amazing things. I just think that, particularly for communications, they need to look past the new hotness for something less exciting that both ages better and works well with existing systems.

Surely there is a middle way? This is a classic throwing the baby out with bathwater.

You still have the issue of middle-managers in the armed forces asking enlisted men to do stupid things, that end up wrecking equipment, but the only way to fix that is for better accountability within the command structure. Good luck with that, since the lack of accountability shields the only people who have the power to fix it. [1]

[1] See this outline of the fatal collision of the USS Fitzgerald for an example of how well accountability for stupid orders works in the US Armed Forces - https://features.propublica.org/navy-accidents/uss-fitzgeral... [2]

[2] The long and short of it is that someone above you tells you what to do, and if you can't do it, you get canned. So you do it, but you cut corners. If those cut corners cause a disaster, it'll be your ass on the line.

Are there statistics on how many things were successfully fixed vs how many ended up in the maintenance center? If not, that is a possible survivorship bias, kind of.

Or is it the purchasers who're procuring the equipment which "needs to be shipped and fixed properly"? Are we fighting wars in a manner "wait a minute, dear enemy, I'll just fix this small issue in a gusto, and we'll continue"?

Just curious.

With that said, if a commander orders someone under his command to smash the piece of equipment with a hammer, the contract won't interfere, but he will report the situation.

The issue is, most of the equipment at this point is well beyond the knowledge a company commander would have, they aren't qualified to know if this is something that can be fixed in the field, and a soldier under his command that is qualified isn't in a position to refuse the capt. order. So they fry the equipment.

The army is pretty big on redundancy, and having contingencies, it would be very rare that an operation couldn't continue without a single piece of equipment. It might be harder, and may be a greater risk, but in a place where lots of things get shot, you make a plan, then you make due.

I'm not saying the scenarios I'm describing fit all situations, but I can say that far less equipment was destroyed after they came on board.

I'm mostly talking about the whole company approach to such time-sensitive customers.

In general, military (electronic) gear is designed to be difficult to repair because repair and reverse-engineer are pretty similar. One radio being out is a small problem. The radio system's crypto being compromised because it was captured and reversed is a big Problem.

The suggested notion that anti-tamper integration exists on arbitrary whathaveyou weapon systems is, at best, misinformed. Field updating of operational software at the box level over e.g. 1553, or ARINC 429 via 615, or even vanilla RS-485 PHY via bespoke protocol, is very much a thing and almost de facto mandatory capability for most systems. If we broaden the scope of the term "field" to include intermediate-level maintenance in forward-operating labs staffed by uniformed personnel, then even firmware updates of cherry-picked circuit card assemblies can be a thing (although much simpler isolation to a replaceable circuit card or modular assembly is a lot more typical; CCA-level firmware updating is usually reserved for depot or OEM activity).

> In general, military (electronic) gear is designed to be difficult to repair because repair and reverse-engineer are pretty similar.

Bullshit. In fact the exact opposite is almost always boilerplate baked into acquisition performance objectives, sometimes even down to defining explicit MTTR targets. Certain systems even modularize the anti-tamper crypto applique portion to enable repair of everything else that isn't anywhere near as sensitive and much more probable to fail based on analysis.

The real meta is an adversarial one where private defense contractors are intentionally exploiting a complex process with the objective intent of locking in the ever so lucrative OEM sole source sustainment dependency, and the corrupt influence on DFARS statutory language as it relates to technical data rights is a critical part of that game.

Ha...you've clearly never been a software engineer in the encumbered world of fielded defense systems.

Basically, a Capt can say what ever they like but if it's just not possible to be fixed in the field then it's not going to happen.

Also, you don't get thrown into jail everytime you disobey an order. Especially when it comes to technical equipment.

And "everything possible" would include tools, electronic or software, and software codes - and yes, possibly engineers who created them on the line, something like what was done when Apollo project had landings on the Moon. Only in this case I'd assume different kind of responsibility than just contractual.

I don't fall in the military hierarchy at the moment, but that doesn't mean I can give aid and comfort to enemy. Again, I'm not a lawyer - that's why it's a tribunal - but for a court to give go-ahead to this (absent more prompt collaboration beforehand) would be something to expect. What would you expect if you'd be in the field in this case?

By contrast, a soldier in the chain of command can be punished administratively, and disobeying orders is a crime that in most militaries is only excusable if the order is criminal, not if it's stupid. Rarely brought, but the spectre of it is there

I have seen it work with one pissed off LCpl. The officer decided to back off.

There is no way you could do your job badly enough for it to be treason nor as far as I know subject one to a military tribunal.

As for your second scenario, yes, that's about it. Although to be honest it's unlikely to ever go to court martial; an officer demanding impossible things happens all the time and is dealt with at the company level.

The proper term "Technical Order" is indeed salient to the Air Force, but the notional concept of a "technical order" is precisely what the parent describes and exists in all branches of service, deriving its authority from a long traceability chain that terminates as a general order issued by the Secretary of the respective branch of service, e.g. in the Navy, its functional equivalent is a "Technical Manual".

I beg to differ on distinction. Whereas both uniformed and civilian personnel are required to comply with executing technical orders per the same chain of authority, the latter is almost completely shielded from the punitive hammer of the UCMJ.

Despite seemingly popular belief, uniformed personnel maintaining military equipment simply aren't at liberty to arbitrarily fix shit that breaks however way they please using whatever support equipment/material is readily available.

Think of it this way: the general order issued would be that all uniformed personnel must comply with the explicit language of technical orders in performing routine/unscheduled/critical maintenance of military equipment---no more, no less. In turn, said technical orders specify the applicable troubleshooting, diagnostic, repair, test, documentation, etc. instructions that maintenance personnel must follow and even describe system theory of operations (albeit fairly high level; nowhere near the level of specificity adequate for an engineer).

Consider this[1] junior Air Force crew chief. He's reviewing the maintenance records (specifically, 781A records which document discrepancies, related maintenance and their associated corrective actions; maintained in parallel with mirror digital database equivalent) of an assigned aircraft because he was trained in tech school on compliance of a certain mandatory technical order[2] to do so. Everything he and all his uniformed colleagues are tasked to perform is quite literally "by the book" (of which there are many) and to be executed without deviation. In theory, nothing is left to guess work and everything is traceable, all the way down to what pieceparts of a weapon system can/can't be serviced at specific levels of maintenance. Technical order doesn't exist? Can't do it. Demonstrated proficiency at a routine task? Doesn't matter; the applicable technical order must always be present, open at task and performed to the letter at all times. In this environment, you're not authorized to wipe the ass end of an aircraft without an explicit technical order enumerating step-by-step instructions. That maintenance officer screaming at this guy's senior enlisted supervisor is in fact a logistician in company-grade uniform. Said officer can steer this airman towards court martial for a serious enough technical order deviation, but really can't do shit if exhaustive technical order compliance doesn't result in a desired outcome.

In your example, the system would be deemed "not mission capable", the required part(s) would be placed on order (whose priority shot call would be determined by the production superintendent) and you'll move on to something else until the part(s) arrive and you're directed to resume maintenance action.

[1] https://www.defense.gov/observe/photo-gallery/igphoto/200245...

[2] https://www.tinker.af.mil/Portals/106/Documents/Technical%20...

The manufacturer/contracting company will always have a contract with the customer limiting liability so no, you're not going to be tried for anything but the company might get sued for breech of contract if things go really wrong.

But my experience has been they they are pretty chill as long as you're respectful, make it obvious that you're competent and you know what you're talking about (which requires actually being competent, not something you can fake for long in that kind of situation!), and offer alternatives and solutions to get things going again.

On the other side of the coin, I was one of those contractors in Afghanistan (2011-2013). I specifically co-developed an in-depth intensive five day course for all designated terminal operators. We didn't care what your background or experience was, we took everything from truck drivers to satcom techs through the same course. Taught the basics of RF theory, signal flow, troubleshooting, fault isolation, operations, and maintenance. Only the first two days were PowerPoint heavy(with very few words on the slides, mostly picture to relate concepts). After that, it was three days of hands on doing setup, tear down, and fault isolation. We would break all kinds of things, and they would repair it. All the problems we presented to them were real life issues that had been observed in the field.

The high point of that work was when one of the satcom techs came up to me after the course and said that they had learned more from me in a week than in six months at the schoolhouse.

As much support as we gave the folks we were there to help, it was all organic knowledge gained through experience. The satellite dish controller was a black box that we, as technicians had to reverse engineer because the company would not provide us with schematics. This was made worse by there being at least four hardware revisions (despite the company saying there were only two) with only some parts compatible across revisions.

The reason for not providing schematics? They feared one of us would clone it and sell it to the Army for cheaper, since it was the only custom piece of hardware in the entire system.

To be completely honest, this sounds like a classic case of conflating hardware revision and configuration as it pertains to a managed configuration item. That you didn't have access to approved internal schematics, and technical knowledge of the black box was constrained to a defined external interface boundary prior to independent reverse engineering efforts is certainly consistent with that.

Class II ECPs consistent with MIL-HDBK-61/SAE EIA-649-1 and the system's documented objective maintenance plan are a thing. Surely the SMR code of this controller (more specifically, its maintenance repair and recoverability disposition indices as specified by AR 700-82 [1] that logistics activity would have assigned it) in the context of reported anecdote would be sufficient to infer an interesting story.

I certainly don't doubt that you encountered 4 different harware revisions, but it strikes me as much more likely that the maintenance plan for said system never intended internal hardware abstraction layers to be serviced by activity at your level, instead structured to be independently managed by contract activity driven by performance-based logistics incentives with the objective of ensuring that, from a black box behavioral perspective, what they reindoctrinated back into the supply system is tested to be operationally indistinguishable ("form, fit, function" as it were) and thus compliant with overarching performance specifications when integrated at the next higher assembly level.

[1] https://armypubs.army.mil/epubs/DR_pubs/DR_a/ARN30303-AR_700...

The amount of training required to get to that point is just massive. It would take us six months of having field techs putting units together and working with the more experienced guys to work through problems with systems in the factory before they could manage going out to the field to do basic tasks. But even then, the systems were so software-based that eight out of ten times they'd need support from the software team to diagnose and fix the problems...

As a systems architect (this was in SATCOM, small specialist vendor, mostly selling to Navies), I tried to push in the direction of designing the systems to have a small number of interoperable LRUs (line-replaceable units) that the customer could keep spares of and change themselves, then ship back to us for refurbishment, as well as redundancy so the system could keep working when one of a type of unit had failed (for when it was mission-critical). I think it's a pretty good middle ground.

But the problem of course is vendor lock-in where the manufacturer screws them by making replacements extremely expensive. There's not enough recourse against the big companies doing that (like banning them from other acquisition programs). Too much lobbying, back-scratching, revolving doors between military and industry goes on for that I guess...

There are plenty of other root causes where many components were overstressed and replacing them all with a new board would result in less downtime than replacing components as they fail, too. Of course, sometimes you can do well with replace this group of components anytime any one of them fails.

Also, availability of full boards and availability of individual components makea a difference.

And their backplanes were breadboards!

You can also see the 1970s evolution of the "breadboard blackplane" into "pseudo-ball-grid-array backplane, with arbitrary collections of bodge wires loosely mated to it via hand-crimped proto-Molex connectors." (The only difference in practice being that you could now plug/unplug a "group" of wires, rather than one at a time. There was still no screen-printed indication of where on the BGA grid to re-attach anything, since the grid wasn't customized to its purpose, but rather was still fundamentally a "generic routing board" in the same vein as a breadboard.)

>When I was in Iraq (08-09), we had a juniper firewall go down, didn't have another on back up, and they had to fly a contractor out from the states to Iraq, to change it out. Took 10 minutes to diagnose the problem, and 3 days of waiting, and 15 minutes to install it. We lost feed to the predator drones in the area, phone, internet, coms, and several other mission critical things for 3 days.... a real s** show.

I'm not going to say it didn't happen, because its the military, but that is super uncommon, and against so much info sec, it just doesn't stand to reason.

This isn't something you really want to try to do in 120 degree heat, hanging out the side of a vehicle in a very sandy environment.

Every so often, something would make it into the post-WWII surplus market with anti-tampering countermeasures still intact. Hilarity seldom ensued.

I did train soldier on technical equipment (not in the US) and we would have broken or known unreliable parts given out so they learn how to find it. Also we had no authorization to repair most stuff and sometimes it was hard to keep broken parts to use the for training because these parts where scheduled for maintenance witch would mean the manufacturer of the part would replace it and turning the perfectly "good" part for training into e-waste. It's the classic problem in hierarchical structure. The top just never gets what the people at The bottom really want and need. They cant warp their head around why a broken part could be of more value than having 100% of the replacement parts functional all the time.

Sure I understand that a full diagnosis/repair of say a water damaged board I might not trust, but if it's just a port or a capacitor then it seems unlikely to cause massive issues if the person has any EE background whatsoever.

I will say though that it has changed a lot since I was active duty - better and worse. It's getting better for software and worse for hardware.

I'm doubtful that it's impossible to train some number of existing personnel to do basic repairs - especially the repairs involving simply replacing a part, as described in the article.

The original bureaucratic logic you're criticizing seems bad. But it seems like you're using a similar sort of reasoning to claim things are all or nothing.

This is the part that's making less and less sense to me... we are paying the going rate, just indirectly, in fact we're paying more than the going wage to cover the overhead of using a contractor. why has it become so hard for the federal government to pay a proper wage directly?

When we are already paying $30k for a $20 part, we can afford to pay them market rate.