I wouldn't call RHEL 6 ancient. Thankfully this may be going away at some point in the future, leaving it up to agencies to certify products or stacks on their own merits, or to instead have them be evaluated for specific purposes if sold as solutions: https://www.niap-ccevs.org/Documents_and_Guidance/ccevs/GPOS...

I was still seeing 5 used often, sometimes 4.

And 7 despite being out for 3 years? is still not certified.

There's a STIG already for RHEL 7, for what it's worth.

>I remember hearing there were going around filling USB ports with epoxy.

And we all laughed at PS/2 keyboards and mouseeses

You can boot an original IBM PC off the keyboard port. Look for the MFG_BOOT function here:

https://www.iee.et.tu-dresden.de/~kc-club/DOWNLOAD/DISK401/R...

Basically, the device connected to the keyboard port has to reply with code 0x65 when it's initialized, then the BIOS will read some bytes into memory and jump to it. Not sure whether this was brought forward to newer models or clones, though so it's just some fun trivia...

The original IBM PC switch was like the back switch on current PSUs. If it were on it would be booting anyway, right?

And that's why you can still buy new i7 and Xeon motherboards with PS2 keyboard connectors--because some sites don't want there to be USB ports on the computers.

My computer is locked in a mesh cage anyway. Well, the thin client is.

That's another great idea! Like the cages around thermostats!

Just put the software in the device in ROM, a forgotten technology. No malware will survive a power cycle.

It's like I read that malware could infect your "internet of things" thermostat and then hackers could remotely turn off your heat until you pay ransom. Just put the dang thermostat code in ROM. Power cycle, goodbye malware.

For more critical stuff, just have it regularly power cycle itself.

If you haven't seen it yet, you may like the stateless laptop idea: http://blog.invisiblethings.org/2015/12/23/state_harmful.htm...

Then you can't do over the wire updates, which means no fixes after it's been manufactured and installed, which would probably increase the costs quite a bit.

> Then you can't do over the wire updates,

Which, of course, is the whole point. If ROMs do cost more, I bet people who want secure systems would be quite willing to pay a bit extra.

It would be too expensive to put an OS in ROM, but the ROMs could contain the hashes of the OS on disk, and can verify the disk image before booting.

And besides, why would I want over-the-air updates to my freakin' thermostat?

Another way is to have a jumper that is required to enable the write cycle to the flash ROMs. That enables the manufacturer to update the ROMs before shipping, then remove the jumper.

Anyone trying to compromise the device would then require physical access.

Make it a button, so the customer can apply updates, but they need to press a button to make it happen. Add another button which, using only software stored in ROM, reloads the firmware from ROM. Then you have updates, but only when the customer knows and wants it, and if they ever screw it up then they can get back to a known good state.

The customer has no way of knowing whether they are installing genuine update, or MITM-modified malicious update. Public key cryptography solves that but then you have a problem how to protect the keys and crypto algorithms from tampering.

Update buttons ain't a good replacement for TPM hardware.

It's an enormously harder problem to convince the customer to push the "update firmware" button on the front panel than it is to insert malware remotely without any action or knowledge on the customer's part.

It’s enormously easier problem to exploit a well-known remote code execution vulnerability, that’s unpatched because updates are too easy to ignore.

For the end user, it’s extremely easy to NOT push that button.

It's not a panacea, but nothing is, and it would help a lot. It's not supposed to be a replacement for anything, just an addition.

Yup, that's the way to do it.

This is literally how everything was made until the very recent past. Next you'll be telling me that you can ship software on physical media.

One may consider that an incentive for manufacturers to make sure their code is working correctly before shipping it.

In many cases that would probably be prohibitively expensive, but for a thermostat or a light switch, it should not be that difficult to do.

Prohibitively expensive? That's how things have been done for decades. Only recently have manufacturers decided that everything should be redoable over the internet.

Are you including the capability to control it over a network? Burning potentially unfixable 0-day exploits into ROM would probably be a non-starter.

If you can update the firmware over a network, you have an uncloseable exploit vector.

ROMs are not unfixable. They do require physical access, though, which eliminates nearly all of the risk.

Even if those updates require signature verification with a public key stored in ROM? I guess the signing key cold get out or you could patch the firmware to skip the signature check (the way the initial iPhone was jailbroken)

We also literally just had the Windows key leaks within the last week. Un-updatable signing keys are still an uncloseable exploit vector. (And updatable signing keys are an uncloseable exploit vector. You can't win.)

Physical-access-only updates: yes please. We know what the security boundary is on those, and how to reset it.

Malware infection module can be stored nearby on different device infecting your target device every time you power it on, until you patch your device.

Right. But each device that uses ROMs is one less attack vulnerability.

After all, if attacks can target A, B, or C, that is no excuse to not fix B.

> No malware will survive a power cycle.

Not true at all.

Please explain.

I'm not the original poster, but unless you're talking about a very simple embedded system, most computing devices have lots of areas for malware to live that will survive power cycles. HDD firmware, graphics card firmware, various controllers have their own firmware, NIC cards have their own firmware, etc.

Even if the boot device is read-only, it would be a huge challenge to build any kind of system without these vulnerable components.

Every one of those devices that is done with ROMs closes another door to malware. It's certainly far more doable and reliable than ensuring the software doesn't have holes in it.

There's a huge market for secure items - routers, cars, thermostats, medical devices, avionics, ATMs, etc. I simply don't understand why ROMs aren't used.

Part of the reason that Windows is an approved OS is enterprise support. When you pay for hundreds of thousands of licenses for a product you can demand features. If the DoD went with Debian they would need an entire corps of developers to maintain government specific patches. Ubuntu offers enterprise support but it's from an African country and that is undesirable as you mentioned above.

I've seen government systems running Windows (USA), Mac (USA) but the strangest I've seen is SUSELinux (Germany). I have no idea how they got SUSE approved when Red Hat is more "American" but some agencies don't have to follow the rules.

SUSE is owned by Novell.

Not anymore, IIRC. When Novell was acquired by ... that company whose name I forgot (AttachMate?), SUSE was spun off as an independent company again.

At least that is how I remember it.

Yeah it seems things have gotten a bit more complicated than i remembered.

Novell bought Suse, then Novell merged with Attachemate, in the process Suse became a business unit (wholly owned company?) under Attachemate, then there was a merger between Attachemate and Micro Focus in 2014.

Anyways, i suspect that as Suse is FOSS, and was initially Germany based, and now is UK based, both being NATO allies, that DOD has few issues with continued usage.

SAP seems to have a strong preference for SUSE. SAP may have stewarded SUSE through accreditization processes.

It's well known that all of Germany's media, intelligence services, and politics was set up by the US after the war.

From what is seen on the geopolitical landscape, Germany is a vassal state of the US.

Too many people without security clearance can access and modify Linux. In any real security environment, open-source is poison. Period, end of story.

^ As if any amount of security clearance can erase human fallibility.

In any real security environment, humans are poison. Period, end of story. This is not an issue exclusive to open-source.

Of course not, but it's exacerbated by open source. Humans are indeed poison, which is why it's normal to involve as few humans as possible. Unfortunately AI is not sufficiently advanced so you still need some of them in order to get the job done.

This is a falsehood. One of many sources: https://news.ycombinator.com/item?id=7203211 (HN discussion of Snowden using wget to scrape NSA internal sites)

So security through obscurity?

Or maybe security through, you know, actual security.

Closed source does not mean obscurity–– and open source does not mean clarity (see OpenSSL, that one Linux 2.6 thing¹, etc).

It's not like being proprietary suddenly means the only security is through obscurity. Why do you think that? Are you just a zealot? Did you not consider that closed source software could be well-engineered and secure? You're welcome to read about some of the security features in Windows NT² (that article is a bit old but still relevant), which are considerably more thorough than (non-SE)Linux (I think SELinux has auditing now, so it's at least comparable to NT).

Now, don't get me wrong, I generally prefer open-source tools (for a variety of reasons) and tend to trust them more, but saying stupid stuff like you are just gives open source a bad name.

On top of that, obscurity is a perfectly valid layer of security. To use a mediocre analogy, of course you want your safe to be strong enough that nobody could break in even if it's in plain sight—but it's certainly not a bad idea to hide the safe as well.

¹ https://lwn.net/Articles/341773/

² https://www.microsoft.com/resources/documentation/windowsnt/...

I wasn't saying it was only secure due to obscurity. I should asked more clearly - "So the idea is to enhance security through obscurity?" For mission critical secure systems, I can see a case being made that software should be closed source, as it allows fewer people to be aware of potential attack vectors. Especially, if you know the software you are creating won't be used externally much or audited.

So you trust the black boxes someone sold you?

The opposite of “open source” isn't “black box”.

Not the person you're replying to but opposite of open source is closed source, and isn't that basically a black box to you since you don't know what it's doing?

Is there something more subtle I'm missing?

Yes. Microsoft offers source access to Windows. IBM and Oracle will rent you people who know the details of their software. None of those companies' offerings are particularly ‘black box’-y, in spite of being very closed source.

‘Open source’ is more about the development model (and freedoms) than about the nature of ‘knowing what the software is doing’.

Heck, I could argue that Linux is a black box to most people who aren't well-versed in kernel development. OpenSSL is notoriously difficult to understand. Sometimes huge bugs look relatively innocent¹ even with people looking at the code.

¹ e.g. https://lwn.net/Articles/341773/

At the level of security where you need to be paranoid about the operating system developers, how do you know the source code Microsoft shows you, corresponds to the Windows binaries running on your machines? (Let alone to the sum and total of all binary patches applied thereto?)

A source license probably also gets you the build system, or if not maybe you could throw more money at them.

But reproducible builds are a hard problem, so… honestly, I can't answer that.

And that's what I really mean by 'black boxes'. Yes, you can buy access to, e.g., Windows source, but that doesn't mean that what you review and what is running on your systems are the same thing. And I can't even speak to all the third-party bits that get compiled in to an instance of Windows. It's a really, really hard problem.

Well, end of story. That's it then. No more discussion can be had!

>filling USB ports with epoxy

This seems apocryphal. Its trivial to disable USB for a mass storage (or all devices) via things like group policy or other security controls. Or disable the controller. Those USB ports aren't perfect boxes, the epoxy would just run out all over the place. More than likely you'd have an OS-level security policy and bios block, which is trivial to do in a managed environment.

I hear this type of story from time to time and frankly its the sysadmin version of "he hacked us with a visual basic gui!"

>$400 device from China vs $40k from US.

I'd rather trust my network with a warrantied Cisco with same-day replacement than a $400 Abibaba sourced ROUTER PLUS SHENZEN SPECIAL with two weeks delivery. I suspect you've never managed a non-trivial network before, let alone one with real security policy if you'd consider running that and thinking you're secure.

> This seems apocryphal. Its trivial to disable USB for a mass storage (or all devices) via things like group policy or other security controls. Or disable the controller.

The question is - where do you stop? The controller could be re-enabled from a lower level, etc. The rabbit hole goes very deep. Sometimes it's best to just take control of the physical layer and call it a day.

> Those USB ports aren't perfect boxes, the epoxy would just run out all over the place.

Epoxy putty would work pretty well, and it's widely available.

There's also value in being able to visually inspect it and say "Yep, that USB port's disabled" versus digging through EFI settings.

Every motherboard is going to have that option in a slightly different place, but if you can put epoxy in one USB port you're pretty well set for any piece of hardware.

This scales to ${number_of_devices_you_can_see}. A hundred or more? Easier to manage remotely. You're also likely to have a very limited number of models in that case.

One thing to keep in mind is that the kind of place which cares about things this much tends to be the kind of place which can hire staff — and it's a lot cheaper to hire technicians who can verify that epoxy plug than the security engineers who can confirm that you've done everything right in software.

Consider another instance of the problem: verifying that your webcam isn't being used to spy on you. Since a surprising number of hardware designers were negligent and made that software controllable it is orders of magnitude easier to simply deploy a piece of tape than try to prove that malware hasn't disabled the status LED:

http://security.stackexchange.com/questions/6758/can-webcams... http://blog.erratasec.com/2013/12/how-to-disable-webcam-ligh...

How much skill and diligence does it take to confirm that you have disabled the controller using each manufacturer's interface (if they even have one documented), that there isn't some way to re-enable it later (or that something like a sleep/resume cycle didn't reset the controller), and that all of that continues to be true for every subsequent configuration change or software/firmware update?

I would suggest that any organization with this level of risk would be better off paying someone $15/hour to check the ports along with the rest of their physical status checks and put the security engineers in charge of other improvements with a higher return.

The thing is, if you manage those things remotely, so can an attacker. I imagine it would not be impossible for a sufficiently skilled and determined attacker to remotely re-enable the USB ports.

If you are paranoid enough or have actual reason to believe somebody would want to invade your network, epoxy is one way you can be really certain no one can hijack your network via an infected USB flash drive.

EDIT: Of course, if gluing the USB ports shut is all you do stop attackers, I am pretty much begging for trouble. And as somebody pointed out, disconnecting the USB ports from the main board, possibly disabling the pins is probably a better idea, as well as disabling the USB controller in firmware and locking the BIOS / setup, if it is part of a ... let's say comprehensive approach to securing your network.

> Sometimes it's best to just take control of the physical layer and call it a day.

If you want to stop your every day user from plugging in USB drives then this is probably all you need to do. In a scenario where you're concerned about insider threats with even a minimal level of computing knowledge, you have to lock down the BIOS and the OS layer as well. "Oh the IT guy put epoxy in the USB ports, guess I'll just take the case off and plug into the USB ports on the motherboard"

You can also cut the traces or epoxy the internally ports as well. It's not hard. It's just about what level of threat do you want live with. I imagine you could always defeat this by cutting through the epoxy or gently sanding the board to put probes directly on the traces, but then again security is all about depth.

I have a friend that worked at LLNL and she used to talk about secured laptops having their USB ports epoxied and the traces physically cut on the camera and microphones to help secure them. I think even the wifi and bluetooth were disabled as well.

After hearing these stories, it made me chuckle at Zuckerberg's masking tape.

Well, of course this doesn't give you a get out of jail free card. You still have to pay attention to the other layers in the stack. This is only about paying attention (or not) to the physical layer directly (as opposed to handling physical security indirectly in higher layers).

Case intrusion sensors are a thing. And i swear i have seen cases with loops for padlocks.

Padlocked cases are common on school computers.

Yeah i should have suspected. Have not set foot back there in ages though. Thinking about it i guess schools may be some of the most hostile computing environments in civilian life.

> Thinking about it i guess schools may be some of the most hostile computing environments in civilian life.

Long time ago, I've briefly managed an environment like that. It was crazy. Kids are really good at breaking stuff in creative ways.

For a hardened PC, the first thing I'd do is burn the BIOS into ROM. Read Only Memory. ROM cannot be infected.

Or to put it another way: "ensuring you've secured all hardware and software exploits in your stack from top to bottom" vs "epoxy and focus on network exploits". Don't knock physical security.

> The controller could be re-enabled from a lower level, etc

In a managed environment you could do it via the BIOS trivially, which is most likely locked as well. I mean, glueing the ports is especially stupid. You can chip glue off with your fingers or a key. If you're doing physical things to the PC, you'd most likely just remove the USB header from the mb and call it a day. Pop-open the case the case, remove it, bend down the pins, or cut it and go about your business. Messing with stuff that takes 60 minutes to cure is ridiculous. Ignoring the OS security policy is ridiculous. Ignoring BIOS controls is ridiculous. Ignoring how security is handled in managed environments is ridiculous.

It would take two minutes for a stoned teenager to pop-open the case and plug in his own USB connector into the header in this scenario. Less time for a determined attacker.

I imagine some middle-manager asshole asked for a piece of plastic to block the panel to make it 'look nice and remind people they're blocked' and some paper-pusher took it as "OMG THEY GLUED THE PORTS TO STOP HACKERS" He was just ignorant of how IT security is really done.

I think its obvious HN is mostly web-devs, not sysadmins or security people if stuff like this is widely believed and comments contrary to it get instant 'disagree downvotes.' If you think the NSA and the DoD just glue ports instead of doing real security, then I don't know what to say here.

Rather than immediately assuming you know more than the people who implemented this, try to consider why someone who is theoretically smart would want to do this. Also consider that most organizations implement multiple layers of security, adding another layer of security can't hurt here.

> In a managed environment you could do it via the BIOS trivially, which is most likely locked as well.

The BIOS may still not be low-level enough. There is nothing preventing a buggy xhci controller, chipset, BIOS, etc, from being exploited by a rogue USB device. It would be prudent to disable USB in the BIOS AND physically disable the ports somehow.

> Pop-open the case the case, remove it, bend down the pins, or cut it and go about your business. Messing with stuff that takes 60 minutes to cure is ridiculous.

You do not need epoxy to fully cure, you only need it to reach a point where the viscosity is high enough that enough of it won't drain out of the USB port when you turn the computer on its side. This can easily be under 5 minutes, depending on the type of epoxy and you could even trivially avoid that wait time by putting a piece of tape over the epoxied port. It may also even be cheaper to implement, since you can pay someone minimum wage to fill ports with epoxy, but it takes a slightly higher skill level to do work inside of computer cases. Additionally, it's easier to visually verify that all USB ports are epoxied than it is to verify that all internal USB connectors have been disconnected. Additionally, consider that many motherboards have rear USB ports directly soldered onto the motherboard, which would take far more effort and skill to disconnect than it would to just fill the port with epoxy.

> It would take two minutes for a stoned teenager to pop-open the case and plug in his own USB connector into the header in this scenario. Less time for a determined attacker.

An attacker who has broken into the government building is not the person who this is intended to guard from. It is intended to guard from employees accidentally inserting compromised USB devices into their computers. If the attacker is opening your computer case, they have many more options than USB ports for delivering an exploit payload. Though it's also very likely that these cases are also physically locked and have case intrusion detection enabled. Not that those protections are particularly difficult to get around either. This may also even help IT avoid support phone calls from users saying "hey, how come my USB port doesn't work?" where epoxy in the ports shows some serious intent.

Additionally, in the case of a real attacker who has physically entered the building, and intends to deliver their payload by flash drive: formerly they could just waltz by some computer, pop a drive in, and walk away. Now they'd need to at the very least open the case, which at the very least makes it take slightly longer for them to deliver their payload, and is much more likely to draw suspicion.

Yeah, the curing time can vary from 1 minute (or even less) to several hours, depending on epoxy type. Now mix it with some filler to make putty (or just buy epoxy putty ready to use) and even the curing time is no longer critical.

Anyway, you know the discussion has gone down the rathole when you're debating the relative merits of epoxy recipes for securing computers. :)

> This seems apocryphal.

I have no reason to believe the person I worked with would make it up. There would just be no point in it.

> Its trivial to disable USB for a mass storage (or all devices)

Except there are hundreds of different kinds of devices, and you tasked with quickly "doing something to fix the problem". Do you have time to go and dig through different types of BIOS menus or open the cases to all of the machines. Or is it easier to get epoxy plungers and send an army of people from desk to desk. Normal epoxy is a pretty decent electrical insulator (some board or components used to even be dipped in epoxy to protect from environmental damage or tempering).

> I suspect you've never managed a non-trivial network before,

No, but I was selling a solution with one in it. And $40k was making a decent size cut in the profit margin.

Why not just unplug the USB header if you want a physical solution? The idea that you're shoving glue in there is incredibly ridiculous. You can chip that off easily with your finger or a key. I seriously doubt this is a real story because it flies in the face of published STIGS and basic common sense. Nor would it stop a remotely determined attacker/idiot.

That said, I could see glueing a panel to block them as a visual to remind people that those ports are off, but not as a primary blocking device. More than likely its done via security policy.

They epoxied the USB ports on my secure machine when I worked at one of the national labs about 12 years ago. One day I came in and the admin had gone through everyone's office the evening before with a bunch of JB weld to fill the USB ports.

So, whether or not you believe that it is a useful solution, it certainly did happen. As for alternatives to gluing, I assume that was time related: filling in a port is certainly faster than opening thousands of machines to give them a physical port-ectomy.

Port-ectomy: a new word in my dictionary, must remember to use it someday!

> You can chip that off easily with your finger or a key.

Because it was more of a reminder for stupidity. "Oh, look there is glue in there, that's right we not supposed to stick random USB devices in there". If the machine is on their desk, yes, they could plug in a PCI device that has an USB thing on it and still connect. But by that point they are really going out of their way, they are opening the case and such, and it will be very hard to maintain the idea that it was just a stupid mistake.

Exactly. The other related threat is the new junior admin being asked by a senior exec to allow (windows policy-wise) the USB storage just temporarily. "Because I need this presentation NOW as I have to get to the airport in 45 minutes!

Rear USB ports are not attached to the motherboard via trivially removable connectors.

It also really is not trivial to chip off most types of even general-purpose epoxy with your finger on a flat surface, let alone a tiny USB port which your finger doesn't fit in. If the epoxy was selected with some level of care, it may be very, very time consuming, or nearly impossible to remove, even with the right tools.

No matter what, it's a helpful measure in addition to every other way you can also disable a USB port.

well the thing is i dont know if you know it foxconn builds your ciscos i only know cause i worked installing fiber in the plant they have in houston its one of a number in the US. in there i saw lots of chinese ladies putting together made-to-order cisco switches of all sizes with boards i assume were made in china. oh dont get me wrong the americans worked in sales out front. :)