News for hackers. Stuff that matters.
Re:Cisco devices must regenerate self-signed certificates be... by yardstick on Friday December 20, 2019 @04:15AM (#21842719)
Ignoring for a moment the fact that they hard coded an expiration date:

This serves as a good reminder of why it’s a bad idea to set expiration dates at the end of a year, where lots of people are on leave for the holidays (this notice came out only 2 days ago). Instead try sometime less complicated, like mid March or Sept.

Re:Cisco devices must regenerate self-signed certificates be... by black_puppydog on Friday December 20, 2019 @04:23AM (#21842755)
I think if someone lacks the awareness that hard-coding expiration dates is not a good idea, expecting them to think to this level is a bit much... :|
Re:Cisco devices must regenerate self-signed certificates be... by eddieroger on Friday December 20, 2019 @09:28AM (#21844350)
It's a good reminder nonetheless. I was on a team responsible for a web server farm and we noticed after a few years that the expiration date kept creeping closer to January since we would regularly kick off the process a few weeks before it actually expired, and decided one year to renew twice just to reset the clock away from January.
Re:Cisco devices must regenerate self-signed certificates be... by jcadam on Friday December 20, 2019 @07:16AM (#21843560)
Why can't you be grateful for the gift of canceled vacations and overtime pay? Cisco cares - making sure you're home (in your data center) for the holidays.
Re:Cisco devices must regenerate self-signed certificates be... by mywacaday on Friday December 20, 2019 @11:22AM (#21845245)
Add to that making production changes two days before holidays. Sitting here waiting for another team to debug what they broke and how to fix it so my app will work. Holidays should have started an hour ago.
Re:Cisco devices must regenerate self-signed certificates be... by C1sc0cat on Friday December 20, 2019 @11:35AM (#21845345)
We had a freeze starting the 18th /19th for deployments
Re:Cisco devices must regenerate self-signed certificates be... by raxxorrax on Friday December 20, 2019 @05:54AM (#21843183)
x.509 certs all have a validity period. Of course they could have set them to 2099, but as far as I know you cannot change the validity without reissuing the cert.
Re:Cisco devices must regenerate self-signed certificates be... by tialaramex on Friday December 20, 2019 @01:44PM (#21846444)
It's called notAfter and since it's part of the signed document you cannot change it without invalidating the signature. In a custom system the issuer could (but probably shouldn't) sign a document which is identical save for the notAfter date, but in the Web PKI this would be a serious miss-issuance and have consequences.

By convention the notAfter field is filled out in Zulu timezone (ie UTC) and so these will expire simultaneously around the world.

Re:Cisco devices must regenerate self-signed certificates be... by zenexer on Friday December 20, 2019 @01:46PM (#21846453)
The problem here isn’t that the certificates themselves have an expiration date, it’s that the code that generates them has a hard-coded date. Even if you try to generate a new certificate after 2020-01-01, the expiration date will still be 2020-01-01 on the new certificate.
Re:Cisco devices must regenerate self-signed certificates be... by phicoh on Friday December 20, 2019 @04:39AM (#21842819)
What I find bizarre is software that accepts a self-signed certificate only to then reject the cert because it has expired. What security problem are they trying to solve there?
Re:Cisco devices must regenerate self-signed certificates be... by saurik on Friday December 20, 2019 @05:07AM (#21842936)
The reason to expiration dates on certificates is so that a compromised private key has a limited lifetime of applicability, under an assumption that you will be actively rotating keys and issuing new certificates that expire at later points in time. As far as I would think this consideration has nothing at all to do with whether the certificate is self-signed.
Re:Cisco devices must regenerate self-signed certificates be... by phicoh on Friday December 20, 2019 @05:30AM (#21843060)
The model in most software that allows self-signed certs is that you eiter disable checking all together, or that the user accepts a specific cert.

If you disable checking all together, then an attacker doesn't need to compromise a private key. Any impersonation attack will just work.

If you require the user to accept a specific cert (and store that information) then you can just as well let the user accept an expired cert.

The current situation just creates a lot of hassle for people without any security benefit.

Key rotation is completely separate from the lifetime of a cert. You can renew certs while keeping keys constant forever. In practice, if your key gets compromised and you have to wait for the cert to expire, then you have a very big problem. Even with letsencrypt it is likely more than a month before the cert expires.

Re:Cisco devices must regenerate self-signed certificates be... by AnIdiotOnTheNet on Friday December 20, 2019 @09:14AM (#21844227)
> The current situation just creates a lot of hassle for people without any security benefit.

In my experience, many "security" people see this as the purpose of their job.

Re:Cisco devices must regenerate self-signed certificates be... by mrspuratic on Friday December 20, 2019 @06:30AM (#21843326)
Also keeps CRLs as short as possible.
Re:Cisco devices must regenerate self-signed certificates be... by 2ion on Friday December 20, 2019 @11:50AM (#21845454)
You can build good PKI using your own private CA just as well as using any publicly trusted root. Esp. in the realm of on-prem infra, that's not uncommon.
Re:Cisco devices must regenerate self-signed certificates be... by thdrdt on Friday December 20, 2019 @06:17AM (#21843275)
I think what Cisco is trying to solve is to give users the possibility to connect to the device over a secure connection in case a user has no access to a signed certificate by an authority.

So the user must still connect the self-signed certificate when connecting to the device (over ssh for example).

So in this case the user generates the certificate and also accepts it.

By the way, most ssh clients will tell you if a certificate changed. So the moment your self-signed certificate is compromised you will know.

Re:Cisco devices must regenerate self-signed certificates be... by oarsinsync on Friday December 20, 2019 @09:55AM (#21844576)
> most ssh clients will tell you if a certificate changed

Most implementations of ssh in the wild don't use CA (or self) signed certificates. I dont know if Cisco even supports the use of ssh with certificates.

Re:Cisco devices must regenerate self-signed certificates be... by jlgaddis on Friday December 20, 2019 @05:08PM (#21847809)
> I dont know if Cisco even supports the use of ssh with certificates.

They do -- in addition to public key auth, as GhettoMaestro mentioned.

Re:Cisco devices must regenerate self-signed certificates be... by GhettoMaestro on Friday December 20, 2019 @11:05AM (#21845110)
Cisco SSH has supported public key auth for as far as I can recall.
Re:Cisco devices must regenerate self-signed certificates be... by oarsinsync on Friday December 20, 2019 @11:52AM (#21845479)
SSH keys and certificates are different authentication mechanisms (I learnt this from https://news.ycombinator.com/item?id=20955465)
Re:Cisco devices must regenerate self-signed certificates be... by C1sc0cat on Friday December 20, 2019 @11:38AM (#21845379)
I would assume you can login via the CON (console) or the AUX port as a last resort.
Re:Cisco devices must regenerate self-signed certificates be... by TrueDuality on Friday December 20, 2019 @12:06PM (#21845598)
Oh god. Thanks Cisco for dropping this 11 days before the end of the year. Wooo, glad I saw it here at least...
Re:Cisco devices must regenerate self-signed certificates be... by black_puppydog on Thursday December 19, 2019 @08:16AM (#21834582)
> Self-signed X.509 PKI certificates (SSC) that were generated on devices that run affected Cisco IOS® or Cisco IOS XE software releases expire on 2020-01-01 00:00:00 UTC. New self-signed certificates cannot be created on affected devices after 2020-01-01 00:00:00 UTC. Any service that relies on these self-signed certificates to establish or terminate a secure connection might not work after the certificate expires.
Re:Cisco devices must regenerate self-signed certificates be... by rb808 on Friday December 20, 2019 @09:08AM (#21844186)
Certificate management is such a huge problem for big organizations. Does anyone have guidelines on how to do it properly? You really need a way to track who is using what and how they automatically get renewed a long time before expiry.

My view now is that certificates should be renewed very often - like every month, if you see something within 6 months of expiry there is a problem.

Re:Cisco devices must regenerate self-signed certificates be... by jlgaddis on Friday December 20, 2019 @05:19PM (#21847882)
> You really need a way to track who is using what and how they automatically get renewed a long time before expiry.

If you have any sort of monitoring system in place, it should also be able to check for certificate expiration (or "validity" in general). Even the old antiquated nagios has had this ability since the dawn of time.

As a sort of last resort and/or where no other solution is in place, you can do what I did on a customer's network a few years ago. Fortunately, there wasn't much "security" in place (internally, at least).

nmap can easily 1) detect/identify TLS and 2) produce "grepable" output.

I simply gathered a list of all IP subnets that were in use and scanned all of it. Afterwards, it was fairly simple to go through the results and figure out which hosts were running services using TLS on which ports.

There are existing tools that can connect to services using TLS and gather the information from the presented certificate. If you can't find one that does what you need, that's no big deal, it really doesn't take much to write your own tool just for this purpose.

(In fact, something like this -- running on a host that has access to connect to all of your TLS services -- might be nice to have even if you have a "proper" solution in place. If (when?) something falls through the cracks, you'd have this as a sort of "fallback" or "backup" to alert you to anything your proper solution might have missed.)

Re:Cisco devices must regenerate self-signed certificates be... by hylaride on Friday December 20, 2019 @09:29AM (#21844364)
I'd like to know as well. At my work we had to partner with a major bank and they demanded a fully signed (sigh) client certificate for the integration. The way their API works means that we have to do a planned hard-cutover before the certificate expires. There's a 60% chance that we'll have trouble reaching out to the right people when the time comes.

Ugh.

Re:Cisco devices must regenerate self-signed certificates be... by uxp on Friday December 20, 2019 @12:22PM (#21845721)
This is a common problem in healthcare insurance. API connections will stop working for about 3 weeks every year as each side performs a complicated dance of noticing the expiration and updating the systems to accept a new public certificate.
Re:Cisco devices must regenerate self-signed certificates be... by ehvatum on Saturday December 21, 2019 @12:12PM (#21852204)
Seems to be a thing that happens at big companies in general. I add —-no-ssl-verify to everything and it saves my ass every single year.
Re:Cisco devices must regenerate self-signed certificates be... by hylaride on Tuesday December 24, 2019 @10:13AM (#21872524)
That's ok if you're not worried about man in the middle attacks, but it does make it a lot less secure.
Re:Cisco devices must regenerate self-signed certificates be... by pnutjam on Friday December 20, 2019 @01:07PM (#21846093)
IMHO, 60% sounds low...
Re:Cisco devices must regenerate self-signed certificates be... by doublement on Friday December 20, 2019 @10:34AM (#21844871)
A short period of renewal would mitigate the problem where every employee on the notification list has become a former employee / invalid email by the time the actual expiry occurs.
Re:Cisco devices must regenerate self-signed certificates be... by r00fus on Friday December 20, 2019 @12:02PM (#21845558)
That kind of turnover seems crazy. You should also be using an email distribution list as the "notification target" which is kept up to date when someone joins or leaves the team...
Re:Cisco devices must regenerate self-signed certificates be... by _jal on Friday December 20, 2019 @11:36AM (#21845354)
We take a belt-and-suspenders approach, mainly because I have a huge fear of the embarrassment of letting a cert (or domain name) expire.

We built a DB to track them; nothing special, it sends email reminders. Been meaning to just publish calendar reminders, but haven't gotten to it.

We also check expiry dates on the live certificates via one of our monitoring systems (Zabbix, but this isn't a built-in check) that will start yelling, I think, 2 weeks before expiry.

Re:Cisco devices must regenerate self-signed certificates be... by Znafon on Friday December 20, 2019 @09:27AM (#21844337)
You can use something like Vault with the PKI backend and an audit engine set up. You will get a full trace of all certs delivered and an API endpoint to update them.
Re:Cisco devices must regenerate self-signed certificates be... by rb808 on Friday December 20, 2019 @11:16AM (#21845198)
Re:Cisco devices must regenerate self-signed certificates be... by allset_ on Friday December 20, 2019 @11:31AM (#21845306)
You should know your assets. If you know your assets, it's pretty trivial to write a script or program that checks when their certificate expires, sticks it in a database, and alerts you when it's getting close to time to renew. Of course, if you don't know your assets, you hopefully at least know your IP space and can scan that.
Re:Cisco devices must regenerate self-signed certificates be... by closeparen on Friday December 20, 2019 @06:06PM (#21848180)
Fully automated issuance and rotation on the order of hours. Use stuff like SPIFFE [0]. Won't help you much with 3rd-party proprietary hardware but you can at least avoid dealing with long-lived certificate management for your own software.

[0] https://spiffe.io/

Re:Cisco devices must regenerate self-signed certificates be... by tyingq on Friday December 20, 2019 @09:13AM (#21844218)
There is software like Venifi, but it requires everyone to follow a specific process. You still have outliers.
Re:Cisco devices must regenerate self-signed certificates be... by somurzakov on Friday December 20, 2019 @01:36PM (#21846386)
check Venafi for scanning your network for certs and managing them

you can also build scanner yourself on top of nmap and store discovered certs in one DB and link it with your configuration management db

Re:Cisco devices must regenerate self-signed certificates be... by Avamander on Saturday December 21, 2019 @05:05PM (#21853613)
Automate every device separately. I've written browser pupetteering scripts for that.
Re:Cisco devices must regenerate self-signed certificates be... by EncryptEntropy on Friday December 20, 2019 @12:20PM (#21845713)
Just had to emergency deal with this issue within a large scale complex ISP env. Fun times!

Edit: And yes as others pointed out, right into the day or two before holiday moratorium.

Re:Cisco devices must regenerate self-signed certificates be... by jodrellblank on Friday December 20, 2019 @04:47AM (#21842856)
I'm surprised there's no workaround "set the clock back to 2019, generate a certificate, then fix the clock time". Would that not work?
Re:Cisco devices must regenerate self-signed certificates be... by NohatCoder on Friday December 20, 2019 @04:58AM (#21842898)
The generated certificate would also expire in 2020, so it would be invalid immediately.
Re:Cisco devices must regenerate self-signed certificates be... by wyxuan on Friday December 20, 2019 @02:14AM (#21842243)
Correct me if I’m wrong, but is this just another measure by Cisco to block off the second hand market for the equipment?
Re:Cisco devices must regenerate self-signed certificates be... by oarsinsync on Friday December 20, 2019 @02:24AM (#21842277)
Doubtful. Cisco software code quality isn't the best, even in their specialist fields (see the number of BGP related bugs that are patched every year, still). This looks like just another stupid bug that has already been fixed in a future release.
Re:Cisco devices must regenerate self-signed certificates be... by tialaramex on Friday December 20, 2019 @03:45AM (#21842578)
Yup. They have lots of smart people, but something has been badly wrong on the code quality front for decades. I remember not long after the turn of the century doing inter-continental live debug sessions with their team because the multicast IPv6 acceleration (MLD snooping, like IGMP snooping but for IPv6) they'd shipped was completely broken in our Catalyst 3750s. As usual the authorised workaround was to disable acceleration, destroying the value added by Cisco's incredibly expensive product so I was on Transatlantic conference calls after hours trying to get them to reproduce the issue so they could actually ship us a fix.
Re:Cisco devices must regenerate self-signed certificates be... by olyjohn on Friday December 20, 2019 @11:39AM (#21845389)
My theory is that the business is more focused on getting contracts and sales, and getting in some useless magic quadrant than they are on actually providing a quality product.
Re:Cisco devices must regenerate self-signed certificates be... by mcpherrinm on Friday December 20, 2019 @02:38AM (#21842320)
You can just get a cert (from your corporate CA, or a public one, or just some one you spin up for the purpose) to fix this issue.

It's very likely just somebody harcoded a date "in the distant future" in some OpenSSL config file or script that generates the self-signed cert.

Re:Cisco devices must regenerate self-signed certificates be... by jlgaddis on Friday December 20, 2019 @05:26PM (#21847931)
Care to explain how you think this issue is even remotely related to the grey market for Cisco products?

I'm having trouble trying to come up with any link between the two.

---

In my opinion, with the exception of perhaps very small networks, no one who's "doing it right" should really be affected by this issue.

I can kinda maybe understand folks using self-signed certificates for VPNs but, really, if you're doing much at all with certificates you should either be running your own internal PKI and/or getting certificates by somewhere else.

My previous employer had literally hundreds of devices using their own self-signed certificates (for device management) on them. Many of them were the "default" certificates that shipped on the devices, so it was the same certificate shared across hundreds of devices. As someone who gives a shit about security, that drove me absolutely nuts but no one else gave a damn. (Even worse, since nearly all of these devices used the same credentials, you'd only need to MITM one of them in order to "pwn" damn near everything on the entire network!)

Re:Cisco devices must regenerate self-signed certificates be... by mavhc on Friday December 20, 2019 @04:21AM (#21842748)
The certificates on the Cisco WLC only lasted 10 years, you have to upgrade the WLC and all the access points to a new version that includes a command to turn off the certificate check. If you want to upgrade an AP after that date you need to set back the clock first, so much fun
Re:Cisco devices must regenerate self-signed certificates be... by tyingq on Friday December 20, 2019 @10:01AM (#21844618)
They seem to be highlighting SIP/VOIP as the main issue. And downplaying ssh access into the router based on an assumption that most don't use x509 certs for ssh. Does that sound right?
Re:Cisco devices must regenerate self-signed certificates be... by tialaramex on Friday December 20, 2019 @07:33PM (#21848677)
Also, you aren't affected if you actually have a CA. This only blows up if you use the Cisco device's own self-signed certs which it always mints (in affected versions of IOS) with notAfter 2020-01-01.

For stuff like web tools that might happen by least effort, but for SSH it's a bunch of extra effort, 90% of the effort of doing it properly, yet with almost none of the security benefits. Anybody who cares will use their own CA and be unaffected, anybody who doesn't care never enabled certs for SSH on the Cisco boxes.

Re:Cisco devices must regenerate self-signed certificates be... by jlgaddis on Friday December 20, 2019 @05:36PM (#21847985)
Yes, most don't use X.509 for SSH (note: public key authentication is not the same thing).
Re:Cisco devices must regenerate self-signed certificates be... by allset_ on Friday December 20, 2019 @11:32AM (#21845319)
Most SSH access doesn't use x.509, so yes that sounds right.
Re:Cisco devices must regenerate self-signed certificates be... by loriverkutya on Friday December 20, 2019 @05:34AM (#21843085)
I still remember the time when 1st of January 2020 was so far in the future that it seemed it will never happen.
Re:Cisco devices must regenerate self-signed certificates be... by iforgotpassword on Friday December 20, 2019 @07:20AM (#21843582)
I did data recovery on an external drive in 2011. It was pretty bad but I only needed a dozen files or so, rest was backed up. One of the files restored had a borked creation time in May 2020. For some reason I never fixed that, so whenever I accessed that folder and sort by date, that file would be at the top. This is will stop happening soon. I grew so fond of this quirk that I'm almost sad.
Re:Cisco devices must regenerate self-signed certificates be... by exikyut on Saturday December 21, 2019 @11:49AM (#21852060)

    touch -d 'Jan 01 2030'
I initially thought 2099 but 2030 lets you poke it again in ten years :>
Re:Cisco devices must regenerate self-signed certificates be... by mtmail on Friday December 20, 2019 @10:43AM (#21844944)
We keep telling us the same about https://en.wikipedia.org/wiki/Year_2038_problem
Re:Cisco devices must regenerate self-signed certificates be... by croon on Friday December 20, 2019 @06:54AM (#21843464)
I've been booking meetings for 2020 and it still looks weird.
Re:Cisco devices must regenerate self-signed certificates be... by jcadam on Friday December 20, 2019 @07:20AM (#21843579)
It's probably going to take me about 2 months after the new year stop writing/entering "2019" on forms. And checks. And anything else that calls for a date.

Pretty much the same thing happens to me every year.

Re:Cisco devices must regenerate self-signed certificates be... by rocqua on Friday December 20, 2019 @08:40AM (#21844005)
I recently decided to switched to a 36 hour week. I accidentally said I wanted the switch to start on January first 2019 rather than 2020...

Things seem to have been fixed, but it was a scary mistake to make.

Re:Cisco devices must regenerate self-signed certificates be... by empath75 on Friday December 20, 2019 @09:12AM (#21844214)
Re:Cisco devices must regenerate self-signed certificates be... by theandrewbailey on Friday December 20, 2019 @09:45AM (#21844487)
I guess that what happens when we get old. We're going to have to push back that shiny future to 2050 (at least).