I’ve seen a horror story in the form of “[HELP] sysadmin wont give encryption keys” post to a forum. Mid-2000s, non-English so I’m paraphrasing.

The sysadmin resigned, no tech person anymore. The boss got a lost password inquiry from a customer, panicked, called former admin for help, not a single word he understood, hit the forum.

And in the forum post he said the admin “wouldn’t cooperate”, “very adamant and confrontational” about “giving out encryption keys” to “decrypt valuable customer database we can’t afford to rebuild but might have to” and how it’s driving the company towards bankruptcy over last few days etc etc.

Eventually forum members realized he’s talking about a password reset, and convinced him it’s actually impossible to recover hashed password nor it’s necessary which is the point, you just verify identity and let them change the password etc etc, but it took hours for the community, on top of a day and half or so of tense hours for the boss guy’s company, to just understand the situation and act accordingly.

So I think it might be worth communicating how passwords work every once in a while.

I mean you could also pay someone who is educated to handle the situation

"Our former system administrator is extremely uncooperative and very confrontational" or "we need to decrypt valuable customer data that we can't afford to rebuild and this is driving us towards bankruptcy" are the kind of words that make most educated people very much unwilling to help, no matter how much money you promise them ;).

It's not a matter of principle, it's a matter of headaches. It's very unlikely that a customer like that would pay enough to make it worthwhile.

There’s plenty of people willing to work at any price point. Just because a $200/hr consultant dismisses the request, a $10/hr one can probably still figure it out.

However if the alternative is bankruptcy I think there IS money to be spent on the solution.

That is also assuming you have the time, the resources, and the general knowledge to pick out a great contractor whom you can trust with the security and safety of a critical system.

^ This. When someone approaches you saying things like "our former sysadmin is confrontational and unhelpful", you don't just have to solve whatever problem the guy who ragequit can't solve. You also have to undo the damage made by the asshole who's contacting you after they tried to solve it themselves, and sometimes by the one or two unqualified people they tried to hire afterwards.

You know how employers are skeptical to hire people who start the interview by saying how awful their last/current workplace was/is? Same here. I've definitely met hundreds of developers or sysadmins that I wouldn't trust, and whose services I wouldn't recommend. I've definitely been wrong and hired some of them myself. But I would never say something like that in public. At the end of the day, who's the smartass that hired these people in the first place? I get it's frustrating but such is life, you still have to be professional about it.

Plus, 200 USD/hour for a quick fix looks like a great deal, but even if the fix itself takes one hour, it's rarely a one-hour job with customers like these. You routinely end up invoicing for less time than you spend deflecting their attempts to negotiate the bill, explaining why it took one hour and itemizing it, and getting them to pay the stupid invoice.

Except when you hit a website that refuses your auto-generated strong random password, which happens to me with shocking regularity.

My favorite is when the website accepts your long random password but then the login fails. Because the set password function truncated the password before hashing it but the login function doesn't do that.

Indian NPS, equivalent of personal pension account, government operated, website forces you to change password every 90 days. Accepts any length for new password. Silently truncates it & then hash & store. Now, you are locked out because you dont know what was the actual truncated password they stored.

Happened to me two times, changed password, locked out, had to reset. then while logging in, I noticed their requirement of 12 characters, at next reset used 12 characters, & did not get locked out.

Yes, but that would be OK if the login form truncated the password in the same way as the password change form. They'd both arrive at the same hash. Which is what the comment above you said. What you describe probably means the password change form actually did run the hash on the full length, while the login form truncated it.

This is definitely not OK because it reduces the entropy of the passwords without the user knowing this has happened.

I never said it's OK, I just explained what's happening.

Ah, now I understand why I got those downvotes. What I meant by "OK" was that it would "work". It wouldn't exhibit the behavior my parent comment was describing. Not that it would be secure or good practice.

Ah that makes more sense, thanks for clarifying.

No, login form did not truncated, it showed javascript error toast that password is longer than acceptable.

The initial signup also checked & showed that toast. But the 90 day reset page did not.

This still happens, in 2020, with breathtaking regularity. Several times a year, a site won't accept my password, so I then start the whole "Will 20 characters work? No, that didn't work. Will 16? Nope... 15? Nope... 12? Nope. Oops I skipped 14... AH YES FINALLY."

One of the hats I wear these days is looking through source code for security vulnerabilities. It's shocking how many SQL injection vulnerabilities I find in newly written code. I remember first reading about SQL injection back in the 90s and yet developers are still making that very basic mistake. It is also a bit scary how many of the code analysist tools miss intentional flaws I've added to the code to test the scanner. These aren't ancient lint checkers, they're ridiculously priced enterprise tools. It worries me that they're giving a false sense of security that is going to get lots of implementations burned.

>These aren't ancient lint checkers, they're ridiculously priced enterprise tools. It worries me that they're giving a false sense of security that is going to get lots of implementations burned.

But no one is going to get fired or held back for choosing those 'enterprise' tools. They've spent a lot of money, everyone has a checklist, and people move on, even if there's a breach. And some jr might have pointed out "hey, this open source toolset does this, but is kept up to date and has found 47 things our scanner didn't find" and this jr will likely be ignored.

“Well with open source tools, you pay with time and we prefer this gold-plated, Cadillac, enterprise-grade tool. We’re a serious company now, after all.”

It’s maddening.

I'd argue it's bad API design. You can't make an API that's easy to use wrong and insecurely and then be surprised that people use it wrong and insecurely.

I can only half agree with you on that. Yes, I also dislike APIs that make wrong or unsafe use easy and correct use more bothersome but seemingly no different in behaviour (until it goes BOOM), but I also find that soooo many people simply don't have the awareness that they are interfacing with another system that interprets their data in a potentially unsafe way. And these people will misuse any API like this.

Unfortunately short of forcing everyone to use an ORM I don't see how we can block the unsafe API, which I'm assuming to be the string-based query interface e.g. `conn.query("SELECT * FROM users")` since any interface that accepts a string will allow a dynamically constructed string which lets developers open themselves to injection attacks. Only ORMs AFAIK can prevent this, e.g. db().users.all() or db().users.select(name="bob").

Maybe there's a clever trick I'm missing here.

It'd be nice if the languages offered a way for the query-compiling function to require that the query strings given to it are static, compile-time strings.

This will never change until engineers can be held criminally accountable for incompetence, particularly when personal information is involved.

Of course any proposal to ever hold a programmer accountable for literally anything is always unpopular on HN, for obvious reasons.

Because it's ridiculous to decide that the bottom of the decision graph should be held responsible for the poor decisions made from above.

Engineers should only be held accountable for decisions they made personally. The unfortunate reality is that, a terrifying amount of the time, terrible decisions are handed to engineering teams as required implementation details from managers, executives, product directors, etc. So should engineers be held criminally accountable for their product manager demanding MD5 hashes on passwords?

That sound you hear is millions of actual engineers, ones with qualifications/certifications/professional obligations/legal liability (think “civil”, “mechanical”, “electrical”, et al.) all rolling their eyes yet again at the code monkeys who call themselves “Engineers”, yet aren’t prepared to tell their boss that the decisions they’re trying to make, with inadequate training and experience, are dangerous...

Do you think the guy signing off on a bridge or power station design gets pushed around by “product managers” demanding shitty or cheapskate design or construction?

You’re welcome to disagree about calling them software engineers, but I’m not seeing how that is relevant or constructive to the whole point of this post and comment chain?

The point GP is making that civil engineers will not (knowingly) design an unsafe product just because they got something in a specification that led them to do so. They’ll object and refuse.

You were arguing that the specs or decisions made above their pay grade forced the software engineers into a dangerously faulty design which they dutifully created and shipped.

It's a bit circular, though, isn't it? It is in part due to the fact that a mechanical engineer _has_ that legal certification that they have the standing to meaningfully object, and actually be listened to by their PHB.

An engineer should be willing to tell his boss "No, because I would go to jail" If the manager insists anyway, the engineer should be obliged to refuse, even if that costs them their job.

Of course management should also be held accountable, but until engineers are forced to have some skin in the game they will continue to be as pliable as wet noodles.

Consider this: who better to blow the whistle on management than an engineer who knows they've been given an illegal order?

I have never seen an engineer who understood the issues be 'as pliable as wet noodles'.

What you're asking is for an engineer to be stuck in a place of legal culpability if they do the work, and to be fired if they don't. Added bonus: you mention whistle blowing, but who the hell would they whistle blow to?

If there's a proper industry or governmental group to blow the whistle to, that will also see the engineer financially compensated until such a time as they find a new job, then fine, it's fair to make engineers culpable. Otherwise, you're just making engineers suffer for the bad decisions of those made above them, by making them either legally liable, or risking their jobs.

Any employee who refuses an illegal order is risking their job. If your boss asked you to perform brain surgery and promised to fire you if you refused, would you give it your best try or would you tell him to fuck off? If software ''engineers'' want respect they need to grow a spine and learn how to say that simple two letter word.

Insecure websites aren't illegal. Only -breaches- are. And that only a civil offense, leading to a fine, the company pays. Right now, both the punishment and the decision are aligned.

You're looking to now make it so that there's a punishment levied on the developer, who has no more power to say no than they currently do. You want them to say no, but all you're doing is making it more unpleasant for them to not do so; you've done nothing to make it easier to do so.

> An engineer should be willing to tell his boss "No, because I would go to jail"

I guess we just fundamentally don't agree on how power dynamics effect these types of scenarios.

I agree with the GP, but I also think it is poorly phrased. It's not that the engineers should be willing to say they'd go to jail, it's that they should be able to.

Right now, the conversation goes, management: "I want x". Engineer: "X is insecure, we should do y instead". M: "Y will cost us X more than x, and it's never going to matter for us, anyway."

This puts the engineer in a position where they need to argue and justify the cost. Compared to "Sorry, I can't do that; it's illegal and I'd go to jail if I did that and was found out." Now the engineer doesn't have to justify anything. The law isn't a burden on the engineer here, it's a shield.

Yes, there are still some scenarios in which management insists. In my limited experience, that's in gray scenarioa where it's arguable whether the law applies. But the point is, it's much easier for an engineer to argue whether the law applies, than whether it is worth the money.

I think you can see these effects in the lengths companies go to protect healthcare data (hipaa) vs any other random personal data.

So this is a reasonable argument, but before we get there, there should be real responsibility placed on the company. Currently, when use data is lost, the companies suffer absolutely no consequences. They have literally no incentive to make sure their system is secure.

The equasion should be, for management: "Y will cost us X more than x, but if X is hacked we will get taken to the cleaners"

For the actual engineer to be held responsible you would have to add a formalised approval process, so that its clear who signed what off. Inagine you signed off on something, and then changes were made without your knowledge - that much easier to do with software than a bridge.

I guess my point is not that engineers should be exempt from being held accountable for their work, but that engineers are frequently asked to do things incorrectly/poorly/negligently and then assessed by their employers for their willingness to comply. Sure, you can say "Just stand up to your employer", but that's an incredibly dismissive stance on a complicated issue. Yes, you should say no to requirements that are flat out illegal, but is it ever that cut and dry? I'd be surprised to find it was ever that simple.

According to my engineering professional association, it is that simple. They'd have no qualms stripping me of my license to practice engineering for knowingly approving an unsafe design, regardless of what effect that decision would have had on my financial well-being.

The big difference is that companies need professional engineers. Professional approval on certain things is required by law. I'm not sure that would be a good idea for software, but that is what makes the system work for professional engineers.

> They'd have no qualms stripping me of my license to practice engineering for knowingly approving an unsafe design

Presumably there would be repercussions at the government level if a company repeatedly demanded engineers do things worthy of stripping their licenses though, no?

This varies by jurisdiction, but perhaps I oversimplified. There is another reason that doesn't happen.

The individual engineer doing the work needs a license, but the company itself also needs a permit to practice. The permit must be held by an engineer, who is personally responsible for the engineering that occurs under their permit.

So, the permit holder needs to worry not just about their own ethical behaviour, but that of all engineers in the company. They are incentivized to ensure the company will hold the public safety paramount, or to walk away if they cannot (thereby leaving the company without a permit).

If the company has a pattern of misbehaviour, it may be difficult to obtain a permit.

_Exactly_. Tech companies regularly exist purely based off of illegal business models (they call them _Disruptions_).

So, yes, we agree, if there are repercussions for a company regularly breaking the law, then engineers can and should refuse work that has negative legal or moral repercussions. But in the world of tech, that's not the case.

Wait a minute, I thought software engineers were in high demand, with companies fighting over them and opportunities everywhere! At least that's what one out of every 10 HN articles tells me. Surely that demand gives them a little power and agency over their work. I think we are pretending here that these developers have only one option: "Sure, boss! Whatever you say, boss!"

I personally believe that you, the software developer typing in the code, should hold yourself personally accountable for what you are typing in. You might also be designing what you type in, or even setting the requirements, but it might be other people. Regardless, you are making the software come into being--you're the one coding it and pushing it to the repo, so you should set the standard of what is acceptable. This "well, boss told me to do it!" rationalization and blame-shifting is how we get dangerous and unethical software.

And, yes, I have quit software jobs where I was asked to write software I considered ethically questionable, and failed to change the boss's mind.

> I think we are pretending here that these developers have only one option: "Sure, boss! Whatever you say, boss!"

No, I am suggesting that there is significantly more grey area between your moral highground and reality.

> I personally believe that you, the software developer typing in the code, should hold yourself personally accountable for what you are typing in

Yep.

> This "well, boss told me to do it!" rationalization and blame-shifting is how we get dangerous and unethical software.

It really is a strange world that, when corporations are attempting to turn profit on illegal behavior, it's the meaningless bodies-in-seats that we're trying to hold accountable.

I am, and have been, repeatedly, suggesting that the lowest level cannot be held accountable without holding the rest of the levels accountable. Jailing engineers for doing things their companies demanded of them is ridiculous if you're not also jailing those doing the demanding. I'm kind of shocked this isn't painfully obvious.

> And, yes, I have quit software jobs where I was asked to write software I considered ethically questionable, and failed to change the boss's mind.

Congratulations, that's a level of privilege lots can't afford.

The engineer should rather reply that the boss would go to jail.

Better if both would face repercussions, since if only the boss has skin in the game passive introverted engineers will silently follow orders, knowing that they're not personally risking much if they comply but risk losing their jobs if they don't.

Of course a certified accountable login screen engineer with a personal defense lawyer and a group of analysts, who make you a form:

  Please login
  Username: __________
  Password: __________

for $5.7M a year is always unpopular in business circles, for obvious reasons.

The mandatory GDPR training session at my current place in Berlin said that we could be personally liable with regard to personal information.

(Not that I can read German to the standard required for understanding laws and looking it up for myself; I can just about manage tabloid newspapers…)

My understanding is bcrypt truncates at 72 bytes by default, which could be 18 emojis I guess?

It should still be consistent with registration/login with things getting truncated, but I think this is also the default in PHP if you are using password_hash() today. Is that a security issue?

> Although implementing a maximum password length does reduce the possible keyspace for passwords, a limit of 64 characters still leaves a key space of at least 2^420, which is completely infeasible for an attacker to break. As such, it does not represent a meaningful reduction in security.

https://cheatsheetseries.owasp.org/cheatsheets/Password_Stor...

> My understanding is bcrypt truncates at 72 bytes by default

I don’t think it’s “a default”, it’s a fundamental limitation of the algorithm.

Specific bcrypt libraries could implement length-reduction by default though.

"Don't roll your own crypto" should also cover password management.

Oh this... I had something weirdly different recently. On a digital ocean account, all of a sudden I couldn't log in. Apparently the login page was updated and said my email adres was invalid. Which is weird cause I definitely logged in before. I used the + trick with Gmail, so I thought this was the issue, icw. new login page. Wasn't the case. Apparently my DNS service which also does filtering, blocked part of their CDN, also part of the email validation javascript. Turned that off, could log in again.

PayPal has this problem but only in certain change password forms. I think it was a password reset form. this happened to me recently. It was incredibly frustrating.

So that's what it was... god damn it

I can't believe sites think it's acceptable to do that. I get it, algorithms like Bcrypt have a size limit. But there are reasonably secure ways to get around that, for instance using HMAC to Sha256 the password before Bcrypting it.

Or better yet, pick one of the other algos with better limits and protection like Argon2 or Scrypt.

I had this happen recently with a finance website! Although technically the reverse, it silently stopped logging me in because the password field was changed to use HTML validation to enforce a max length of N, but they had previously accepted my password that was length N+1. Maddening.

I think FF recently deployed a change so that it no longer silently truncates inputs that are too long. It won’t solve the problem in all cases but hopefully in some.

The reason this is a problem even for new applications [1] is that it's the result of a leaky abstraction in common password hashing libraries which forces the site operator into a tradeoff between login service reliability/consistency and security.

Iterated password hashing algorithms like bcrypt [2] use a parameter tuneable by the site operator that drastically varies the computational cost of calculating the hash, so a brute force attack on the hash could be required to use orders of magnitude more computational power to crack. The tradeoff is that it will make users wait longer to log in, and (might) incur additional operational cost to the site. This is why it's a tuneable parameter: Up increases security at the cost of user delay and operations.

In theory. In practice, the user delay and operational cost are also proportional to the user's password length, which you don't control. Observe, the leak. This can cause wildly varying response times for authentication attempts, and opens your authentication system up to a DoS attack. The only reasonable solution is to put a small cap on password length so that the longest passwords don't take more than 2-5 times as long to compute, as per each site's tolerance of variance.

So why does the runtime of the iterated hash function depend on the length of the password? In each iteration the user's password is joined to the previous iteration's hash, and hashed together to get this iteration's hash. The runtime of any hash function is proportional to the length of the data fed in, the data fed in includes the user's password, thus the runtime of an iteration is proportional to the length of the user's password. So the runtime of the whole iterated password hash is `O(Iterations * PasswordLength)`.

There are various schemes one could come up with to change the time to `O(Iterations + PasswordLength)` ~ `O(Iterations)`, such as concatenating the first hash of the user's password instead of the password itself on all successive iterations, so all iterations but the first are independent of the user-chosen password length. There could be some security/entropy-based arguments for avoiding this solution, though I don't know what that could be.

[1]: https://news.ycombinator.com/item?id=22749706

Um...you're missing the point.

It's fine to truncate the password (though ideally you don't do it at a super short length).

The issue you are replying to is referring to when a site doesn't do it for both account creation and login. Which is a bug. A stupid, hard to detect, hard to explain, likely never to be fixed, bug. That only affects people trying to be secure.

I get that different parts of a service truncating the password to different lengths is a problem, and that it's a different (perhaps worse) problem than the one that causes site operators to limit password lengths.

Your password must contain at least one special character. Except !, that isn’t allowed.

Worst think about 1password, and lastpass when I uses it, it doesn't let you pick what special characters it uses, despite it being such a common thing on websites. So you have to manually add them, or swap out thing.

Surprising since Keepass has had that for years

Enpass’ password generator has a field were you can enter characters not allowed.

Regretfully Enpass doesn’t store this field nor the rest of the complexity rule as part of the password entry. The next time when password has to be changed you have to figure out the underlying complexity rule again.

I just add a random thing to the end of the password that fits the criteria

Bitwarden let's you choose

The old “abchkkunenukzimejienejsidmdjiwknevgjk bgiknhhhnnisplwkslandhgabsndmskalpaapowhsoslxiaiapjsbsnsnaja” is not secure, but “P@55w0rd” is super duper secure.

in our app we have a requirement that is similar.. I kicked and screamed and sent them spec documents from the NIST.. no one cared.. we have a max length of 10 chars... that SERIOUSLY hurts... 8 and 10 chars are our current requirements... plus some combination of numbers and special chars... WTFBBQ !!!!111...

hypothetically it's "ok" but c'mon..

I mean, if you insist on a 10-char maximum, then mandating symbols to increase the search space is a good idea, right? (Granted, that doesn't make a 10-char max sane)

Allowing symbols increases the search space, but requiring them reduces it.

And in practice this effect can be exaggerated when people don't use random passwords but must actually choose a password they'll remember - because what they'll actually do is choose something easy and then shove a symbol in there to meet your requirement. You may well allow 30+ different symbols, or even more, but the users will invariably pick one of a dozen or so that were easiest to reach on their keyboard and they may learn to be shy of characters that sometimes "don't work" such as quote marks and any local currency symbol even if those are easy to type.

Could be worse.

They might write it on a flipboard next to a window.

https://grahamcluley.com/plymouth-passport-offices-pitiful-p...

The scariest one is when ' is not allowed.

Must include 1 special character, except for the following: ;`'"-

"Our client-side Javascript should be enough to prevent any SQL injection attempts" /s

My favorite one is when it silently removes those characters but doesn't tell you...

Also, the chance that a site will refuse a strong random password is directly correlated to the importance of the account. Case and point, the appointment site for my barber happily takes a 30+ character randomly generated password. My old bank would not allow you to have a password longer than 12 characters and only recognized 4 special characters. That is why they are my old bank.

This is less annoying than sites that accepts your strong password, except their backend actually couldn't handle it. I love being locked out immediately on account creation.

Password!Drop!table

You're not even seeing all the sites that truncate your password down to 6 characters.

Hi Westpac.

That's the worst, especially when the truncation is silent. Boo!

Little enrages me more, especially when the forbidden special characters betray a shockingly backwards mitigation for injection attacks.

[1] https://benhoyt.com/writings/dont-sanitize-do-escape/

Or where their fancy SPA custom animated UX means your password manager can't auto insert a password.

I've had that happen before, but even if you have to degrade your password, at least it is random and not associated with any other password/account.

But yes, at that point all you can do is either complain to the website purveyor or vote with your feet (if the latter is an option).

How can you kick the server if you're not in the datacenter?

Many password generators have options for such things though, like configurable length, character sets, etc.

If a person ever even considers writing code which could generate either of these error messages:

"Your password is too long!"

"Your password uses special characters!"

... they are not only incompetent to write code which handles passwords, they have been so misinformed that they are an outright liability. They need to relearn everything they currently know on the subject and start over. Few things in the technical world instantly convey such utter anti-knowledge as the presence of either or both of those error messages in a codebase.

Some passwords are too long - I wouldn’t expect any website to accept a gigabyte long password - and I wouldn’t judge a site that doesn’t accepts \0 either.

I don't remember the math on hashing/bcrypt but isn't this the case that all passwords sort of hash to a fixed length string? Like why even have something like "your bank password must be 8-12 characters" long.

Obviously for a gigabyte long it's a bandwidth and hash-computing issue :p

> Obviously for a gigabyte long it's a bandwidth and hash-computing issue :p

Yes, that’s why you put in limits which are way beyond reasonable passwords but way below that. Say a few hundred or thousand bytes.

Also worth consideration: most of these work on bytes, probably utf8. A user wants to be cute and put emoji in there, that’s 4 bytes a pop. So depending how the system counts them, “hospital plane” might be considered 2, 4 or 8 characters.

But wait! Group emoji are concatenation combinations thereof, you can have a single multi-character emoji which is composed of half a dozen codepoints, and two dozen bytes once encoded.

bcrypt's input is limited to 72 characters in most implementations.

https://cheatsheetseries.owasp.org/cheatsheets/Password_Stor...

If that’s true, can you please share a link to your website so that I can stop using Dropbox and migrate my encoded data to be stored in your password field?

They don't encrypt your password, they hash it.

Hrm, yes, good point. So my snarky comment loses its charm, if it ever had any. Still, though, I think it’s reasonable to alert the user if your password exceeds its allocated storage rather than silently truncate.

bcrypt has a maximum length of 72 characters and its what Mozilla recommends for encryption. Do you think Mozilla is too "incompetent to write code which handles passwords" and "an outright liability"?

I think the idea behind the article was to explain why people are encouraged to stop using passwords from Site A if Site A gets 'hacked,' even on Sites B, C, D, etc. It's not just that encryption was broken on one site. It's that now that password has been matched up to a hash, so its strength is kaput.

I do think the message got a little lost in the post though, with the follow-on discussions of salting, bcrypt vs. md5, etc.

But hasn't the password + salt been matched up, not just the password? Or is his point that you don't know, so you need to treat it like the password is compromised (which I agree with). But the password is compromised either way (hash or encryption). I'm confused, maybe I need to go back and read it again.

Compromising the hash and salt, since they must be stored close together, makes it possible to identify if the salted hash is a password in a corpus of previously compromised passwords. An attacker can do Hash(PW, Salt) for all PW in a list of leaked/cracked plaintext passwords. If they've guessed your password and it's shared across multiple services, lateral compromise. Salting only prevents the rainbow table attacks, where an attacker precomputes all possible hash values for a known keyspace (like, say, 8 alphanumeric length passwords) and just look up for a match. Encryption is concerning because it necessitates the ability to decrypt since they're often inverse operations of each other, and presumably there's a shared key stored somewhere to do the comparison, which means it's likely trivial to recover the password compared to hash cracking and undermines any strength or complexity benefits. This also likely points to other bad behaviors utilizing this "feature", such as helpfully emailing you your plaintext password when you forget it.

Rainbow Tables are a specific innovation in time-space tradeoffs (precomputation) rather than the name for all such attacks.

The specific clever trick in Rainbow Tables is the observation that rather than storing

  hash(password) : password
  5f4dcc : password
  c2fe67 : jimmy
  25d55a : 12345678

... we can build a function that takes the output from hash(password) to deterministically create a new candidate password, let's call this function pass(hash), and then chain the hash and our new function together as many times as we want. This lets us store much less data, while doing more work during our look-up phase.

  hash(pass(hash(password))) : password
  153dfc : password
  92fe87 : jimmy
  213eea : 12345678

Now if I find a hash 92fe87 in a password hash file, I do not learn that the password was jimmy, instead I need to compute pass(hash(jimmy)) and that's the password I was looking for. And if I find 39a4e6 which isn't in my list, I calculate hash(pass(39a4e6)) and discover that's 213eea, then I look this up in the table and I discover the password I need was 12345678. Obviously real Rainbow Tables don't just run the hash twice like this, but instead some fixed number of times chosen by the creator to trade off less space versus more work to find a password.

I should actually fix this. What I've described above is basic "chaining", but Rainbow Tables are a further improvement still by Philippe Oechslin. The additional insight in Rainbow Tables is that we can reduce collisions in our hash-pass-hash-pass back and forth if we modify that pass function so that its behaviour varies by depth, this way if a collision occurs but at different depths in different chains (e.g. maybe the chain starting with password "password" hashes immediately to 5f4dcc but in another chain the value 5f4dcc is found for the password "j58X_m04" after six steps) the next call to pass() will diverge again, so the collision only wastes a small fraction of our precomputation effort. If the collision does happen at the same place in the chain, the final hash output will be identical to another chain, so it's easy to discover this problem and apply whichever mitigation seems appropriate.

Interesting, I haven't worked with rainbow tables very much since by the time I got into the world of hash cracking it had either been deprecated by salting or wasn't relevant (i.e. NTLM). That is a clever trick of trading back some of the space for extra time; I remember some of the rainbow table file sizes being ridiculous to the point of almost unusable haha.

edit: spelling

If one uses bcrypt for hashing passwords, as currently the best practice recommends, building basically a salted rainbow table becomes rather expensive, too. Not impossible, since the amortized cost for many common passwords is relatively low, but still sort of expensive.

Ideally a machine that generates and checks the hashes should be a box without a NIC, connected to the rest of the servers via a bunch of RS-232 ports. This would make extracting the salt much harder, down to effectively impossible. Few orgs can afford such a setup, though, due to the hassle of administering it.

> Not impossible, since the amortized cost for many common passwords is relatively low, but still sort of expensive.

This statement seems like it gravely underplays the numbers.

Traditional Unix crypt uses a 12-bit salt. So this means your precomputation (whether a Rainbow Table or not) is 4096 times more expensive. That's just about plausible though already uncomfortable ("Sorry boss, I know you said the budget was $10 but I actually spent forty thousand dollars").

But bcrypt uses a 128-bit salt. So now your precomputation is so much more expensive that if the equivalent ordinary brute force attack on a single password cost 1¢ and took one second on one machine, you'd spend a billion dollars per second, over a billion seconds, on each of a billion machines, and still not even have scratched the surface of the extra work you've incurred to do your precomputation.

Or to put it another way: Impossible in practice.

I see! But the case I'm arguing about is when the salt is known, pilfered along the password hashes during a compromise.

So what are you precomputing?

Rainbow Tables are a precomputation "time-space tradeoff" attack. You do a bunch of preparatory work which is amortized over multiple attacks and results in needing space to store all your pre-computed data. This is nice for two reasons:

1. You get to do all the hard work before your attack, leaving less time between the attack and your successful acquisition of the passwords compared to work that's necessarily done after stealing the credential database.

2. You can re-use this work in other attacks

But if you're waiting until you know the salt you don't get either of these advantages, so Rainbow Tables are irrelevant.

It's like if somebody mentions the F-14 fighter jet in a discussion about the fastest way to get from Times Square to Trump Tower. Yes the F-14 fighter jet is a fast aeroplane, but it can't go to either of those places so it isn't relevant whereas Usain Bolt is a very fast human so he really could run from one to the other.

People should be encouraged to stop using non-unique passwords for anything they remotely care about. Password managers are awesome.

It does make logins on other machines difficult. You may open up your password manager's web interface on your friends laptop, but then you opened up all your passwords to potential malware,including banks etc. When you just wanted to log in to Reddit, where you have an account you somewhat care about: you spent years under that username, but its not a huge catastrophe if it's taken away, unlike, say, your domain registrations, your email password, your e-government logins, banks etc. The "all eggs in one basket" still bothers me.

We haven’t lived through a big password manager leak yet. But a password manager gives the list of usernames, passwords and websites. This kind of compromission is probably worth a million dollars per user. That’s a lot of data in the same vault. Just saying.

KeepassXC is the kind that can not leak, giving passwords to third party is insane.

Mist people arent worth a million dollars, that downright silly.

But i do agree that centralised storage of everyone's passwords does not make sence.

I've decided to use KeePass and store the encrypted password file in cloud storage - that way it syncs between my phone and conouters, and i can always get at it if i really need to. But it does make logging into reddit on someone else's oc very difficult.

The problem is not that the term is imprecise. To say "We can securely check your password." would be imprecise.

The problem is that the term is _inaccurate_ because it is _overly precise_.

---------------

When you find yourself arguing over details you find pedantic, then it is a sign that you either:

A. Are failing to realize why some detail actually _does_ matter in that context.

B. Are using overly-precise terms and should see if you can "go up one abstraction layer", likely by asking "why does someone care?"

The knowledge it gains the end user is the confidence that the service I am using actually knows what they are doing. I have often found that if someone uses imprecise terminology, it is a symptom of a bigger issue.

> I have often found that if someone uses imprecise terminology, it is a symptom of a bigger issue.

What a spot-on observation. Maybe this is why I am always so pedantic, even when I tell myself that it's not necessary in said context. Subconsciously, I'm implementing this connection, and I can't turn it off.

relevant MCU quotation, from Civil War,

[ROGERS/CAPT] Sometimes... I wish I could.

[STARK] No you don't.

[CAPT] Yeah, you're right - I don't.

I've found it enormously valuable at the beginning of projects to hash out (erm...) a glossary so we know that everybody has the same understanding of the terms we use.

As an end user who uses Chrome and who's mum does too it means you can tell her to use Chrome's suggested secure password rather than 'P@ssword' or similar.

And my mum who's not really techie does care because she gets emails from Yahoo and the like saying sorry we've been hacked yet again please reset your passwords and gets in a panic over it.

I care as non technical person to know the difference between encryption and hashing. And to know why easy passwords are useless. This was an extremely informative article.

> I get he's ranting about imprecise terminology, but really, what does this knowledge gain the end user?

Reminds me of an experience I had in college. I was working phones over the summer at a call center and had this software developer call in. This was during the runup to Y2K, and I mentioned the "Y2K bug" and he starts ranting about how it's not a bug.

And I'm like, yes I understand it's not technically a bug, but most non-technical people don't know the difference.

I've never understood why people are like this with non-technical folks. If you're having a technical discussion with another technical person, then yes, the distinction absolutely matters. But to non-technical people? Who exactly benefits from that?

I agree, to an extent. Yes, the average person isn’t going to care that, for example, (original) Doom and Marathon are 2.5D engines rather than 3D engines, nor the technical differences between CDs, DVDs, or Blu-ray.

On the other hand, the general water-level of knowledge is frustratingly low at times: I’ve received customer support which insisted that they couldn’t register proof of identity to activate a SIM card for an iPhone, only for Android; a salesperson saying iPhones couldn’t be used in those cheap strap-a-phone-to-your-face 3D headsets; and a previous boss had to tell off one of their own customer support team for saying their product didn’t support Firefox when the only browser installed on the customer support team’s computers was Firefox.