maxtaco

Use extreme caution running arbitrary code on your machines, especially obfuscated code that tickles kernel bugs! (edited)

Backpointers to earlier epochs in append-only cryptographic data structures like key transparency logs. If the client last fetched epoch 1000, and the server reports the current epoch is 3000, the server can return log(2000) intermediate epochs, following skip pointers, to provide a hash chain from epoch 3000 to 1000.

https://github.com/foks-proj/go-foks/blob/main/lib/merkle/bi... https://github.com/foks-proj/go-foks/blob/main/lib/merkle/cl...

Amazing man, with many important contributions over a very long career. The Rabin Cryptosystem (like RSA, but with public exponent 2) is notable for two reasons. First, unlike RSA, it is provably as hard as "factorization" (as he would call it), and second, unlike RSA, it wasn't protected by patent.

I think in practice it doesn't work to deserialize only verified data. Snowpack has a mechanism for this but I found it impractical to require all use cases fit this form.

I'm not sure exactly what system you're describing, but I hope it doesn't involve hand-marshaling and unmarshaling of data structures. Your requirement (b) seems at odds with forward-compatibility. Lack of forward-compatibility complicates upgrades, especially in federated systems when you cannot expect all nodes to upgrade at once.

I might be biased, but it's been possible to write a sufficiently complex system (https://github.com/foks-proj/go-foks) without feeling "tortured." It's actually quite the opposite, the cleanest system I've programmed in for these use cases, and I've tried many over the last 25 years. Never am I guessing how to verify an object, I'm not sure how that follows.

I also think it's worth noting that the same mechanism works for MAC'ing, Encryption and prefixed hashing. Just today I came across this IDL code:

  struct ChunkNoncePayload @0xadba174b7e8dcc08 {
    id @0 : lib.FileID;
    offset @1 : lib.Offset;
    final @2 : Bool;
  }

And the following Go code for making a nonce for encrypting a chunk of a large file:

  pld := lcl.ChunkNoncePayload{
    Id:     fid,
    Offset: offset,
    Final:  isFinal,
  }
  hsh, err := core.PrefixedHash(&pld)
  if err != nil {
    return nil, err
  }
  var ret [24]byte
  copy(ret[:], (\*hsh)[0:24])
  return &ret, nil

As in signing, it's nice to know this nonce won't conflict with some other nonce for a different data type, and the domain separation in the IDL guarantees that.

Bingo!

It seems like in that PR, the fact that the OID wasn't checked is part of the problem. I think a better system wouldn't compile or would always fail to verify if the OID (domain separator) is wrong, and I think you'd get that behavior in the posted system, though admittedly I'm not certain.

Hi, post author here. Agree that the idea isn't tricky, but it seems like many systems still get it wrong, and there wasn't an available system that had all the necessary features. I've tried many of them over the years -- XDR, JSON, Msgpack, Protobufs. When I sat down to write FOKS using protobufs, I found myself writing down "Context Strings" in a separate text file. There was no place for them to go in the IDL. I had worked on other systems where the same strategy was employed. I got to thinking, whenever you need to write down important program details in something that isn't compiled into the program (in this case, the list of "context strings"), you are inviting potentially serious bugs due to the code and documentation drifting apart, and it means the libraries or tools are inadequate.

I think this system is nice because it gives you compile-time guarantees that you can't sign without a domain separator, and you can't reuse a domain separator by accident. Also, I like the idea of generating these things randomly, since it's faster and scales better than any other alternative I could think of. And it even scales into some world where lots of different projects are using this system and sharing the same private keys (not a very likely world, I grant you).

It should be possible to change the name of the type, and this happens often in practice. But type renames shouldn't break preexisting signatures. In this scheme you are free change the type name, and preexisting signatures still verify with new code -- of course as long as you never change the domain separator, which you never should do. Also you'd need to worry about two different projects reusing the same type name. Lastly, the transmission size in this scheme remains unaffected since the domain separators do not appear in the serialized data. Rather, both sides agree on it via the protocol specification.

I would say two problems with the asn.1 approach are: (1) it seems like too much cognitive overload for the OIDs to have semantic meaning, and it invites accidental reuse; I think it matters way more that the OIDs are unique, which randomness gets you without much effort; and (2) the OIDs aren't always serialized first, they are allowed to be inside the message, and there are failures that have resulted (https://nvd.nist.gov/vuln/detail/cve-2022-24771, https://nvd.nist.gov/vuln/detail/CVE-2025-12816)

(edit on where the OIDs can be, and added another CVE)

Also on the front page of HN right now is a job posting for Optery (YC W22). Seems like they are growing really fast.

An intended use case for FOKS (https://foks.pub) is to allow long-lived durable shared secrets between users and teams with key rotation when needed.

I think one could build something nice on top of FOKS (https://foks.pub).

No incremental fetch right now other than what postgresql provides by default. If you're hosting a FOKS server, there is important metadata to backup too.

The best docs for the KV store are in the white paper, Section 5.1. White paper is linked to from foks.pub

Postgres seemed to check all the boxes for me. I wanted to keep things very simple at first in terms of setup and ongoing management, so didn't introduce other storage backends for other parts of the system.

An attack that might be of concern with this configuration is the server suppressing updates to this JSON file, or showing different versions of the JSON file to different clients. What you're describing is pretty close to what FOKS is getting at with signature chains and Merkle Trees, but maybe it's overkill for this particular application.

I wonder if the policy you describe could be implemented as world-visible team with world-visible users. Others have commented on the need for something like this, so I think it should be pursued with high priority. What's slighlty fuzzy to me is how these totally world-viewable teams and users would interact with more closed-down users on other servers.

Thanks for these great questions!

- limiting users to delete/push/force; this is possible but I don't see how to cryptographically guarantee it. The server can't really help since it doesn't know what's a pack, index data block or ref. The clients can enforce this policy, but then it would be possible to make an evil client that skirts the policy. How much protection do you think you need?

- the server repo right now is implemented 100% as a postgresql DB, so yes, I think that means it's incremental backup-friendly? [1]

- e2ee git has trouble being as efficient as regular git since the server can't tell you which blocks it has; however, there are pretty good optimizations made using indices and packfiles, the white paper has more details, and I hope to write a blog on it soon.

- I'm not sure about the sqlite question. Is there a good way to backup sqlite incrementally over standard git? If not, then maybe the KV-store is better fit for this application.

I agree that git over E2EE is the best storage, even for things like PDFs and photos. Yeah, FOKS should be hostable with a very thin VPS. The storage needs will scale (n log n) as the number of users due to the Merkle Tree, but for small n, this is likely fine!

[1] https://www.postgresql.org/docs/17/continuous-archiving.html...

This would be a great application for us! We are not exactly there yet, for reasons of privacy. Right now, there is no way for alice@host to allow unauthenticated users to view her profile. But we can definitely allow this on a host-by-host basis. With this small change, I think your application fits very naturally.

I wonder, what sort of interface is right for you? A library to compile against or a CLI app to shell out to? If a library, which languages?

Easy multi-accounting is something that I hope we already have (`foks key switch` is pretty smooth). It's a feature I use a lot (I have a personal account on @foks.app and our company account is on @ne43.foks.cloud).

This is a great point and I thought a lot about this. This is the sort of thing that can be changed later if it's really a good idea, but I got to thinking that having non-local admins would mean more server-to-server communication and more server-to-server trust, and I was trying to avoid that.

Imagine alice@foo is an admin of bluejays@bar. One thing alice@foo will need to do is to make signed changes to bluejays@bar, when adding or removing members, let's say. Right now, the server at bar will check the validity of these signatures, that they were made with the alice@foo's latest key. So in other words, there would have to be some way for bar to authenticate to foo to allow bar to read alice's sigchain and to determine her latest key.

I was thinking that keeping foo and bar separated was a good idea both in terms of privilege separation and keeping the network simpler (which would in turn be good for uptime and would simplify software upgrades).

Thanks Danny! The Keybase team (not including me) deserves all the credit, I've been gone for over six months. It's a great team and I miss working with them.

Thanks! Would love to see a file sync app, an MLS-based chat (where the encryption key is essentially a combination of the keys output from MLS and the PTK from FOKS). Password managers. I think there's the potential for something like a Hashicorp-Vault-style server-side secret key material manager, but many details left to reader. Maybe a Skiff-style Google-docs clone? I think there are lot of potential directions to go in.

I'm not familiar with Radicle, but I'll check it out. For (1), consider the case of that server being hosted on AWS. Even though only members are authorized to SSH into it, the plaintext is still known to the cloud hardware, and can be exfiltrated that way. In FOKS, the server sees encrypted data only, so that attack is greatly mitigated. I would say that if the SSH server was hosted on one of the workstations of one of the team members, then the security advantages of FOKS would be much less.

The KV-Store and Git server are implemented as "applications" on top of the FOKS infrastructure, so they aren't coupled. They see a sequence of Per-Team-Keys (PTKs); they use the older ones for decryption and the newest for encryption. I'd really love to see all sorts of other applications built on top of FOKS but we might need to do some work as to nailing the right plugin architecture.

It's not as seamless. You need to first make a team, then invite (or add) that user into the team, and then use `foks kv put --team <your-team>`. One key difference is that in Keybase, all user's profiles were essentially world-readable. FOKS aims for more privacy by default, so in order to add Bob to your team, Bob has to first allow you view his sigchain, so you can learn his public keys.

The add vs invite distinction referred to above is because servers can choose different visibility policies. You can set up a server at foks.yourdomain.cc, and set it to "open-viewership", which means that any user can see any other user by default. If you and Bob are both on that host, you can add him to your team without his permission. But other hosts, like foks.app, do not work this way, and Bob has to authorize you to view him.

Correct! Remote members of the team get access to shared team keys, and the team's data, even though they don't have accounts on that server. Knowledge of the team key suffices to allow a remote user to authenticate and transfer (encrypted) data to and from the server.

There is very little server-to-server communication, which simplifies the design and software upgrades.

E2E-encrypted git. Keybase has KBFS, and FOKS has a poor man's equivalent, which is E2E-encrypted Key-value store.

Max here, author of FOKS. I find it interesting how much glue is required to perform basic cryptographic operations, even in 2025. Imagine a very simple idea like encrypting a secret with a YubiKey. If it's an important secret, that you really don't want to lose, then now you need a second YubiKey as a backup, in case the primary is lost or breaks. But now how do you encrypt and how do you rotate the primary out if needed? To the best of my understanding, there aren't great solutions short of a system like FOKS. If not FOKS, I really believe a system like it ought to exist, and it ought to be entirely open, so that arbitrary applications can be built on top of it without paying rent.

- Sorry about the outage yesterday. It lasted about 1 hour but service was promptly restored.

- As Zoom employees, our primary objective is now to bring the technology at play in Keybase to Zoom products.

- We are still making regular updates to Keybase. These updates consist primarily of bugfixes, security fixes, performance improvements and patches and updates to third party libraries. For instance, the app is currently undergoing a major rewrite to be compatible with recent versions of React and React Native. We are not currently pursuing any major new features. The paper trail for these updates is visible for all to see here: https://github.com/keybase/client/tags

- The current plan -- which can be inferred from our ongoing maintenance to the product -- is to keep Keybase running in a performant, usable, free, E2EE, high-security form for the foreseeable future. Should this plan change, we promise to give as much advance notice as possible.

Note that the implementation of EdDSA that the authors investigated (libgcrypt) is not a constant-time implementation. Better implementations are more likely to be safe.

See: https://news.ycombinator.com/item?id=21352821

Congratulations and thank you for posting this inspirational story! I became interested in programming 26 years ago after taking the same class you did. So thanks to David Malan and his forebearer Margo Seltzer for CS50.