News for hackers. Stuff that matters.
Re:Willow Protocol by wavemode on Wednesday January 17, 2024 @09:28AM (#39028133)
> Some questions in protocol design have no clear-cut answer. Should namespaces be identified via human-readable strings, or via the public keys of some digital signature scheme? That depends entirely on the use-case. To sidestep such questions, the Willow data model is generic over certain choices of parameters. You can instantiate Willow to use strings as the identifiers of namespaces, or you could have it use 256 bit integers, or urls, or iris scans, etc.

> This makes Willow a higher-order protocol: you supply a set of specific choices for its parameters, and in return you get a concrete protocol that you can then use. If different systems instantiate Willow with non-equal parameters, the results will not be interoperable, even though both systems use Willow.

Help me out here - isn't the point of a protocol that two independently developed systems don't have to agree on how to implement the protocol? What value does Willow have if two systems that both purport to be "Willow-compatible" aren't compatible with each another?

Re:Willow Protocol by cobertos on Wednesday January 17, 2024 @09:36AM (#39028239)
By having a more generic protocol on top, it allows the same tools to be used for different specific end results. So shared libraries and common debugging tools can benefit more use cases. You might even make a "higher-order" tool that can work with any willow data, at the cost of specific UI affordances that can be used when you know more about the underlying data.

This is sort of how ActivityPub is a thing, but it underpins multiple, sorta-but-not-really interoperable systems like Lemmy and Mastodon.

Re:Willow Protocol by Joker_vD on Wednesday January 17, 2024 @10:34AM (#39029118)
> What value does Willow have if two systems that both purport to be "Willow-compatible" aren't compatible with each another?

That you can claim to support Willow, or be Willow-compatible, without actually having to interoperate with your competitors. See e.g. the usage history of x509 in the nineties.

Re:Willow Protocol by layer8 on Wednesday January 17, 2024 @11:18AM (#39029734)
Protocols can have parameters. The protocol will be interoperable between parties who choose compatible parameters.

For example, SSH only works if both sides support and can agree on the same cryptographic algorithms, which is something that SSH is parametrized over.

Re:Willow Protocol by wavemode on Wednesday January 17, 2024 @11:36AM (#39030010)
Unless I missed it somewhere, Willow has no specified handshake procedure. So there's no standardized way for the two sides to come to an agreement on how to communicate. (Willow appears to be completely agnostic even of the data encoding used to communicate in the first place.)

In that sense it is even more high level and abstract than parameterized protocols like SSH.

Re:Willow Protocol by layer8 on Wednesday January 17, 2024 @11:44AM (#39030122)
True, it was an example of how “the same protocol” doesn’t necessarily imply compatibility. The algorithm negotiation procedure doesn’t guarantee that there will be an agreement, so it is somewhat secondary to the argument.
Re:Willow Protocol by sph on Wednesday January 17, 2024 @05:20PM (#39034388)
And I think it's a good idea. Take HTTP for example. It is much more than its protocol. Freeing it from those details means that what is valid today:

    GET / HTTP/1.1
    Host: example.com
Could have the same semantics in any format:

    { 
      "method": "GET",
      "version": "1.1",
      "headers": [{"name": "Host", "value": "example.com"}]
    }

Of course a server accepting the former won't be able to communicate with the latter, but that's just an implementation detail that Willow does not want to commit to at this stage, and does not make it any less complete. Just a bit impractical.
Re:Willow Protocol by sillysaurusx on Wednesday January 17, 2024 @06:28PM (#39035137)
The history of cryptography failures suggests that it’s better for a protocol to be opinionated and complete rather than open ended.
Re:Willow Protocol by ianburrell on Wednesday January 17, 2024 @09:08PM (#39036608)
Format agnostic layers are a good thing. HTTP2 depends on it with different encoding but same meaning. But have to choose some encoding for communication to happen. Otherwise it is a meta protocol.
Re:Willow Protocol by jcul on Wednesday January 17, 2024 @05:55PM (#39034777)
Not all protocols have to have a handshake to negotiate how they will communicate. Sometimes you just need to have both sides configured the same way. E.g. PTP clocks being in the same domain, having the same delay request mechanism etc.
Re:Willow Protocol by skissane on Wednesday January 17, 2024 @10:22PM (#39037298)
> Not all protocols have to have a handshake to negotiate how they will communicate. Sometimes you just need to have both sides configured the same way.

True, but the industry has been moving away from "you have to configure both ends" towards autonegotiation.

e.g. RS-232 you had to configure data rates (e.g. 9600 baud) and encoding (e.g. 8N1 - 8 data bits, no parity bit, 1 stop bit), and if you didn't have the same config at both ends, communication wouldn't happen. USB, the primary successor, determines stuff like that by negotiation. Or similarly, with Ethernet autonegotiation, you no longer have to worry about manually configuring speed/duplex on each end–which I can remember being a big drama 20 years ago.

Re:Willow Protocol by jcul on Thursday January 18, 2024 @02:08AM (#39038889)
Yeah, that is true. And even as I typed that I thought, PTP can be fairly hand off and auto negotiates most stuff. I guess E2E vs P2P it can't really because they can represent different physical infrastructure.

Probably not the best example, I just happened to have the PTP spec open at the time!

Re:Willow Protocol by whizzter on Wednesday January 17, 2024 @11:40AM (#39030070)
I think the idea here is that Willow is meant to handle "higher order" issues like encryption and the especially prickly sharing encrypted data within a more cooperative environment, so that application builders can focus on their more specific applications.

Say that I want to implement something Figma-like for designing drug-runner operations, Willow seems to be an excellent building block (Yes, the example is kinda out-there but it's meant to indicate that genericity intended here).

Re:Willow Protocol by conradev on Wednesday January 17, 2024 @01:26PM (#39031468)
I wish the web page came out and said what Willow actually provides and what is up to the developer.

As far as I can tell, this is primarily a crypographic specification, like Noise (http://www.noiseprotocol.org) except for a stateful key-value store instead of stateless connections

Re:Willow Protocol by binary132 on Wednesday January 17, 2024 @10:19AM (#39028880)
FWIW, HTTP allows subprotocol / backwards-compatible negotiation. Perhaps that could be similar to how implementations of this might need to cooperate.
Re:Willow Protocol by smaudet on Wednesday January 17, 2024 @10:15AM (#39028823)
No I think you are right, this isn't a protocol. Its a protocol generator...

"higher order" is some nonsense, and would make me shy away from using it...

Re:Willow Protocol by gray_-_wolf on Wednesday January 17, 2024 @10:26AM (#39028988)
It is unusual term in the area of protocols, but it seems understandable it tries to draw a parallel to higher order functions. So "some nonsense" might be a bit strong...
Re:Willow Protocol by smaudet on Wednesday January 17, 2024 @12:17PM (#39030551)
Protocols are useful, and as such the costs are high to using a bad one. Confused verbiage does not instill confidence that the authors know what they are doing, or that there is real benefit.

So, perhaps it is strong language, but I think it is a reasonable reaction.

Re:Willow Protocol by black_puppydog on Wednesday January 17, 2024 @10:38AM (#39029186)
Huh? I'd understand your phrase "protocol generator" to be the equivalent of "higher order protocol" in the same way that a "higher order function" can be seen as a "function generator"...
Re:Willow Protocol by smaudet on Wednesday January 17, 2024 @12:12PM (#39030499)
The difference would be that a function, order non-withstanding, is (a) callable (site). "Higher order", merely means one that takes/returns other functions, but they are still fundamentally, functions (callable sites).

A protocol has the property that it is implementation independent, but that it has a defined interface (i.e. it is immediately usable).

This is neither (not a defined interface, implementation dependent). If it doesn't share either property with a protocol, then you can't claim that it is truly a protocol, "higher order" or otherwise.

This confused verbiage is what should be cause for concern - note that I can claim a "higher order" protocol with JSON or gRPC - its all the basic building blocks for a protocol, just both sides need to implement the same stuff!

Except, neither JSON nor gRPC are crazy enough to claim to be a "higher order" protocol, which to me puts this in the rubbish bin of over-complicated technologies looking for a problem, like SOAP, JavaBeans, OSGi - all of these could also be claimed to be "higher order" protocols as well.

The term is meaningless, and so I assume, is this project.

Re:Willow Protocol by jazzyjackson on Wednesday January 17, 2024 @12:37PM (#39030840)
its wild to me that you would dismiss a project as meaningless because you can't immediately understand it
Re:Willow Protocol by smaudet on Wednesday January 17, 2024 @12:45PM (#39030939)
Oh I might read their specs: "Willow is a family of specifications:" https://willowprotocol.org/specs/index.html#specifications

This part looks useful.

But is it useful to be a "willow" family of protocols? Probably not.

Their claims on the front page are extraordinary. Extraordinary claims require extraordinary evidence, and heading the page with nonsense is not a good start.

Re:Willow Protocol by mplewis on Wednesday January 17, 2024 @02:42PM (#39032418)
Pretty rude to dismiss a project without reading any of its documentation.
Re:Willow Protocol by rklaehn on Wednesday January 17, 2024 @10:41AM (#39029220)
Think of it like const generics in languages like rust or C++.

You can make two data structures with a const parameter.

If the parameter is not the same, they are not compatible (not the same type). The parameter can be tuned according to the specific needs of the application.

Re:Willow Protocol by tenebrisalietum on Wednesday January 17, 2024 @03:41PM (#39033152)
Is IP useless because the other end might not support UDP or TCP?

More practically/less absurdly: is SSL useless because side A doesn't support the same ciphers as side B.

Perhaps a Willow negotiation protocol would be needed to reconcile, but it's a bad idea from a security perspective because it enables downgrade attacks.

Re:Willow Protocol by smaudet on Wednesday January 17, 2024 @04:28PM (#39033751)
Comparing apples and oranges.

A protocol is allowed to have presumptions, and then provide an interface.

Its not allowed to have no interface at all and no presumptions (A void).

Re:Willow Protocol by alex-mohr on Wednesday January 17, 2024 @06:37PM (#39035205)
Willow appears closer to a "Protocol Construction Kit" than a protocol itself.

As a construction kit, it has value for people who want to make protocols where they'll control both ends, but don't have to re-implement basic table stakes.