Actually Gleam somewhat shares this view, it doesn't pretend that you can do typesafe distributed message passing (and it doesn't fall into the decades-running trap of trying to solve this). Distributed computing in Gleam would involve handling dynamic messages the same way handling any other response from outside the system is done.
This is a bit more boilerplate-y but imo it's preferable to the other two options of pretending its type safe or not existing.
I might give it a look this weekend.
the dynamic messages have to have static properties to be relevant for the receiving program, the properties are known upfront, and there's no "decades-running trap of trying to solve this".
I’m not as certain. The fact that we’ve gone from ASN.1 to COBRA/SOAP to protobuf to Cap’n’web and all the million other items I didn’t list says something. The fact that, even given a very popular alternative in that list, or super tightly integrated RPC like sending terms between BEAMs, basic questions like “should optionality/absence be encoded differently than unset default values?” and “how should we encode forward compatibility?” have so may different and unsatisfactory answers says something.
Not as an appeal to authority or a blanket endorsement, but I think Fowler put it best: https://martinfowler.com/articles/distributed-objects-micros...
It absolutely is a decades old set of problems that have never been solved to the satisfaction of most users.
> It absolutely is a decades old set of problems that have never been solved to the satisfaction of most users.
ASN.1 wasn't in the same problem space with CORBA/DCOM, both CORBA and DCOM/OLE were heavily invested in a general-purpose non-domain-specific object model representation that would suppot arbitrary embeddings within an open-ended range of software. I suspect this is the unsolvable problem indeed, but I also believe that's not what you meant with your comment either, since all the known large-scale BEAM deployments (the comment I originally replied to implied BEAM deployments) operate within bounded domain spaces such as telecom and messaging, where distributed properties of the systems are known upfront: there are existing formats, protocols of exchange, and the finite number of valid interactions between entities/actors of the network, the embeddings are either non-existent or limited to a finite set of media such as static images, videos, maps, contacts etc. All of these can be encoded by a compile-time specification that gets published for all parties upfront.
> basic questions like “should optionality/absence be encoded differently than unset default values?”
However you like, any monoid would work here. I would argue that [a] and [] always win over (Option a) and especially over (Option [a]).
> and “how should we encode forward compatibility?”
If you'd like to learn if there's a spec-driven typed way of achieving that, you can start your research from this sample implementation atop json: https://github.com/typeable/schematic?tab=readme-ov-file#mig...
So it’s hard to see how types get in the way instead of being the ultimate toolset for shaping distributed communication protocols.
Types naively used can fall apart pretty easily. Suppose you have some data being sent in three chunks. Suppose you get chunk 1 and chunk 3 but chunk 2 arrives corrupted for whatever reason. What do you do? Do you reject the entire object since it doesn’t conform to the type spec? Maybe you do, maybe you don’t, or maybe you structure the type around it to handle that.
But let’s dissect that last suggestion; suppose I do modify the type to encode that. Suddenly pretty much every field more or less just because Maybe/Optional. Once everything is Optional, you don’t really have a “type” anymore, you have a a runtime check of the type everywhere. This isn’t radically different than regular dynamic typing.
There are more elaborate type systems that do encode these things better like session types, and I should clarify that I don’t think that those get in the way. I just think that stuff like the C type system or HM type systems stop being useful, because these type systems don’t have the best way to encode the non-determinism of distributed stuff.
You can of course ameliorate this somewhat with higher level protocols like HTTP, and once you get to that level types do map pretty well and you should use them. I just have mixed feelings for low-level network stuff.
Of course it’s different. You have a type that accurately reflects your domain/data model. Doing that helps to ensure you know to implement the necessary runtime checks, correctly. It can also help you avoid implementing a lot of superfluous runtime checks for conditions you don’t expect to handle (and to treat those conditions as invariant violations instead).
For things that are invariants, that’s also trivial to check against with `if(!isValid(obj)) throw Error`.
I like HM type systems a lot. I’ve given talks on type systems, I was working on trying to extend type systems to deal with these particular problems in grad school. This isn’t meant to a statements on types entirely. I am arguing that most systems don’t encode for a lot of uncertainty that you find when going over the network.
For ensuring bits don't get lost, you use protocols like TCP. For ensuring they don't silently flip on you, you use ECC.
Complaining that static types don't guard you against lost packets and bit flips is missing the point.
Again, it’s not that I have an issue with static types “not protecting you”, I am saying that you have to encode for this uncertainty regardless of the language you use. The way you typically encode for that uncertainty is to use an algebraic data type like Maybe or Optional. Checking against a Maybe for every field ends up being the same checks you would be doing with a dynamic language.
I don’t really feel the need to list out my full resume, but I do think it is very likely that I understand type systems better than you do.
Sure thing. Unless dev forgets to do (some of) these checks, or some code downstream changes and upstream checks become gibberish or insufficient.
Gleam is still new to me, but my experience writing parsers in Haskell and handling error cases succinctly through functors was such a pleasant departure from my experiences in languages that lack typeclasses, higher-kinded types, and the abstractions they allow.
The program flowed happily through my Eithers until it encountered an error, at which point that was raised with a nice summary.
Part of that was GHC extensions though they could easily be translated into boilerplate, and that only had to be done once per class.
Gleam will likely never live to that level of programmer joy; what excites me is that it’s trying to bring some of it to BEAM.
It’s more than likely your knowledge of type systems far exceeds mine—I’m frankly not the theory type. My love for them comes from having written code both ways, in C, Python, Lisp, and Haskell. Haskell’s types were such a boon, and it’s not the HM inference at all.
The type is a way to communicate (to the compiler, to other devs, to future you) that those are the expected invariants.
The check for invariants is trivial as you say. The value of types is in expressing what those invariants are in the first place.
IMO the right approach is just to parse everything into a known type at the point of ingress, and from there you can just deal with your language's native data structures.
Once you add distribution you have to encode for the fact that the network is terrible.
You absolutely can parse at ingress, but then there are issues with that. If the data you got is 3/4 good, but one field is corrupted, do you reject everything? Sometimes, but often Probably not, network calls are too expensive, so you encode that into the type with a Maybe. But of course any field could be corrupt so you have to encode lots of fields as Maybes. Suddenly you have reinvented dynamic typing but it’s LARPing as a static type system.
I get what your saying, but can't you have the same issue if instead you have 3 local threads that you need to get the objects from, one can throw an exception and you only receive 2, same problem
When you have to deal with large amounts of uncertainty, static types often reduce to a bunch of optionals, forcing you to null check every field. This is what you end up having to do with dynamic typing as well.
I don’t think types buy you much in cases with extreme uncertainty, and I think they create noise as a result.
It’s a potentially similar issue with threads as well, especially if you’re not sharing data between them, which has similar issues as a distributed app.
A difference is that it’s much cheaper to do retries within a single process compared to doing it over a network, so if something gets borked locally then a retry is (comparatively) free.
On one end, you write / generate / assume a deserialisator that checks whether incoming data satisfies all required invariants, eg all fields are present. On the other end, you specify a type that has all the required fields in required format.
If deserialisation fails to satisfy type requirements, it produces an error which you can handle by eg falling back to a different type, rejecting operation or re-requesting data.
If deserialisation doesn't fail – hooray, now you don't have to worry about uncertainty.
The important thing here is that uncertainty is contained in a very specific place. It's an uncertainty barrier, if you wish: before it there's raw data, after it it's either an error or valid data.
If you don't have a strict barrier like that – every place in the program has to deal with uncertainty.
So it's not necessarily about dynamic / static. It's about being able to set barriers that narrow down uncertainty, and growing number of assumptions. The good thing about ergonomic typing system is that it allows you to offload these assumptions from your mind by encoding them in the types and let compiler worry about it.
It's basically automatization of assumptions book keeping.
What the hell is really the alternative here? Do you just pretend your process can accept any kind of data, and just never do anything with it??
If you need an integer and you get a string, you just don't work. This has nothing to do with types. There's no solution here, it's just no thank you, error, panic, 500.
The "its only bits" thing makes no sense in the world of types. In the end its machine code, that humans never (in practice) write or read.
Within a single computer that’s easy because a single computer is generally well behaved and you’re not going to lose data and so yeah your type assumptions hold.
When you add distribution you cannot make as many assumptions, and as such you encode that into the type with a bunch of optionals. Once you have gotten everything into optionals, you’re effectively doing the same checks you’d be doing with a dynamic language everywhere anyway.
I feel like at that point, the types stop buying you very much, and your code doesn’t end up looking or acting significantly different than the equivalent dynamic code, and at that point I feel like the types are just noise.
I like HM type systems, I have written many applications in Haskell and Rust and F#, so it’s not like I think type systems are bad in general or anything. I just don’t think HM type systems encode this kind of uncertainty nicely.
You absolutely can make all assumptions relevant to the handling/dispatching logic expressed at type-level.
> and as such you encode that into the type with a bunch of optionals.
Not necessarily, it can be `Alternative f` of non-optional compound types that define the further actions downstream.
> Once you have gotten everything into optionals, you’re effectively doing the same checks you’d be doing with a dynamic language everywhere anyway.
Not true, your business logic can be dispatched based on a single pattern comparison of the result of the `Alternative f`.
Honestly, and I realize that this might get me a bit of flack here and that’s obviously fine, but I find type systems start losing utility with distributed applications. Ultimately everything being sent over the wire is just bits. The wire doesn’t care about monads or integers or characters or strings or functors, just 1’s and 0’s, and ultimately I feel like imposing a type system can often get in the way more than it helps. There’s so much weirdness and uncertainty associated with stuff going over the wire, and pretty types often don’t really capture that.
I haven’t tried Gleam yet, and I will give it a go, and it’s entirely possible it will change my opinion on this, so I am willing to have my mind changed.