Let me turn this around: if you're saying possibly-missing fields inevitably propagate throughout the types in your codebase, it sounds like you're saying malformed input propagates arbitrarily deeply into your core logic. Wouldn't you rather catch that at the boundary?

Possibly. If it's a key part of the application, like Credit Card processing or auth - absolutely. If it's simply auxiliary or supplemental information - probably not. The thing for me is the type system shouldn't decide whether or not a certain piece of information is critical to the system. That typically falls to deeper logic in the system. If the type system isn't deciding, then it's pretty much meaningless (as everything would be optional).

For example, I'm using a Stripe JS integration to allow customers to manage credit cards in app. Stripe messes up or changes something without me paying attention. Instead of returning a complete object, Stripe starts returning only the credit card ID.

This isn't a big deal if we simply need to know that a record exists. We can still tell the user has a card and we have an ID to attempt a transaction against server side. We might not be able to display details about the card, but users could still submit a transaction (likely against their last used card).

However, it is a big deal in a credit card management interface. Without supplementary data, all of the cards would look exactly the same and the UI might break if expected fields aren't present.

----

I don't want my type system deciding what the use case is or how important one case is compared to the other. The actual implementation needs to decide if it has the information it needs to do its job.

No, that is in fact the stated purpose of the type system. If you don’t want to make such a decision statically, you mark a field appropriately (e.g. via an Option type).

If everything is optional, then you might as well not use a type system in the first place.

    data Stripe1_0 = {
      field1: type1
      field2: type2
    }

    data Stripe2_0 = {
      cc_id: str
    }

    data Stripe = Stripe1_0 | Stripe2_0

Most of the time optional fields:

* Encode that your module doesn't care about such-and-such fields, so the parser / type checker should just ignore them.

* Are used to encode variants of the same data that slightly changed in another module [id changed from number to string], so you should use an union type.

In practice, designing a statically typed ecosystem for wire data is hard [0]. Grpc has basically gave up on validating field presence altogether [1]. These failures are directly caused by the immaturity of the ecosystem [deep decode batches of unrelated messages in middleware, WTF?]. It would be a mistake to throw the baby with the bathwater and blame the mathematically sound idea that your module can only process data of a certain shape, encoded by a suitable type signature.

[0] https://capnproto.org/faq.html#how-do-i-make-a-field-require...

[1] https://github.com/protocolbuffers/protobuf/issues/2497#issu...

However, the problems you describe also happen in dynamic languages and they have just as much potential to cause problems. The biggest issue with dynamic languages is that you don't get immediate errors when something changes. The system will just keep doing its thing, until it doesn't and you discover that something has changed and has already been propagated to other parts of the system.

I don't think dynamic languages are always better in these cases. They basically let you defer the error/change handling, but with the potential of much bigger data integrity/quality problems when things eventualy do go wrong because of changes in an integration.

> It always seems to end up in types where every field is optional

“Always”? Not by a long shot. These situations where this is the case can be isolated and dealt with explicitly. You may object to the explicitness but — again, for reasons of robustness — proponents of static typing prefer this.

I didn't work much with static typed languages lately, so I'm really puzzled by your comment. If I expect an int in the age field of a JSON response and the server suddenly sends me a string, maybe with value "NaN", how do you check for that at compile time? I expect the program to either crash or manage the error at run time (try/catch, die-and-restart a-la-Erlang, etc)

Maybe JSON doesn't count as "statically typed API", but how many of them do common Internet services expose to the public? And how to we make sure that the server or any in between proxy doesn't misbehave and violates the contract?

Either way, you need to handle such failures, both in statically and in dynamically type-checked languages. The only difference is whether the handling happens explicitly throughout your code or automagically by an API that gives you back a well-behaved object. And both of these ways are possible in both static and dynamic languages (but static languages tend to go for the latter, whereas dynamic languages tend to make you do the former).

Does that make sense? I can show you some code if you’d like.

In statically typed language, at least when troubleshooting it's an easy guess that the error is highly likely to be at the external boundary of your application code since the rest of it type checked.

It's more of an issue that you have unreliable 3rd party vendors that you buy from/use.

It happens very, very rarely - but the problem is when it happens it can happen suddenly and be very, very hard to address properly.