This seems like good library design. As annoying as it is, it means the things you can use are well tested and supported.

What was your solution to this? Parse the things the library didn't?

The library didn't allow you to see the things (e.g. particular headers or options for those headers) that it didn't know to parse. Ultimately we had to migrate to a different library that didn't restrict us to just what the library knew. The decision not to let us even see things that the library didn't know about is particularly egregious where best practices are changing over time.

In my view it's a very bad design for an http library, although it would have been a lot less frustrating if it had at least provided an escape hatch.

Sounds like its model is not the HTTP RFC, but something more specific to some domain.

Which I agree, is a poor design choice. A type modeling an HTTP request should model the RFC definition as closely as possible.

> Which I agree, is a poor design choice. A type modeling an HTTP request should model the RFC definition as closely as possible.

I couldn't disagree more. A type modeling an HTTP request should model HTTP requests. Not some theoretical description of an HTTP request.

HTTP requests are not things that humans discovered in nature. They are abstractions, created entirely by specification. In some sense, an HTTP request is exactly that which conforms to the specification.

To an extent, that sounds like saying the thing I am sitting in is not a chair since it has 5 legs.

I mean, if chairs were things with formal specifications, and that specification said so, yeah.

But in this universe, no.

If you can completely ignore HTTP request data that happens to not perfectly meet the RFC at your work (or, more specifically for us, Modbus RTU responses), I salute you. Sadly, I can’t, we get some wild stuff that we still need to attempt to handle. Both HTTP and Modbus!

You don't have to run an HTTP server very long on the internet to start discovering HTTP requests (including malformed HTTP requests, which are also a kind of HTTP request), 'in nature'.

> HTTP requests are not things that humans discovered in nature. They are abstractions, created entirely by specification.

They're created by something, but that something has more to do with a million blog posts and hallway conversations than it does with the formal RFC process. Certainly for most specifications of this kind, the working code came first and the specification was based largely on discovering what existing implementations did. If what the specification says is different from what HTTP clients send and HTTP servers understand, so much the worse for the specification.

I’m confused, if the RFC does not accurately model HTTP requests, what does?

Your customers define what an HTTP request is.

I suppose the “parse don’t validate” philosophy would recommend first transforming the I’ll formed request into a data structure that only models well formed requests, before it’s processed by any other part of the program.

> The library didn't allow you to see the things (e.g. particular headers or options for those headers) that it didn't know to parse.

I typically design code around things like this with a sum type like:

    data Header = KnownHeader1 | KnownHeader2 | UnknownHeader String String

Then I typically don't offer any extra support or extended functionality for the cases where the type is `UnknownHeader`.

I discussed how/why the point of this article is very much not to “parse everything” in this followup: https://lexi-lambda.github.io/blog/2020/01/19/no-dynamic-typ... In particular, it articulates precisely why it is fine to use relatively wide types for any input which the program treats opaquely, and it gives an example of how the same techniques can still be useful even in that context.