REST is about hypertext-driving[0] and essentially nothing else.
[0] https://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypert...
The very first comment from that post asks what he means by "hypertext", a fairly fundamental prerequisite for following his thesis. He replies with the following:
> Hypertext has many definitions
> When I say hypertext, I mean the simultaneous presentation of information and controls such that the information becomes the affordance through which the user (or automaton) obtains choices and selects actions
I challenge anyone to transform something as simple as "hypertext is text with links to other text in it" into a more convoluted and unnecessarily abstracted definition as the above.
What's unclear or abstract about that? To choose an action, the agent must know what actions are available and how to call them. Whatever presents that information to the agent is the affordance. Fielding says that in hyptertext, you get that as part of the actual content as opposed to via documentation or some other channel.
> I challenge anyone to transform something as simple as "hypertext is text with links to other text in it" into a more convoluted and unnecessarily abstracted definition as the above.
Because that's not what it is, in general. See Xanadu for an example of vastly more general kinds of linkages than references.
The purpose of very abstract, generalised, academic (also legal) language is to attempt to achieve perfect technical correctness in description. Unfortunately this usually comes at the cost of understanding.
"Text with links to other text" obviously won't include 100.000% of applications conforming to RESTfulness, but it does describe what RESTfulness intends far better and more accessibly than a convoluted explanation, which very effectively serves the purpose of informing people calling RPC REST that it is not in fact REST (for example). It also gives a broad audience a general direction to work toward if they intend to use REST (or a reasonable argument not to use it if it's not what they feel they need).
The accessible explanation serves an actual practical purpose, the pedantic explanation has no actual benefit. The fact that you had to reference a project as esoteric and dead-end as Xanadu is testament to this.
But then that would be wrong. You had to know, through documentation and other out-of-band education, what you’re supposed to do with these `<a href=...` things. Ditto for any of the other options a page might present.
When you first used the web, you didn’t know what href meant. You had to read some instructions, or depend on UX designers’ decision to graphically represent a-tags in a way that suggests clickability and followability. That’s already outside his REST model.
Fielding is not being clear on how REST’s assumptions about “what you already know out-of-band” differ from other models.
Except text+links does not encompass the full meaning of hypertext. Forms and inputs are also possible, for instance. The more general definition you cite encompasses all of this (information+controls), and more possible innovations than your very specific definition which lists only the current ways.
I always find myself sceptical of people who appear to hold an almost religious level of belief about some programming construct or protocol and yet are unable to express themselves plainly. It comes off like a developer-ish version of business speak and isn't generally useful for moving a discussion forward or solving any problems.
Moreover, this is blog post and comments on a blog post: academic juries aren't the intended audience.
Any term this confused just isn't useful.
"Monad" seems to be similarly confused, but there's a concrete math concept to fall back on, and concrete definitions in various languages. REST doesn't have that. No matter where I try to stick the pin in the landscape and say "That is REST", I am reliably informed by half-a-dozen people that no, that's not it, it's over there... and I get six different "theres".
(No, the thesis does not define it, in practice. Even when people reference it, they still point to different places! And besides... so a guy sat down and wrote a thesis paper on a word... that's great and very nice for him, and not worthless, but let's not go overestimating how much claim that gives you to a particular word. I certainly don't grant it enough claim to overcome the fact that still nobody knows what it means.)
Useless.
(I've carefully written this to be clear that I'm speaking about the term. Whatever $YOU think REST is may very well have good ideas in it, but labeling it REST only makes things worse.)
In most cases Monad goes the opposite direction than REST: most people discover that yes, what they've been writing is a monad, they just didn't know it. Conversely, most people are told that no, they are not doing REST right, because REST is actually about <something else> ;)
There's large companies calling their MVP "MVC", and then you have to explain that "MVC solves the problems of MVC" (-> real MVC solves the problems of Apple MVC, and is being rediscovered all the time with "unidirectional data flow" of things like React).
And then there's "React", which doesn't actually have anything to do with "Reactive", but has similar enough marketing that you think it might.
Which gets us to the current champion of muddled terminology: "Reactive Programming" or "FRP".
So although the various "reactive" (Rx) approaches are often called "FRP" (Functional Reactive Programming), they actually are not at all the same thing. At least according to the person who came up with FRP and coined the term. Of course, he no longer thinks that was a good name for what he did, preferring "Functional Temporal Programming".
And that's a good point, because as Backus points out (see https://news.ycombinator.com/item?id=17564186), FP doesn't handle time well, and so Functional Temporal Programming tries to address that deficiency.
Does that mean that the others are off the hook in calling their stuff "FRP" or "Reactive"? Nope, as it still doesn't make much sense, "reactive" being a description of systems not of implementation technology, and the implementation technology being a form of (synchronous) dataflow that can be combined with either imperative or functional languages.
So there is nothing essentially functional or essentially reactive about this stuff, and so when you add Rx or "FRP" to a non-functional language, you are adding a somewhat convoluted way of expressing dataflow, convoluted because of the unnecessary detour via FP. And while it does add the ability to build reactive systems to FP languages (which otherwise lack this ability), it does not add reactivity to non-FP languages.
And although the word "reactive" and the language used to describe these systems suggest high performance, most "FRP" libraries add about an order of magnitude of overhead...so they are slow.
Compared to that, REST is straightforward and well-defined.
Purity is useful in places, but serves no purpose if complicated enough that I can't explain it to my team.
FWIW React is a nice enough library, JSX is the first and only angle bracket UI design system that I like (sorry XAML), and it all works well enough with comparatively little ramp up.
I'll let other people argue what the proper abbreviation to describe my code is.
There are two hard problems in computer science: cache invalidation and naming things. Yes, and off-by-one errors.
Yes, you can just build stuff and not worry. And that can work out fine. On the other hand, muddles like these sow confusion and make sure you can't figure out how things hang together.
Because HTTP error codes are fixed and have well-defined meanings for HTTP that may not map well to your application. Repurposing HTTP error codes conflates your application-specific errors unless it maps to an exact subset of one of the predefined errors.
> Why create your own authentication mechanism when you can use HTTP authentication over TLS (or client certificates, or etc.)?
Do you have a client cert? I've never encountered any server that supports client certs in my entire 15-year career. Also, TLS is not a part of HTTP and HTTP only supported Basic Auth until fairly recently.
...and beware even the errors that match exactly! I've seen multiple systems in the wild that treat 404 as an application-specific error for "content not found". It sounds perfectly reasonable at first, but it puts you one misconfigured reverse proxy away from inadvertently broadcasting to the world that all your content is gone.
This can have data-corrupting effects. If an external system deletes resources via a reliable message queue and treats 404 as a success condition, they run the risk of dropping these messages and creating an inconsistent state. I've seen it happen.
Keep application-specific error messages distinguishable from the 'plumbing'.
If a system treated 5xx responses as successful, you'd consider that a problem with the system's response handling, not a problem with your own communication of state. So why consider abuse of 4xx to be a problem with the communication protocol?
This would reflect an incorrect design on the part of the external system.
For example:
PUT /queue/_hyn3
=> {"name": "_hyn3"}
GET /queue
<= {"name": "_hyn3"}
... processing ...
GET /queue/_hyn3
404
Queues might be "reliable", but the consumers should never be assumed to be, and so it's good to have consumers provide a "complete" state update somewhere, and ideally, if the queue is able, for the queue to restore the incomplete item if the consumer fails to complete it in a specific (lengthy) period of time, because the consumer itself might have ceased to exist or failed for some other reason.
If you need to ensure that each message is consumed once and only once, you need to ensure that your queue also has something like a "completed state":
PUT /queue/_hyn3
=> {"name": "_hyn3"}
GET /queue
<= {"name": "_hyn3"}
... processing ...
PUT /status/_hyn3
{"name": "_hyn3", "status": "completed"}
GET /queue/_hyn3
404
GET /status/_hyn3
{"status": "completed"}
Client certificates are very common for API authentication (well-known examples: Kubernetes, Puppet, Saltstack, service meshes like Envoy and Consul...)
TLS client certificates are strictly better than passwords because they don't provide an impersonator with a wildcard (and if you are running over the internet, especially with mobile devices, then you can get into that situation w/o being subject of a targeted attack). There are fairly few ways to achieve that other than client certs (SRP and SAE come to mind, both of which have virtually no deployment).
this self signed certificate now has to sign the client certificate(s). Otherwise, the clients aren't allowed to address the master.
At least thats how i've come across it. Its actually pretty common in the infrastructure world. Your OPS team can probably tell you which services utilize it in your software stack, though you as a developer probably never had to worry about it.
Though a lot of services only use a singular client certificate across all nodes and just revoke the hole chain for rotations
* remote docker daemon execution
* setting up a kubernetes cluster with kubeadm and creating certificates for remote kubectl execution (this one might be related to the previous one. I'm not sure as i don't know anything about the kubernetes internals.)
* rabbitmq cluster iirc. At least thats how you're supposed to set it up with the sensu monitoring framework
* previously mentioned puppet does its as well, though the process is mostly invisible to the user
i'm not sure how its supposedly possible to spoof though?
the master needs to sign a certificate for clients. IF the master is compromised, everything is compromised. Thats true no matter which authorization protocol you're using
The kubernete's document says:
For an identity provider to work with Kubernetes it must:
1. Support OpenID connect discovery; not all do.
2. Run in TLS with non-obsolete ciphers
3. Have a CA signed certificate (even if the CA is not a commercial CA or is self signed)
You only need a 'real' CA if your service is public facing.
Heck,even a Windows AD server uses self signed certs
/Edit: after re-reading your comments I get the feeling that were really talking about different things. The self signed certificate becomes the base for your chain of trust. Everything signed by this is allowed to talk to the master. This verification happens on both sides. A client getting an unexpected certificate will refuse to talk to the master just as the master won't talk with a node, unless it's got a signed client cert.
Because the master cert is self signed, it's easier to just throw everything away and recreate the hole chain of trust instead of bothering with revocations.
Finally, there is no benefit in using an official CA if you are in control of everything accessing the resources.
Am I making sense now?
The master (this is the self signed part) cert becomes part of the client certs by signing.
These client certs are now only valid with the original self signed cert.
It's essentially a CA, only for the client certificates. It just isn't formalized because it's never imported. you'd also lose the ability to rotate the certs quickly without gaining any security by importing this self signed certificate as a CA.
I'm always surprised by the resistance to client certs and all the gleeful usage of pre shared keys I find. OWASP top ten bad ideas for as long as I've been writing software.
Agree when it comes to the difference between POST/DELETE/PUT/etc. but almost every time I've seen somebody doing a GET which mutated state somehow, the people doing it were always dismissive of the importance of adhering to HTTP semantics and it always inevitably came back to bite us in some way.
It was sometimes caching related but there are a lot more things that can go wrong than just that.
Using POST where GET should have been used hasn't caused horrendous bugs in the same way, but it still introduces annoyances.
In my experience, "using POST where GET should have been used" is normally a consequence of URL length limits and HTTP not supporting request bodies for GET. I've never understood the latter decision.
I've also found focusing on the resource/noun
aspect ( ... ) to be a bit of a "nerd trap";
Coding up the REST paths was a breeze, and I had implemented like 99% of perhaps a month's worth of work in under a week. I laid out some quick docs for how to interact with the URL patterns and, proud of how quickly the whole thing snapped into place, I showed it to the guy.
He immediately jumped all over me, complaining loudly that the URLs "were not RESTful!!!" Which, of course, shocked me. Was he joking? Certainly not. This was a very serious error.
Well, okay! Big deal!
So I spent 15 or 20 minutes performing search and replace, changing words from things like:
/blue/104
/read/9008
/fetch/18
/car/104
/book/9008
/page/18
During this time, I briefly mulled over the idea of trolling him with a many-to-many look-up table, and introducing variable synonyms that include opaque MD5 hashes of my original path names, and produce the same effective resource, such as:
/48d6215903dff56238e52e8891380c8f/104
/ecae13117d6f0584c25a9da6c8f8415e/9008
/5374034a40c8d6800cb4f449c2ea00a0/18
The pedantry and neurosis some people raise over obsessive semantics is painful to behold.
What the article is saying, and what I think GP is saying, is that trying to model (non-CRUD) actions as "resources" in the pursuit of being "correct REST" is often a waste of time. `/nouns/8000/verb` is somewhat obvious to me (it does `verb` to `noun` 8000) despite not being "proper REST" whereas `/verb/8000` doesn't (what exactly am I `verb`'ing?), `/green/8000` would just make me scratch my head, and `/9f27410725ab8cc8854a2769c7a516b8/8000` tells me I'm wasting my time with this API and should go do something else.
As a user or developer I think being obvious and usable is more important than being "correct REST" and so I don't even use the term anymore, I just say HTTP API.
You might think it is just "pedantry" or "semantics" but consider that you are not the only one who will be using or reading these APIs, and design accordingly.
(However, if your coworker really was jumping up and screaming loudly at you because of this, this suggests he may not be as enjoyable to work with as you think)
Must everything be left-to-right? What if the nouns and verbs were not English words? What if the natural order of possessives or adjectives is reversed in a romance language? What if the words are not even cognates, and the characters to express terms are not even letters, but opaque pictographs?
The paths chosen are an abstraction, and REST is designed such that it should be easy to bind any one resource to perhaps multiple URLs. But honestly, to the vast majority of the world, these URLs never see the light of day, and every consumer of a REST URL is probably going to do so programmatically, perhaps once, while reading the docs and setting up tests, and then they'll call it a day, and never look back, because honestly, consuming data from API endpoints isn't actually anyone's idea of "fun" despite wild claims to the contrary.
For the sake of getting shit done, I usually avoid this conversation in practice, but honestly, also because arguing is worthless. People have their opinions as a matter of convenience to their own efforts, and not as considerations to others.
These are the tastes of a given developer creeping in as a leaky abstraction of human readability.
Then again, if humans are part of the system, it's also probably a good idea to use more reserve with coworkers than "jumping all over me" implies.
The HTTP layer is a very popular place to build applications, but it's fraught with problems and underspecification (or an impossibly large number of poor implementations: take your pick).
The number of TLS guides that recommend ignoring server certificates, or don't check CRL, or don't check certificate-transparency, or don't log and whitelist changes to the public key -- seem to have no end.
In the banking Enterprise, getting whitelisted on the HTTP proxy is infinitely harder than getting a port open.
And so on.
If you control the client, then there's absolutely no reason to use HTTP: Getting security, reliability, and redundancy on HTTP is extremely complicated, to which end I recommend people simply don't use it: Pre-shared certificates and smart clients are much easier to get right. You wan't caching? Specify and implement the strategy yourself (ideally using content caching instead of timestamps or some arbitrary tag).
If you don't control the client, treating HTTP as a tunnel for a protocol with pre-shared certificates and smart client (e.g. OAUTH1a instead of OAUTH2) is still better. You wan't caching? Assume cache-infinity and build cache-busting into the protocol. HTTP clients will lie about the correctness of their caching strategy often enough to spoil any benefit a "browser-side" cache might bring.
And REST?
My advice to developers is to ignore it.
When you say "why develop your own caching strategy", I say, because you have to anyway, otherwise you're leaving clients in the dust.
Why implement authentication? Because you have to anyway: Passwords/basic authentication are insecure, and everything else requires so much attention that you don't want decades of deformities around HTTP implementations getting in the way.
Error protocols? The error protocol is how you deal with it: Do you retry? Pause and let the user fix something? Alert the operator? HTTP proxies are notorious for modifying error codes so no matter how hard you try to signal in the HTTP stream, you're still in for pain.
As the author of perhaps the fastest dynamic HTTP server, I find this part particularly funny.
> informally-specified
Oh and this part. Anyone who thinks HTTP is well-specified and correctly implemented probably should be kept away from a computer.
What about ensuring that your desired ports are opened and your connection won't be eaten by a random firewall because you happened to pick a closed port and the user doesn't want to be bothered with firewalls and your application suddenly is crap because it can't even get its network connection to work when every other alternative works out of the box?
That's not a problem home-users have: They NAT.
Meanwhile in the enterprise, 80/443 is frequently locked into a particular (cheap) HTTP proxy server that can't manage basic features of HTTP correctly.
Actually, it is.
Furthermore, I'm assuming you don't expect applications to work only in the confines of a lan.
There are strong reasons why HTTP is highly favoured for communication protocols. Heck, whole industry protocols have been scrapped in favour of HTTP-based ones for this problem alone. See RTP and why it has been abandoned by HTTP-based protocols such as MPEG-DASH, and why the industry made it their point to develop a whole new video streaming protocol based on HTTP.
I'm saying almost all home-users have NAT and can communicate to remote services on most high ports.
If you're claiming otherwise, you should be able to provide some evidence to back that up.
> There are strong reasons why HTTP is highly favoured for communication protocols.
Yes. Like where they don't control the client, as in the part right after the part you quoted.
What exactly is your point?
Sorry, that's patently false and flies in the face of reality.
>If you're claiming otherwise, you should be able to provide some evidence to back that up.
You can start here:
https://news.ycombinator.com/item?id=17569715
> Yes. Like where they don't control the client
That is patently false. The problems isn't caused by not controlling the client. The primary problem is that they don't control the network, which means each and every single node between client and server, and HTTP reduces or eliminates the chance that one of those nodes doesn't cooperate.
Once you've made those decisions from a business perspective, you can have your services send the HTTP headers to effect that caching behavior.
You also probably should not let the browser cache it either (you’ll typically get dinged for it in pentests because your app may be used on a shared computer).
Or, if you have a CDN, you could cache closer to the end-user (assuming you have the necessary controls on your CDN to meet regulatory [and common-sense] requirements).
But end-to-end cacheability at the HTTP layer, part of the original rationale behind the REST API design style, does not make sense for most APIs anymore in 2018.
Why not? What were the relevant changes between the 90s and 2018?
* HTTPS has broken intermediate caches.
* Most responses are personalised or highly dynamic.
* Many responses contain PII or sensitive information and should not be cached.
* Many requests are rate-limited, billed or metered.
Observation: people go out of their way to cache static resources but tend to disable caching entirely on their JSON APIs.
I do have some bias; recovering pentester who rarely worked with APIs that served the same response to all users.
You can force some of it by putting everything behind TLS, but even then there will be downstream caches for some of your customers (especially the enterprise customers with their sweet, sweet enterprise money) which will still interfere with your expected caching.
That's just using HTTP.
If you're doing caching, you want to do that throughout the network (via proxy) so you can have tiered control of traffic flow. If you're going to block access to networks, you want to do that at a firewall so you can do some access control independent of your API endpoints. If you're designing your API, you want your HTTP method to reflect the kind of operation happening on the server, so you can categorize traffic by method when you're looking at service metrics.
These aren't just "using HTTP well", they're deep architectural decisions. They deal with HTTP, but you make them not just because you're trying to use some transport protocol the right way, but because it improves your quality of service.
Where I do have trouble is specifically with error codes. There never seems to be the right one. Returning any frequency of 5xx error codes could give any load-balancer a reason to remove the instance and replace it. Returning far too many 400 codes makes many errors difficult to distinguish and often the cause is shared between the client and server and could succeed later, so would be inappropriate.
The main source of difficulty is where codes describe conditions rather than subsequent actions. It's better to think of 429 as the client shouldn't reissue the same request without backoff. This handles both rate-limited requests as well as a server being overloaded. In this case all is well. Status 400 tells the client not to reissue the same query--it's not clear if that only refers to the lexical structure of the query/body or also the server state, in which case 409 may make more sense. I don't have a clear understanding of what other codes mean to clients or servers.
When should one use 410 Gone:
"The 410 response is primarily intended to assist the task of web maintenance by notifying the recipient that the resource is intentionally unavailable and that the server owners desire that remote links to that resource be removed."
These codes were created for HTML and not REST in mind so we'll have to make do, but don't think they're complete or perfect as-is. New status codes keep getting created with varying adoption.
By thinking this way, you can see how REST helps you re-use already-existing infrastructure---proxies, load balancers, caches, etc.---instead of spending time and effort making custom solutions. It also helps provide guidance as to when REST may not be appropriate: can you answer those "why not HTTP?" questions?
I've also found focusing on the resource/noun aspect (beyond considering caching) to be a bit of a "nerd trap"; it's possible to spend a lot of time chasing the perfect, "correct", solution. As the article notes, there's nothing wrong with "verby" URLs as long as you are using HTTP instead of doing your own thing.