News for hackers. Stuff that matters.
Re:Problems with RESTful APIs (2015) by quanticle on Tuesday February 28, 2017 @07:54PM (#13759918)

    The "true spirit" of REST, to me, is that there's a certain set of things 
    you can do when creating an API that will let you re-use the huge amount of 
    HTTP middleware that's been written and get correct (and useful!) semantics 
    from it. Caches (browser-, edge-, and server-side-), load balancers, 
    forward- and reverse-proxies, application-layer firewalls, etc. will all 
    "just work" for your software if you do REST correctly, and won't have any 
    weird edge-cases.
Doesn't that assume that the middleware also implements REST correctly? As the article points out, there are all too many HTTP libraries that only support GET and POST, not the more "esoteric" verbs like PUT and DELETE.
Re:Problems with RESTful APIs (2015) by jasonkostempski on Tuesday February 28, 2017 @08:27PM (#13760131)
The article says "most client and server applications don’t support all verbs or response codes" which, in my experience, is not true. There are some but it's definitely not "most". For one, if you're writing both sides, don't build on client or server software that sucks. Sure, HTML forms don't support PUT and DELETE but how often to you use HTML forms instead of ajax requests (which do support those methods)? And, if you subscribe to a little bit of CQRS and Event Sourcing ideas, PUT and DELETE don't really make sense since you should really be POSTing commands to do those things. Where they do make sense is when manipulating files directly which I don't think comes up much these days.
Re:Problems with RESTful APIs (2015) by dragonwriter on Tuesday February 28, 2017 @07:57PM (#13759934)
> Doesn't that assume that the middleware also implements REST correctly?

It assumes that the middleware supports HTTP properly.

Re:Problems with RESTful APIs (2015) by SEJeff on Tuesday February 28, 2017 @11:36PM (#13761164)
Or (for REST apis that might need concurrent updates of the same model) the most awesome "esoteric" verb, PATCH
Re:Problems with RESTful APIs (2015) by Mikhail_Edoshin on Wednesday March 01, 2017 @07:54AM (#13763283)
Look into WebDAV and you'll find plenty :) LOCK, UNLOCK, COPY, MOVE, MKCOL, PROPFIND, PROPPATCH. And also REPORT, SEARCH, and various version-control operations.
Re:Problems with RESTful APIs (2015) by SEJeff on Wednesday March 01, 2017 @09:43AM (#13763969)
Oh sure, but the context is REST apis, not WebDAV :)

PATCH allows two users to update different fields on the same resource at the same time with no conflicts. Otherwise, they'd use PUT and the last update would win unless they use something (client-side) such as compare and set.

Re:Problems with RESTful APIs (2015) by pc86 on Wednesday March 01, 2017 @10:07AM (#13764196)
PUT and DELETE are just HTTP verbs and don't directly have anything to do with REST. You can support all verbs and have a garbage REST implementation, and (while obviously more difficult) have a great REST implementation for a handful of verbs.
Re:Problems with RESTful APIs (2015) by naasking on Tuesday February 28, 2017 @08:57PM (#13760270)
> As the article points out, there are all too many HTTP libraries that only support GET and POST, not the more "esoteric" verbs like PUT and DELETE.

GET and POST are all you really need for REST.

Re:Problems with RESTful APIs (2015) by int_19h on Tuesday February 28, 2017 @09:14PM (#13760348)
What we really need is a set of verbs that allow to reliably distinguish between these three types of operations:

1. Pure read.

2. Impure (stateful) read.

3. Idempotent write.

4. Any other write.

There's no particular reason to separate inserts, updates, deletes etc as part of the protocol - they're all just different kinds of writes, and middleware doesn't derive any benefit from being able to distinguish them. Thus, this can be a part of the payload.

On the other hand, the difference between reads and writes, and the two subdivisions within each, do matter for purposes such as caching and automated error recovery (e.g. a proxy can repeat an idempotent write a few times before returning the error to the originator of the request).

In REST, we have GET for #1 and #2, PUT and DELETE for #3, and POST for #4. In practice, this is often simplified to just POST used for both #3 and #4, but that loses a valuable distinction (but is unfortunately often necessary because of the lack of support for other methods). On the other hand, the PUT/DELETE distinction is largely pointless.

Re:Problems with RESTful APIs (2015) by naasking on Tuesday February 28, 2017 @09:40PM (#13760482)
It's not worth making the distinction between idempotent and non-idempotent writes given the possibility of network partitions. Every write must be made idempotent because all you can do is retry in this case. PUT is merely an optimization on idempotent POST requests, one that will become progressively rarer as HTTPS continues to spread.

POST requests can be made idempotent by binding the result of processing that request to a future, so every subsequent request simply returns the already computed result.

Impure reads are the default, and pure reads are designated by long-lived cache headers, ie. it's an impure read that lasts as long as the server responds.

So really you just need GET and POST, and server-side frameworks should make POST requests idempotent by embedding some notion of futures for side-effecting operations. See the Waterken server for the first development platform to really get this right.

Re:Problems with RESTful APIs (2015) by int_19h on Wednesday March 01, 2017 @05:38AM (#13762756)
An insert cannot be idempotent by the very definition of the operation.
Re:Problems with RESTful APIs (2015) by joncrocks on Wednesday March 01, 2017 @07:41AM (#13763213)
It can be as long as you have some kind of identity information associated with the request, and provided alongside any requests that are the 'same'.

e.g. Instead of saying "Insert a transaction for £20", you say, "Insert a transaction that entity A calls 1234, for £20".

Stripe use something like this to ensure that as long as you call their API to ask for a payment and use the same key, they won't charge the person twice.

https://stripe.com/docs/api#idempotent_requests

Re:Problems with RESTful APIs (2015) by naasking on Wednesday March 01, 2017 @08:18AM (#13763417)
Requesting an insert can be idempotent. Each non-idempotent operation is tagged with a unique identifier, if the server doesn't have inserted data tagged under that identifier, then it performs the insert, if it does, then it returns the usual success code as if it had just performed the insert. This unique identifier is simply a durable representation of a future which I described above.
Re:Problems with RESTful APIs (2015) by int_19h on Wednesday March 01, 2017 @01:50PM (#13766904)
This would seem to require the identifiers to be attached to records in perpetuity (i.e. it would basically require client-generated IDs everywhere), so that the server can reliably verify that this operation has already produced a record. I can see it working, but it's a far-reaching change, that may not be easy to adopt for existing data storage schemas.
Re:Problems with RESTful APIs (2015) by naasking on Wednesday March 01, 2017 @03:49PM (#13768140)
> This would seem to require the identifiers to be attached to records in perpetuity (i.e. it would basically require client-generated IDs everywhere)

Not sure what you mean by client-generated. The server-side app generates the id because it has to store and interpret it. This can be as simple as a sequence number, similar to how TCP guarantees delivery. It depends on the schema really.

This is the simplest way to ensure you can perform arbitrary retries of POST in case of a network partition.

This data doesn't even need to be integrated with your app's schema, although that's ideal so you can manage the storage lifetime. But you could use an entirely separate store for GUIDs and cached replies, and so it becomes transparent to your app and just becomes another layer.

Re:Problems with RESTful APIs (2015) by int_19h on Wednesday March 01, 2017 @05:09PM (#13768985)
> Not sure what you mean by client-generated. The server-side app generates the id because it has to store and interpret it.

Per the description above:

"Each non-idempotent operation is tagged with a unique identifier"

Since the operation originates on the client, the client has to tag it with the identifier, no? And the server has to store this identifier in a way that associates it with any data affected by that operation in a non-idempotent way.

Or are you saying that the client first has to make a round-trip to the server to generate the ID, and then use that server-provided ID for the actual POST?

Re:Problems with RESTful APIs (2015) by naasking on Wednesday March 01, 2017 @07:13PM (#13770101)
> Or are you saying that the client first has to make a round-trip to the server to generate the ID, and then use that server-provided ID for the actual POST?

This is always the case for REST given HATEOAS, ie. you've already made some hypermedia requests to obtain the URL of the endpoint to which you will POST.

Unless the resource you're posting to actually is the public entry point of your service, but that would be very unusual.

Re:Problems with RESTful APIs (2015) by int_19h on Wednesday March 01, 2017 @07:55PM (#13770372)
Wait, but what about the request used to obtain the URL of endpoint to which you're posting? Isn't that one then not idempotent (since it would create new URLs every time)?

Generally speaking, what is the flow like? Suppose I allocated myself an endpoint, but then never posted anything to it - what does the endpoint actually contain then, if queried? Do unused ones get "garbage collected" somehow eventually?

Re:Problems with RESTful APIs (2015) by naasking on Thursday March 02, 2017 @09:43AM (#13774261)
> Wait, but what about the request used to obtain the URL of endpoint to which you're posting? Isn't that one then not idempotent (since it would create new URLs every time)?

Not necessarily. The numbers don't have to be stored before they're actually used by clients. For instance, you could return a simple integer, like an object version #, and an HMAC(integer, resource URL) to ensure the client can't tamper with it. You only store data under that integer when the user successfully POSTs to that resource for the first time.

> Generally speaking, what is the flow like? Suppose I allocated myself an endpoint, but then never posted anything to it - what does the endpoint actually contain then, if queried?

There are many possible designs here. I prefer something like this in CRUD-like contexts [1], because it gets me a fully auditable change history, optimistic concurrency control and all the storage needed is fully integrated into the app schema in a sensible way.

Combined with the HMAC described above, you don't need to preemptively allocate any storage for idempotent POSTS and you can use simple integers as your unique identifiers. So a POST against a version number that's not the latest version would simply return the version that followed if their POST was the one that succeeded, or a redirect to the latest version if someone else had updated it.

Hopefully you can imagine various relaxations on this to make other trade offs. For instance, a much simpler approach would be to return GUIDs for each possible POST, and you just store each GUID associated with the object when a POST succeeds. If the object's current GUID is the GUID the user is posting, return the current data with 200 OK, otherwise return a 301 Moved to a URL referencing the latest version and let the client try to apply their updates to that. The storage can be reclaimed after a suitable period of time; say a month if your app is for browser clients

[1] https://higherlogics.blogspot.ca/2015/10/versioning-domain-e...

Re:Problems with RESTful APIs (2015) by richardwhiuk on Tuesday February 28, 2017 @11:07PM (#13761024)
Why oh why oh why do you want a stateful read? That will be the beginning of the end for your architecture.
Re:Problems with RESTful APIs (2015) by int_19h on Wednesday March 01, 2017 @05:39AM (#13762763)
Because applications have state. In fact, the vast majority of reads in any applications are stateful (the database backing your app is also state, you know; I'm not talking just about session state here).

By "pure" here I mean that we're talking about a pure function of its inputs.

Re:Problems with RESTful APIs (2015) by icebraining on Wednesday March 01, 2017 @08:08AM (#13763366)
Why do you need to distinguish between those? In fact, if the function is pure, why not run it on the client?
Re:Problems with RESTful APIs (2015) by int_19h on Wednesday March 01, 2017 @01:51PM (#13766921)
Because the server has more computational resources, for example. It's a very rare scenario in general, but it does happen.

Why distinguish? Because if you know that the results are not affected by server state, they can be cached much more aggressively.

Re:Problems with RESTful APIs (2015) by jandrese on Wednesday March 01, 2017 @09:37AM (#13763903)
You mean like a cursor in a database?
Re:Problems with RESTful APIs (2015) by tdriggs on Tuesday February 28, 2017 @11:22PM (#13761091)
Logging/analytics? Paywalls?
Re:Problems with RESTful APIs (2015) by nippoo on Wednesday March 01, 2017 @10:49AM (#13764623)
Rate limiting APIs?
Re:Problems with RESTful APIs (2015) by edoceo on Tuesday February 28, 2017 @09:32PM (#13760450)
PATCH verb? I've seen it in the wild
Re:Problems with RESTful APIs (2015) by int_19h on Wednesday March 01, 2017 @05:40AM (#13762766)
Indeed, I forgot some. Then again, last time I tried to use PATCH, I found that support for it in various places in the stack was patchy enough to make it a non-starter.
Re:Problems with RESTful APIs (2015) by ianamartin on Tuesday February 28, 2017 @08:24PM (#13760119)
Can you not imagine a REST-ful use case where the DELETE options should never work?

Come on, person.

I don' know about you, but I provide REST stuff where the only option is update.