But in the case of GraphQL, like I may get an incomplete list, or I may get a complete list that will have some fields with missing data, or some combination of both, going down deep depending on the structure of the query. I don't see a clear way to request the missing data without duplicating part of what I already have, and I don't see a clear way to combine the results.
But I may be missing something about GraphQL or something about the tools that makes this simple.
For example something like this:
{
users(first: 10000, after: "cursor123") {
edges {
cursor
node {
id
name
friends(first: 10000, after: "cursor235") {
edges {
cursor
node {
id
name
}
}
}
}
}
}
}That said, paginating multiple resources on the same page is still going to be complicated.
Probably the nicest way to expose this that fits contemporary sensibilities would be as a lazy sequence that does sensible prefetching and asks the server for more data as it’s iterated through.
Isn't this an accurate description of cursor-based pagination? You can have the server hold the cursor and associate a request for the next page based on the client ID (like in the PostgreSQL wire protocol), or you can make the flow stateless (at the network level) by handing the client a ticket for the next page.
I see you mentioned gRPC streams in a sibling comment, which are a great alternative to REST-based pagination, but they are highly stateful! Streams are built into the protocol, so a generic gRPC client will handle most of the state juggling that would have been your responsibility with a REST client.
We're also talking about APIs in general, not just http REST.
>It's still multiple requests on a streaming API
Perhaps, but the requests can be pipelined. You don't need to wait for the response to complete before asking for more.
gRPC is able to offer powerful streaming abstractions because it utilizes HTTP/2 streams, which are cursor based rather than strictly connection oriented. The state is still there; it's just a protocol-level abstraction rather than an application-level abstraction.
> Perhaps, but the requests can be pipelined. You don't need to wait for the response to complete before asking for more.
That sort of defeats the purpose of grpc flow control, doesn't it?
Why do you say that? I don't know the full implementation details themselves but generally there's no reason you can't safely ask for even more after having asked and consumed some. If you have a buffer of M bytes and ask for M, then consume N, you could immediately ask for N more without waiting to receive all of the original M.
Although, I wasn't speaking about gRPC in that case though. I'm not sure how exactly gRPC achieves back pressure. I was only speaking abstractly about a pipelined call vs many pages. You seemed to claim that multiple requests was a requirement.
But fine, ignoring gRPC, you could possibly tune your stack such that normal http calls achieve back pressure from network stacks and packet loss. Http is built on top of TCP streams, after all. That doesn't make it inherently stateful does it?
Going all the way back, I still think its fair to say that a streaming response with backpressure can be stateless and without multiple requests. If you want to argue that multiple packets or TCP signals are needed then perhaps so, but I think that's a far cry from the many separate requests a paginated call requires and I dont think its accurate enough to conflate them.
I think we're saying the same thing but using different formulations. If you send an HTTP request for a list with 20 items, then get back a response with 10 items and a link for the next page, that is essentially the same as cosuming 20 items over a stream with flow control. The point of returning a cursor in the response and having the client send a new request for the next page is to support a stateful stream over a stateless protocol. In neither case are you waiting for the response to be complete before processing items, since your message indicates that "complete" here means the full result set has been sent to the client.
> But fine, ignoring gRPC, you could possibly tune your stack such that normal http calls achieve back pressure from network stacks and packet loss. Http is built on top of TCP streams, after all. That doesn't make it inherently stateful does it?
That's pretty much how streaming responses are implemented in TCP-based protocols (like the SQL query APIs exposed by Postgres or MySQL). TCP connections can be terminated for unrelated reasons, which is why you don't see this pattern very often in recent protocols. When a TCP connection is dropped and restarted due to, say, network congestion, you have to restart the paginated operation from the head of the list. H2 (and, vicariously, gRPC) streams are built to be more resilient to network noise.
But to answer your question, yes, that pattern is inherently stateful. Pagination has to be, since the server has to know what the client has seen in order to prepare the next batch of results. You can manage this state at the protocol level (with streams), or you can push it to the application level (with cursor tokens embedded in the messages exchanged). The streaming approach requires a stateful server and client, whereas the application-level approach only requires a stateful client.
A paged response needs to sync state across many machines. As you have said, you need a clientID, a cursor reference, or a sticky session. That's not nothing.
I think they're inherently different approaches.
OK, I think this is what I was misunderstanding in your comments above. You can keep pagination state in a distributed store, but it's not necessary. The implementations I've seen and worked with have all embedded all the information needed to generate the next page of results in the cursor token itself. Kind of like with a JWT or TLS session ticket, there's no need to sync pagination state or use sticky sessions. That approach also provides some resilience in case the server that generated the previous page of results becomes unreachable (something that the client needs to handle by restarting pagination if pagination state is just held in memory by the server keeping the stream connection open).
gRPC's streaming API is a good example. It has backpressure built in and you just need to configure the buffer sizes. Clients just consume items off the stream and async writes will suspend as needed.
Or are you focusing more on the backend machine to machine case? In that case, some kind of automatic prefetching, lazy loading/yield setup sounds pretty nice if that's abstracted away when you want to iterate over a large dataset from a separate resource server. It's not a pattern I've used before, mainly because it's not often that one server wants to know everything that another server has.
There's no worry about wasting a full request on the last page with a single item. In fact there's no worry about round trips at all, it's just a matter of buffer back pressure that communicates the need for more or less.
Hmm, when you think about it, its a pretty good way to go.
In my professional experience, lots of devs will implement fetching APIs and not consider pagination, and then when the responses start growing (testing data, use, whatever), user experience sucks. Pagination should be a default API design for anything returning an unbounded list of items.