It's a good instinct to be suspicious of new technology. But I have personally seen, many times, what REST APIs can grow into - unless you are very, very careful, "get user/1" can turn into god objects with every field under the sun, non-optional. I've seen `users/l` be over a megabyte with (eg) comments, friends, comments of friends, likes, likes of friends, and every other thing the front end team ever asked for. GraphQL solves that.
Yes, it can be avoided with strict, diligent design. But it often isn't. GraphQL solves the whole class of problem. And that's why I like it, despite my mistrust of "new hotness" technology.
Well, within one of our frontend apps, someone wrote a "GetUser" query fragment, wrapped it in a react hook to make it easy to use, and now anytime anyone anywhere wants to get a user, even with one field, they're getting 100+ fields they don't want. Anytime someone needs a new field on a user, they don't create a new query; they just add the field to that "GetUser" query.
Now, I've told several GraphQL advocates that setup (in our local community and online), and unfailingly they balk at it. Well, you're doing GraphQL wrong, everyone makes mistakes, you should be more selective, etc etc etc. I think that's fair; being more specific sounds like a good pattern.
Except, we are not certain the performance wouldn't suffer should we move everything to more selective queries. The issue is caching. If you have ten queries each requesting one field, the cache can't saturate until after ten network requests. But, one mega query, and now the remaining nine queries can just pull from the cache. Sure, the mega query takes longer than one mini-query, but its not longer than ten mini-queries.
I only outline this to say: GraphQL is not a magic bullet. Every single thing you can do which would ruin a REST API is also possible in GraphQL. REST API responses naturally evolve into god objects. GraphQL APIs solve the god object issue server-side by saying "god objects are fine", then push the responsibility of not abusing it onto the client (and it should be obvious, the client is the worst place to solve almost every problem, but that's another topic).
GraphQL is easier and better for some things, far harder for other things. At the end of the day, its no worse than REST. I don't recommend it, but that's only just because its new; unless a new technology offers some very easy to express, tactile benefits for your business, just wait for it to mature. When you get down to it, GraphQL's benefits are definitely not tactile, but they're still there, and I think over time it will evolve into something very interesting.
What you described is absolutely correct, except it is only the case if we cache by query. If we cache by objects and fields, none of those issues you described become relevant, and caching by object/field (as opposed to caching by query) in general seems like a better practice imo, aside from certain very specific scenarios.
In fact, it seems like the official GraphQL docs recommend that approach as well [0].
If, instead, you have QueryC which does User { firstName, lastName }, but two usages of it (QueryC1/QueryC2), after QueryC1 requests, QueryC2 can use the cached results. This works whether you're doing Query/Operation level caching or Field/ID level caching. The former example works in neither. And QueryC is trivially faster than QueryA+QueryB because of network overhead.
This isn't necessarily an issue with GraphQL (and, I thought I was clear about this, but: I'm not against GraphQL). Its a behavior of both typical REST implementations and GraphQL. And, to be clear, GraphQL's out-of-box caching story is more powerful than any REST implementation I've seen short of hyper-engineered FAANG companies, because it enables really powerful field/ID level caching across multiple queries. But it doesn't work in this case.
The point is that its still very immature. Even the thought leaders (Apollo being the biggest one and worst offender) write these blog posts filled with Best Practices and Recommendations that often convey horrible advice and flat-out misrepresent GraphQL's actual advantages compared to REST. GraphQL solves a lot of REST's problems; it does NOT solve REST's "god-object" class of problems, like the grandparent comment suggests; and it introduces many new classes of problems that remain unsolved in the ecosystem because of how immature it is (one great example is OSI L7 inspection in many tertiary tools a typical SaaS app has. Many products like Datadog, CloudFront, AWS ALB, etc are capable of doing some really cool and powerful stuff out-of-the-box just by inspecting standard HTTP headers. REST is basically just standard HTTP; your resource is in the path, query parameters, http headers, its very standard. GraphQL is not, so many of these tools don't work out-of-box with GraphQL. People are catching up, but again, its immature).
The backend performance you could gain is kinda spurious anyway. We're talking about N requests being coalesced into 1 mega-request; I would rather clients send me N small requests, not 1 mega-request. Horizontal scaling is easy to set up and quick to do in real-time; vertical scaling is harder.
And I think, while a solution like this is interesting in the domain of "you've got two sibling components rendered at the same time who's queries could be combined", that's not exactly the issue I described a few comments up. Caching really doesn't enter into play here; it would not be measurably faster to just issue one mega-request for one component and let the other wait for the cache (which is, in the end, what we're talking about). I mean, it saves one network request, but 90% of a network request is just waiting on IO, and if there's one thing Javascript is really good at, its sitting around and waiting. It can wait for dozens of things at a time.
The issue is more specifically surrounding two components being rendered at different times. Imagine a User Preview card which displays a user's first name; the user clicks on that card, which takes them to a new page, the User Profile, which displays both their first and last name. Short of using a God Query for the Preview which covers a broad set of common user fields, like firing a shotgun and hoping you've got the cache saturated for unknown upcoming components, this situation will take two requests. The shotgun approach is what we do, and what many companies do; it works, but its imprecise. It can hurt the performance of the leading components which pulled the short stick, if that God Query gets too big, and as an application evolves you're gonna forget to keep that God Query updated with new things you may need.
This problem is enticing because it feels like there should be a solution, and we just haven't, as a community, found it. I think GraphQL, at its core, has the capability to solve this; its not like REST which is so undefined and hazy that any solution to something super-complex like this would only work in the domain of one company's "way of doing things". I hope one day we can figure it out, because I suspect that a general, easy to use way of preemptively saturating a cache like this would be a MASSIVE performance boost to every GraphQL client that's ever been written.
I'm a big fan of the saying "complexity has to live somewhere" and I think that's exactly what GraphQL is doing: moving the complexity to another layer.
https://lucene.apache.org/solr/guide/6_6/common-query-parame...
This alone makes it impossible to get backend to return god entities with dozens of relations without explicitly specifying them.
If `User` had `groups` and `groups` had `permissions`, then I can do:
let results = await fetch('/rest/Users/?fields=' + JSON.stringify(
{
id: true,
first_name: true,
groups: {
name: true,
permissions: {
name: true
}
}
});
My system has its limitations however. It can't do relational constraints like GraphQL or renaming of fields, but it is likely the 20% that gets you 80% of the way.My endpoints also have fields that will only be sent if you specifically request them.
The existence of a God Response / God Object / or just a generally horrific and coupled system strikes me as a particularly human induced problem which no specific technology can actually fix.
I've always held a strict rule where The R in REST stands for Resource (which isn't true). There's a user, an Invitation (and not an invite action on user, nor a state:invited or invited_at on user). there's a Session. There's links that clients follow, and everything is as small as possible. I really don't see how REST encourages God objects.
GQL solves this problem elegantly by allowing you to specify the data that you need in a single request.
That is just one example, there are many many others that you can google if you are actually interested.
Fair point.
But I ask my question again, differently worded: how does Graphql solve this? Why does GraphQL solve "many requests" and why can't you solve or avoid this in REST.
I work at a small startup. We use a REST and Redux architecture. It's worked very well for us, but we're starting to hit some pain points. As we look to re-architect, our list of challenges pretty much mimic the challenges GraphQL looks to address. These aren't scaling issues or "big company issues". In my opinion, they're core issues with every client-server code base.
* Client Data and Statement Management
* API Coordination and Versioning (even more complex if you have a public API)
* Payload Management and Relational Data (it can be very expensive to pro-actively return embedded/dependent data, so you need to manage it somehow)
* Caching and De-deduping
* Typing and Data Interfaces between Client/Server
* Error Management (not that hard to standardize, but still something you need to account for)
* Pagination, Filtering, Sorting, etc.
This isn't to say that GraphQL is bad or anything, but there are definitely some gotchas that are important to evaluate before diving in.
> I work at a small startup.
This means you can afford a lot of technical debt, even such technical debt as might be incurred by GraphQL (as argued by the post and many people in this conversation).
The concerns you've posted are certainly valid, but GraphQL is not the only (or even necessarily best) way to address them.
If you have a huge engineering staff like FB, then perhaps it's better to just "move the complexity to server side", but for most businesses that not what's going to happen. What's going to happen in practice is DoS galore.
I'd rather take on technical debt related to our product features than to load us up on fundamental networking and caching issues.
EDIT: Sorry, a bit of a rambly post, too tired and emotional to rephrase.
I will say, though, that "fundamental networking and caching issues" sounds very... weasel-wordy, if you see what I mean? It's not very concrete about what the actual problems you might experience would be. (Yes, latency and huge numbers of requests are issues, but is it really an issue until you get big enough?)
Almost all successful applications in existence until about 2018 (or so? Not sure exactly when GraphQL was invented) has done just about fine. If we're talking about a brand new type of application which couldn't have been done without it, then fine. If not... well, we're not talking about technical limitations.
I truly do see the appeal of GraphQL for developers, and especially the fast decoupled iteration that it enables, at least theoretically. The thing is that when you need more data on the client, you still have to add that data on the server... somewhere. It's great that the client can choose the overall "shape" of the data, but that doesn't solve the problem of the data not actually being on the server.
I'm a developer of my own projects for fun. I don't work in tech. My frontends are on iOS / Android / the web with Typescript. So my confusion is from a perspective without expertise.
I started with REST via Django and a few others, and now I've switched to GraphQL. I love GraphQL over REST backends due to the type automation tools such as GraphQL Code Generator and query tools like Apollo. Also being able to construct a query and access children via one request is super nice. For example my old REST APIs call for a post, then get pictures for that post, then get comments for that post, then gets the username and other user data for the owner that made the comment. It's four requests. My GraphQL requests just get them all in one customizable request. Post can contain pictures and comments and all their properties. Comments can contain the owner of the comment and all its properties to include things like username.
The result is a typescript object that is strongly typed and has all the data I need. Before GraphQL, in REST, this would be four requests, three of which are in series (post -> comment -> owner to get username). I know I could make a custom REST API to do the same thing, but it was just so easy in GraphQL, I didn't have to worry about it.
I'm not working in teams, I'm not creating a super large backend, it's not complex projects... so I don't know what I don't know.
But what am I missing? Maybe not everyone needs / uses the typing I do? That's the big benefit for me. Maybe not everyone cares about being able to query children (or create children via a nested create)? Maybe these things were easier than how I was doing it in REST?
That might be the case. Those things (such as returning children - and generally crafting a custom endpoint for specific use-case) are indeed very easy in my language/framework of choice (ASP.NET with a little help from LINQ).
I also get automatic generation of client-side data classes and strongly typed functions that encapsulate endpoints on the server using a small code generation framework that literally took 2-3 days to write (it's really not that complex).
You're suffering under Dunning-Kruger.
For example, what you described about making 4 requests with your 'REST' API, you didn't have to do that. YOU made that terrible API. That's not a feature of REST, but you seem to have mixed the two up in your head.
Resource is not what you think it is. The conflict you’re talking about doesn’t exist.
REST: Any time there is a new "view"; you need to go to your server and write a new endpoint.
GraphQL: You can stitch data together however you want; as long as that data is defined.
So for example if I need name of someone and the url of their children; querying it alone is enough with GraphQL whereas with REST either I need to fetch multiple times or write a new endpoint that serves both together.
GraphQL creates a data graph that powers the UI, REST creates a database that you can query on the UI and you have to compose it all together and manage.
Ultimately the thing that speeds up teams, again in my experience, is predictability not flexibility. If you tell someone they have a RESTful API they know exactly how to work with it, document it, etc. GraphQL which is shiny, new, and used by in-vogue orgs is fine, you can use it to get the job done, but I don't believe that it warrants much praise.
Then you don't share many of the reasons GraphQL was originally created for. Act accordingly.
Web API contracts are primarily routes and request/response data structures. This allows front end and backend concerns to be separate which allows a lot more flexibility over time for both front end and backend developers. From my limited knowledge of GraphQL, this is still possible but more work than just exposing your db schema types/DTOs.
> because they don't need to ask the API team to change an endpoint
Bollocks. What happens is you need some field implemented in the microservice or DB that your GraphQL layer talks to. This task gets tossed into your ticket system, gets managed by half a dozen managers, and weeks to months later you finally get the field you need and can finally bubble it up through your fancy toy API.
> GraphQL creates a data graph
does anyone even know what a graph is today? Serious question.
I'm not sure that the query layer solves team collaboration and prioritization problems.
Adding brand new capabilities to a system (or refactoring existing ones) will usually requires changes to the GraphQL layer, but this is only a subset of frontend work. Iterating on what already exists also represents a lot of development time.
It absolutely is the bottleneck in almost every web app that relies on Ajax fetches. The latency is an absolute killer.
On every app that I've optimised, I have to get people to stop going with their gut and look at the traces (both synthetic and real-user). People generally think they should be optimising their JS execution. But in fact the most important thing is usually sequencing loading correctly, followed by minimising JS bundle sizes.
Modern browsers are getting really good at parallelising loading and parsing JS and other resources. But if you're whacking some fetch in there that happens as a result of JS execution then you're generally looking at a wasted 300-500ms for most endpoints/device combos.
While I agree with your beginning sentence, the one I quoted doesn't sit well with me. It would be good to remember that not everybody has 100 Mbits fiber or 5G download speed.
We all saw the data on how milliseconds affect user retention then proceed to completely forget about it once building our apps.
I think we can do better.
GraphQL has never made sense to me for anything beyond toy or small-scale projects.
In a company you often have APIs geared for client apps, and then internal APIs between services. But rarely anyone develops good APIs for backoffice. And then you have an admin interface struggling to retrieve the data it needs from a dozen different API calls.
Slap a simple GraphQL server in front of these dozen calls, and you have a more streamlined development for internal tools.
Simply point Insomnia at the base url and it’ll introspect the schema and provide typed query auto completion. No dozens of requests to different endpoints with different query parameters than can only be kept track of by having the api docs open on a second screen.
I'm not being flippant. If you're already using REST tools such as Postman and every caching tool in existence (browser, http proxy, etc. etc.) then GraphQL tossing it all out is the exact opposite of "ergonomic."
The caching story in GraphQL is a joke. What the browser gives you for free with GET caching takes weeks and months of fine-tuning and tweaking and head scratching with something like Apollo. Then you'll probably try decreasing the payload size (there's a third-party solution for this), or batching requests together (guess what... there is a third-party solution for this too). The amount of tooling and implementation work GraphQL needs to get up to par with built-in REST is pretty incredible.
I do agree a lot of software is built just for the sake of software - there is loads of needless complexity in things. But GraphQL is not that imho.
Being able to very quickly connect a series of tools and tech together in a way that is selective of the data being moved around saves me money and time, period. That matters.
If inter-team communication isn't a hard problem in software development, why are we even talking about API technologies?
If inter-team communication wasn't that hard, you could just email pc and ask him to email you a snippet of Ruby to process a credit card payment.
I've def been in this situation before. But I agree that it it is not the norm
Sure, but that's only one advantage of GraphQl. Personally, I'd use GraphQl even for the smallest of apps. With statically generated sites, you can forgo a back-end server and just setup a GraphQl instance. It's a one-time cost to learn, and then you easily setup your APIs and if you are using React/Vuejs, it's even easier to integrate with your API.
Some people also want an agile api surface so they can iterate quickly, GQL is an excellent solution to that problem.
It is also an excellent solution for: - client side caching - unified RBAC - service stitching / microservice hell - api documentation via graphiql - decoupling FE dev from api dev - api automation - typed api surface
The list goes on.
To say that GQL is a just a band aid for bad leadership is nonsense, and discounts the very real reasons that many many experienced teams are switching to it.
I would encourage you to re-evaluate your position as if you voiced those opinions in an interview with basically anyone I know you would quickly get passed up in favor someone that actually knows what they are talking about.
Picking a non-sexy, non-scalable, "wrong" technology that does nothing more than solve the actual requirements at hand is a rare and amazing quality for engineer to have imo.
Everyone wants to build infinitely flexible, infinitely scalable machines using the most rapid iteration tools possible as though that's what "engineering" is. But sometimes... all you need is a REST API. Sometimes you need GraphQL. Sometimes, all you need is to stuff data in a bucket somewhere and call that an "integration point". All solutions have trade offs. Pretending otherwise, or faulting others for weighing those trade offs differently than you, is silly.
That is exactly the kind of problem solver I like to work with. Most of the time, “boring” tech will solve the problem cheaper, faster and as reliable and scalable as any shiny new tech. I work with GraphQL, REST APIs and even JSON RPC APIs on a daily basis and when you truly use those tools, you get to know where they really shine.
Non sexy "wrong" tech is exactly that. Pick smart tools that do the work for you so that you don't have to babysit your teams choices and micromanage every project.
I have weighed the trade offs, and it is literally my job to identify the problems that come from them.
I am glad you understand now.
I would encourage you to re-evaluate how you react to opinions that diverge from your own, especially in an interviewing context. GQL is not a panacea, and candidates should not be discounted because they understand this.
"At best it's a band-aid around dysfunctional inter-team working."
I would absolutely discount a candidate for having a moronic opinion like that.
Ha. Now we’re introducing threats. Don’t agree with my technical opinions? You’ll never work in this town again! Funny, funny stuff.
Also, React is a hype stack now?
React and GraphQL are extremely hyped. People build static HTML pages with React because... React. People use GraphQL APIs because... GraphQL.
These days the first thing every JS developer I've seen asks is: can we use GraphQL (which in fact they mean Apollo)?
I'm not saying it doesn't have its use-cases and merits, but it's very much hype tech.
Edit: typo
> React is a hype stack now?
Yeah, kinda. You definitely need some sort of front end framework, but I feel like there should be something better than react. I use react every day and it’s fine, but I’m waiting for something else to come and take it’s dominant position.
I build apps for living and hiring a svelte dev is not going to happen. Stick to react and go hiking on the weekends with the guys in your svelte meetup.
If you can't see the positive benefits of a new type technology, then you might be passed up for someone else more open-minded.
But yes, I agree that you should be critical of all new tech, and not just accept it blindly because FANNG says so...
Conversely, if you are also incapable of seeing the drawbacks, then you are a bad engineer who also deserves to be passed up for someone else.
If you are a person who is not capable of seeing drawbacks as well, then you would also be a bad engineer who should also be passed up for other candidates, who are more capable of evaluating both the benefits and drawbacks of certain solutions.
Is everyone you know solving FAANG problems? If they are not are they emulating those FAANG companies tooling because they expect to be one or work at one in the future? Or is it a type signaling?
There is this over tooling over scaling wave going on.
Most startups fail because in their limited investments they over tool and under sell.
Do you work at a startup or a FAANG?
I don't see the tradeoff that people are talking about, with GQL there has been nothing but upside.
I'm not saying it's enough, far from it. Bit it can be a very liberating 1st step.
As a backend engineer, I publish a schema of all the available data - the frontend can fetch it in any shape it prefers. Web UIs and mobile UIs often have different views, so they want different data, and fetching it in one request or many is up to the preference of the UI.
I don't have to build different REST endpoints presenting some data in a different shape for performance reasons or because different applications wanted to request the data in a different way.
As a frontend engineer, all the data is often already there unless I'm building some new feature the backend doesn't understand yet. I'm free to refactor my application without bothering changing the REST API - and GraphIQL lets me inspect the api and mock queries for how I might fetch the data for a given view.
It's a cleaner contract to let different people get their job done with less fuss - game changer.
You'll likely never be in a situation where over-querying via a non-granular REST call will ever be an issue worth optimising around.
If you're shipping multi-megs of JS to a client don't then pretend that micro-optimisating the API call waterfall is your KPI, it's just disingenuous at best.
At best it's a band-aid around dysfunctional inter-team working.