I think the additional issues come when you want to define access by a relationship instead of just the entity, which means you need to check access based on the edges, and not the nodes. You'd have to do this anyway with REST, but I think it's easier in some ways to make mistakes and allow more access than intended.