One "hybrid" approach that seems to work well is pulling permissions from the DB but then caching them in-memory on the server. That way you can still keep them granular but you'll only hit the DB on the first request or if the cache expires. It also lets you immediately kill access (rather than waiting for the JWT to expire) if you can send a message to your servers to clear the cache for that user.