1. Log in with your service, receiving a JWT token
2. Send that token back to your service, as the auth key for authenticating as themselves with your service.
3. Also send that same token to the third party service, to authenticate requests against your service's account.
(One place where I just set this up the other day: allowing users of a crypto app to make authenticated requests to Infura via the MetaMask browser extension, where we're paying for the Infura credits to enable that. We send the user a JWT token; the client JS uses that token to build an Infura URL, and configures the MetaMask SDK with it. Infura accepts the requests as coming from "us" because it recognizes the JWT signing key we configured.)
Having the emitting entity sign their request in a standard format that can be checked by anything having with their public key helped a lot.
Sure there are some gotchas but it isn't that hard to read up on that once, configure your JWT check and then leave it alone.
Also, we pass additional claims in the JWT that avoids the need for the web service having to check certain permissions or auth status itself (like what resources the user has requested) and this is going to be a mess if you try and "just HMAC(SHA256())"
The point of JWT is that it's an ecosystem of stuff that all has built-in support for doing this. You don't have to arrange for this kind of signing "by hand"; on most services that have JWT "accept" support, there's a portal with a configuration field where you can just paste in a public key. Same as adding a line to ~/.ssh/authorized_keys, but for web requests.
(Yes, all of this is just a bad imitation of TLS client certificates. But browsers have a horrible, intrusive, non-syncing UX for client certificates. If they did with client certs what they do with cookies/passwords, nobody would have bothered to invent JWTs.)
The real use of JWTs is when you want third parties to be able to sign their own API requests, that your API will accept using the very same machinery by which it accepts your first-party-client API requests. If you use JWT, you can just pass the third party a private JWT signing key to use, add the corresponding public key to your validation set for JWT decode on your backend, and everything "just works."