Fair point. I didn't realize that the access token was referenced in the OIDC spec, but sure enough, there it is:
"After receiving and validating a valid and authorized Token Request from the Client, the Authorization Server returns a successful response that includes an ID Token and an Access Token"
https://openid.net/specs/openid-connect-core-1_0.html#TokenR...
Clarification - only the id_token defined by OIDC needs to be a JWT. This is because it is an authentication statement to the client, so the client needs to be able to verify and understand the data.