The closest thing to RMarkdown is MyST — the native .md format for the jupyterbook project: https://jupyterbook.org/content/myst.html

I've switched to that instead of notebooks, and loving the text-based life... gittable, diffable, peer-reviewable code, etc.