My understanding is that Reaper has a more mature feature set in this regard. That said, it is possible to do a number of things in Ardour already using Lua scripts: https://manual.ardour.org/lua-scripting/ and https://github.com/Ardour/ardour/tree/362b9cb4fa4fbacd0660b1...
I wrote some plugins for fun a few years ago in Lua: https://github.com/robsco-git/ardour-plugins
My understanding is that the plugins one writes for Reaper are compiled on the fly so I imagine they offer better performance.
As an Ardour user, you don't actually have to write your own plugins.
You could just use any of the thousands of VST plugins out there, which Ardour fully supports.
Yeah, of course Reaper also supports all types of plugins, including of course all types of VSTs (32 or 64 bits), and DX, LV2, etc.
But sometimes the thing that you need does not exist. I needed a "MIDI notes compressor" and couldn't find one, so I wrote it (the compressor brings all notes to a chosen range, so for example if the range is A3-A5, a C7 will be converted to C4).
But, Ardour is free, Reaper isn't, so that is a big difference.