I’m not sure what transfer rates you’re looking for but modern SPI slaves can do > 120 MHz (roughly 15 MB/sec); if you give it two more pins you can do 66 MB/s (bytes, not bits) via quad spi (cf the datasheet for this very run-of-the-mill NOR flash that can do SPI, Dual SPI, and Quad SPI [0]). Of course that’s a unidirectional transfer rate - you won’t be able to simultaneously both send and receive at that rate, but you can alternate symmetrically between the two.

You won’t be able to bitbang at those speeds, so you’ll definitely have to drop down a layer in your stack/abstractions but since you mentioned I2C I figured you’d probably be OK considering this.

[0]: https://www.winbond.com/resource-files/W25Q128JW_DTR%20RevD%...