The only m5stack device i've used is the "cardputer", which supports (a slightly old version of) ESP-IDF out of the box. Honestly, I think ESP-IDF is a solution to a non-problem and wish espressif just shipped a toolchain and maybe a linker script like every other embedded systems developer, but I'm currently working to tease that out so i can get to a sane development environment. I think environments like Arduino and ESP-IDF tend to encourage half-assed code, to be honest.