As a personal testament: I did this for a side project, and it worked perfectly. Phoenix does very well as "just" an API/backend environment.