I don't identify as an ML person and fight this type of complexity all the time. Start simple and then move complex. Starting complex is simply overkill for so many problems.