I teach Data / AI courses regularly and this topic is a recurring one with students. We the instructors like to make students start without even ML in the first place for them to construct a baseline model, usually using simple heuristics (e.g. conditions, regexes, etc.).

At first, students don't like it and tend to skip it because it is not that impressive, but in the end they learn so much about the data and the problem at hand! Then they are allowed to try simple ML models to see if that's a performance improvement and so on with more complex models. In some cases, improving baseline models with complex ML/DL models is really hard (e.g. time-series). Another benefit of simple models is explainability, for which the industry demand is growing everyday.