I disagree that build systems need to be complex. They get complex because people try to invent too many different ways of doing things instead of working towards standards. webpack is a prime example of that.
I'd love to see a full demo seed example of your ideal build system. Something that includes hot code swapping and even more importantly, unit testing.
Yes, build systems are complex, because people need features, even if you personally don't need them. Webpack doesn't offer many ways to do the same thing; it just simply has a lot of features. Whenever we get enough tooling around ES6 modules to match it (or come close), I'll gladly switch.
You'll see a fuller example later. I'd love to see a matching set up using systemjs/jspm. You could convince me to switch.
Webpack solves a LOT of problems with application development. Handling assets properly is very difficult, and webpack makes it straightforward.
Build systems are complicated, I'd love to see something that handles the same problems as webpack but somehow manages to avoid the required configuration.
For hot module replacement, sure, it gets a little more complex. You add another plugin and another entry point, that's all.