But if you're going to use screen to connect to your serial port, what was the point in adding serial to support to the terminal emulator?

To make the argument that scrollback is bloat is insane if you at the same time add serial support but need scrollback for serial usage, so you use screen instead which had both serial support and scrollback.