Almost certainly is the extended RTTI information - VCL is huge after all and relies heavily on RTTI. The forms, objects, etc you are editing in the form designer are not just stand-ins like in, e.g. Glade, wxFormBuilder, QtDesigner or similar tools, they are live instances of the actual objects that are serialized on disk whenever you save the project with the serialized data becoming part of the program's resources and are deserialized when the program starts (VCL/Delphi calls this "object streaming"). This means that the full RTTI information needed to serialize and deserialize the objects (and any other object) needs to be part of the executable.

At some point they extended the RTTI information so that you can also make dynamic calls to methods, obtain fields, etc so the RTTI data became larger.

Lazarus adds a ton more data to the executables so where you'd get a 1MB in some recent Delphi, you'd probably get something like a 5MB executable with Lazarus (without debugging information - that'd make it much larger) on Windows. On Linux it can be even larger, e.g. the simple calculator shown here[0] (up left window) is ~2.88MB, stripped.

[0] https://i.imgur.com/MkfN4pO.png

You can easily remove the RTTI info from your binary: https://stackoverflow.com/a/3975363/12458

Glade uses GtkBuilder: a format where the UI is described as an XML and built at runtime.

Pretty much all of these programs (Glade, wxFormBuilder, etc) have a similar approach, the point is that the way Delphi (and Lazarus) works is with you working directly with the live instances of the objects that will be serialized on disk - there is a 1:1 mapping between all those and the information is extracted directly from the objects themselves.

In a way Delphi is kinda similar to how Unreal Engine works whereas Glade is kinda similar to how the original Quake engine worked.