You can always be asked to install additional stuff during the set up process, but it still doesn't make sense why you can't delete them. Deleting the apps won't necessarily remove the installer.

iPhones come baked in with a number of apps but you're able to delete the ones you don't want to use.