The Roman emperors spoke latin, and that was the language of the senate, army, and politics in general, as well as everyday life in Rome. A latin ancenstry was also a must for the most part - even supposedly "ethnic" emperors are usually just descendants of latin parents, Roman royalty or army deployed in some part of the empire.
That the elites also were taught Greek as a cultural thing is another matter. For cultivation it was ancient (attic) Greek that they read classical Greek texts in, not Koine. Byzantium had both: the people spoke Koine Greek, but the "higher education" Greek in which "serious" literature and treatises was written, and which was taught, was attic, the Greek dialect of ancient Athens.
The population in the whole eastern part of the Empire, did speak Koine Greek, a big majority as their native tongue, and others as a trade language (or in older Greek settlements). The latter that was more commonly the case for the western provinces when it came to Greek too.
The educated and elite of Rome, nay, the majority of the Mediterranean would have spoken a form of Greek. Paul of Tarsus writes his letters now collected in the New Testament of the Christian Bible in Koine Greek; likewise the rest of the New Testament, as well as most of the writing of the Church Fathers, largely written by people from middle-class backgrounds.
Hence, the Germanic warlords who would come to be the ancestors of Western European aristocracy today, would come to speak Latin because they primarily were use to engaging the militaries of Rome, both as allies and as enemies, rather than the intelligentsia of Rome.