In France they seem to prefer <LAST NAME> (yes, all caps) <First name> even for less formal things.
Also this "reversal" had me confused when we hosted a French exchange student without knowing French (I spoke German, but the teachers knew my family were happy to host if they needed). His name was written like "KHALID Mohamed" and he was very polite so he didn't really correct me when I was calling him "Khalid", and since I was unfamiliar with Algerian names I had experience to suggest "Khalid" was more surname-y than firstname-y. It sorted itself out after a day or two though :-)
Julius Caesar has the 'group/last' name first because it doesn't include his first name at all: including it he would be Gaius Julius Caesar. The cognomen (there Caesar), meanwhile, was a bit of an inconsistent mess that in various times and contexts could be personal, hereditary, honorific, given, adopted...
[1]: https://en.m.wikipedia.org/wiki/Roman_naming_conventions
So Caesar's close relatives were all also called Caesar, which is how it got to be an imperial title. You might think that with the original cognomen locked into place for such families, members would get a second cognomen so that there would be at least one part of their name that could identify them. But apparently not.
The whole part about the Roman naming system indicating heritage became even more complicated when Caracalla (properly named Marcus Aurelius Antoninus) made every free man in the Empire a Roman citizen and now basically everyone was called "Marcus Aurelius" without ever having had any ties to the Aurelii family.
There is very likely no connection. The introduction of surnames in Germany took place in the 15th century (somewhat earlier in German speaking parts of Switzerland). In Franconia, the "ethnic" distinct northern part of which is today Bavaria and which was never a part of the Roman empire, the last-name first custom also occurs in a few places such as Bamberg.
In inscriptions (short): CAIVS IVLIVS CAESAR
In speaking he would (in English;)) rather formally introduce himself as "Gaius of the Iulii, called Caesar".
His mother called him Gaius, his peers called him Caesar once he got or choose?) this name.
His father was 'Gaius Julius Caesar' his grandfather (supposedly the first 'Ceasar') was 'Sextus Julius Caesar'.
The Cognomen is presumable something that evolved as a way to tell apart people belonging to different branches of the same family/clan because the number of first names was very limited.
Initially, it was more of a nickname or even an earned name, like with Caesars contemporary Gnaeus Pompeius Magnus.
Nicer than his other nickname 'adulescentulus carnifex ("teenage butcher") ' I guess..
But yeah, the cognomen seems weird/hard to understand e.g. Pompey's dad was called Strabo but he seems to have never used that name himself, while other like Ceasa, Metellus or Cicero were pretty much just surnames.
Romans probably found their naming system as confusing as we do so they started using nicknames ad hoc.
It was even more awful if you were a woman, you technically didn't even have your own name until the imperial period. You and your sisters would just be called Aemilia I, Aemilia II, III etc (or the 'elder'/'younger'). if your father was belonged to the Aemilius clan for example...
Only that... it didn't? The vast majority of bureaucracy and even many private entities in Italy use and strongly prefer the "Surname Birthname" format.
It's probably a Roman thing. Julius Caesar was actually "Caesar, of the gens Julia", so last-name first.
In Italy we also used to list last-name first, particularly in official and military communications. This has changed significantly in the last 40 years, particularly in everyday bureaucracy, but it's still used by police and other bodies.