The imitation game is adversarial (both players try to convince the interrogator that they are the woman, or the human). GP was talking about non-adversarial settings.
"Warm Worlds and Otherwise" is a significant text.
I'll grant this qualifier for purposes of discussion. In that case:
- if someone writes using adversarial methods, can that person's gender still be detected? How much text is required?
- Here are male authors who wrote romances using a female pseudonym - https://www.goodreads.com/list/show/104973.Male_Authors_Who_... . Can we presume they used other adversarial methods? I do.
- How many people are trained to write using an adversarial style?
(Quoting the top Google hit for "write like a man" at https://www.timsquirrell.com/blog/2017/3/29/write-like-a-man... : """Teachers often make up for this by telling their female students, either explicitly or implicitly, to "write like a man". This means that they should write confidently, "objectively", not hedging their arguments, coming down boldly on one side of an issue, etc.""")
- Are all gender mis-identifications due to the authors using deliberate adversarial techniques?
It's clear that many factors contribute to one's choice of language.
In the mid-1950s people studied the differences between the English spoken by the upper class ("U") and middle-class ("non-U") in the UK - https://en.wikipedia.org/wiki/U_and_non-U_English .
In the US, a Black American may grow up with speaking African-American Vernacular English (https://en.wikipedia.org/wiki/African-American_Vernacular_En...), and code switch to speak a more general American dialect.
My freshman year roommate spoke more "Southern" when on the phone to family than when talking to others in the dorm.
The language of a scientific monograph is quite different than the language of an HN comment, and being able to write in that style suggests an academic training.
So I'm not surprised that gender can also be a factor in word choice. Indeed, a friend of mine who reads romance novels claims a common difference between male and female authors is how they describe clothing.
If you want to describe all of those as different languages ... I've long given up trying to understand the differences between 'dialect' and 'language' (much less 'sociolect', 'ethnolect', 'stylistic register', etc) so go right ahead.
But I'm hesitant to believe that "Any significant text" predicts a specific author's gender "with considerable confidence."
I know of several cases where people tried and failed to guess an author's gender. Robert Silverberg, for example, found "something ineluctably masculine about Tiptree's writing"; James Tiptree, Jr. was the male pseudonym of a female author. (https://en.wikipedia.org/wiki/James_Tiptree_Jr.)
Searching Google Scholar, I find articles like https://arxiv.org/ftp/arxiv/papers/2109/2109.13890.pdf saying:
> However, other gaps need to be addressed as well, including that no method of classifying age or gender—both of which are aspects of author profiling—that also considers the genre or nature of the analyzed text has been widely endorsed. Beyond that, to the best of our knowledge, no research has involved surveying or comparing the different approaches used in author profiling. In fact, current knowledge about the task is largely based on small-scale experiments conducted to find a reliable classification method with a near-zero error rate. In order to use forensic authorship profiling as admissible evidence in courts, the proposed methods must have about 100% accuracy. The average accuracy of good proposed methods are ranging from 70-85% [5]–[9]. However, the proposed methods still suffer from low accuracy.
That suggests that the "with considerable confidence" you refer to is on a population level, and not a per-person level. Rather like how, with considerable confidence, I can say that in general men are taller than women, even though many women are taller than many men.