Yeah, the point taken from the study is really designed to stoke tension when it is not really reflective of anything meaningful when it comes to job applicant filtering.

It reminds me of a learning from one of Thomas Sowell's books. There were communities that thought that background checks for employees in certain industries were racially discriminating by their very nature. They moved to ban them. Race disparities in those industries consequently went up, not down. The reason? Without an impartial and indifferent scan of a person's background, employer's were left to use the signals they could pick up on in person, filtered through their own biases. The solution was to allow background checks again.

It sounds like this study is basically saying that without the background check (in this case, actual interviews assessing qualifications), that people are more likely to filter through whatever biases they have at their disposal. Sounds like no surprise... That's the whole point of the interview, to be the impartial and indifferent filter that anyone would be judged by. The solution is to just apply the interview and not judge based on a few seconds of dialogue. Which nobody does anyway. So it's not testing for anything real.

(Cue many knee-jerk anecdotal counterpoints to "nobody does that anyway." Please. You're not getting that software engineering job by being an artist with no prior experience who flashes high-class speech for 10 seconds.)

> So it's not testing for anything real.

You are too quick to dismiss the study, I think. What the study shows, I think, is that other things being equal the person who sounds to be from higher class is likely to get the job. The study simulates the other things being equal scenario by eliminating all other avenues for the hiring manager to judge.

Here is the quote from the TFA, lightly edited for emphasis:

  The study, ... , demonstrates that:

    (a) people can accurately assess a stranger’s socioeconomic 
    position — defined by their income, education, and occupation
    status — based on brief speech patterns and 

    (b) shows that these snap perceptions influence hiring managers 
    in ways that favor job applicants from higher social classes.

> The solution is to just apply the interview and not judge based on a few seconds of dialogue.

Agree, but ALSO be aware of such biases and try prevent those.

> What the study shows, I think, is that other things being equal the person who sounds to be from higher class is likely to get the job.

In an interview where you're not allowed to evaluate "all other things".

In other words, absent any other information, people believe those from a higher class are more likely to be competent. Is that true? The link says nothing about it. If it is true, the hiring managers did a good job at estimating average competence from social class based on their knowledge (i.e. "bias").

It does not show that in an interview process where you are allowed to evaluate the candidate on factors that matter for the job, higher class matters at all (i.e. given two equally qualified candidates that are interviewed by an experienced hiring manager, the higher class one gets better offers/evaluation of competence).

And this is the bias that matters because it applies to the real world.

I wonder what the class breakdown would be if the hiring managers had access to all other relevant information about the candidates, EXCEPT socioeconomic class... I'd expect the more 'upper class' candidates probably would do better on average.

Do you have a hypothetical explanation for why, if people are only allowed to evaluate speech and accent, they prefer a certain group, but if allowed to evaluate lots of different more meaningful things, which all lead to the same conclusion about competence EXCEPT for speech and accent, that preference no longer applies?

That seems like an extraordinary claim unless you have some evidentiary base for it.

Sure, it's very credible that a general tendency will remain if other sources of information are available. The unresolved question is what weight is assigned to each preference.

If a hiring decision is influenced to 0.01% by speech and accent and 99.99% by whatever you deem reasonable criteria, then what do you learn from filtering out the latter?

And of course, decisions like this need not have linear behavior in reality either. A gross mismatch between different indicators could result in significantly worse hiring manager opinions than the (weighted) sum of the individual aspects.

Imagine that a candidate shows moderate (but sufficient) aptitude but then tries to disguise it with "high brow" rhetoric. In isolation using such speech may be considered a positive, but in this case it will be probably be to their detriment.

Isn't the existence in a more realistic hiring process of a huge weight of objective and reasonable criteria an unjustified, unlikely and politically subjective assumption?

Given that one small element of interview presentation has been shown to involve irrational bias, perhaps it's more reasonable to assume that most others also involve unconscious or explained-away bias? So perhaps hiring decisions are influenced 0.01% by speech and accent, 64.99% by other class, race and gender markers, and 35% by reasonable criteria such as competence? Why wouldn't this be your base case assumption?

> Given that one small element of interview presentation has been shown to involve irrational bias

No such thing was shown in this study. The study makes no attempt to measure E(skills | social class). For all we know, the hiring managers provided an unbiased (i.e. the opposite of "irrational bias") estimate of the expected skills conditioned on the social class of the applicant, absent any other information.

Interesting that you are claiming both that people making hiring decisions are not biased along class lines and that any such bias would be rational and based on ability.

Exactly. It's not discrimination if it is objectively true.

I would certainly want to see proof before I accepted it was true. And even if it were true, we might find it socially undesirable for people to act on (it's "objectively true" that hiring disabled people costs an employer money too). I have previously seen a similar study to this one where landlords were able to determine a prospective tenant's race in brief phone conversations and were less willing to rent to minority tenants. Was that also just sound, evidence based reasoning?

Few challenge that the children of highly educated parents have higher academic achievement. As highly educated parents is strongly correlated with upper class. This is enough to make it quite likely that someone from an upper class background will be the better candidate in expectation, even if the only information you get is the socioeconomic background. We can quibble over details, but the more you dig into this the more convincing this will become. It does not matter as you already believe this.

With regards to minority tenants, institutionalized racism often leads to the case that non-racist agents will have sound evidence based reasons to discriminate. Not based on the properties of the group discriminated, but based on the behaviour of a third party. A decent example is if banks discriminate against a group, then their availability of credit is worse. This can create self perpetuating circles of discrimination, which can be sustained even if all involved organisations and individuals aren't racist. For a hilarious example of something similar, see: https://i.snipboard.io/kdu77.jpg This is a communication and organisational failure, not racism, but its difficult to distinguish, but critically yes, it means it could be.

Non-discrimination and positive discrimination is best viewed as taxes designed to address this kind of problem, and its working. But the habit people have of challenging this kind of thing as a kneejerk reaction is harmful. What you are doing is essentially claiming that their judgements are corrupt or trying to shame by implied association, and that isn't helpful.

As shopworn as the term is, this is what is meant by "institutional racism." Racist sentiment is not necessary for people to behave in what is ultimately a racist way.

I would also challenge your notion. It is true that wealth is correlated with educational achievement. But educational achievement will not, in itself, lead to someone speaking in a way that is more patrician. Is it the kind of society we want to create where you are essentially "locked in" to your parents' social position? It may even be the case that, assuming two people with the same education, the person from a lower class background is more likely to be exceptional.

!As shopworn as the term is, this is what is meant by "institutional racism." Racist sentiment is not necessary for people to behave in what is ultimately a racist way.! With the way you put it now, the people living within such systems would be racists taking racist actions based on sound evidence based reasoning. The term itself is also problematic, as no institutions nor race need to be involved for this kind of problem to occur, its simply the most familiar form to americans. Inadequate equilibria is better, but the corresponding the problem xxx would disappear if people weren't idiots isn't productive either.

Just like it is not necessary for the upper class to be more educated, its not necessary for the more educated to speak upper class. But it is more likely.

That such would be enough to stagnate social mobility seems like pessimistic hyperbole to me. Given that we are talking about high value added, high paying, i.e. worker shortage jobs and that there are significantly more demand for such workers than the upper class could provide, meaning, even at worst, such behaviour could only slow down social mobility, not stop it.

If the people who are evaluated are drawn randomly from the population as they claim to in the study, then no. If its a real hiering situation, then possibly yes, though it depends on how the position was advertised. That however may not even change the outcome of the study. In most big companies recruiters are not looking for the best applicant for the job, or atleast have no incentives to do so. They are looking for the easiest to motivate hiring for the job. Similarly, their boss may not have ordered them to find the best worker for the job, they likely asked for a good enough worker who causes minimum issues and who if they cause issues were easy to motivate having hired in the first place. Note that I never claimed that upper class speech meant better at the job, I explicitly wrote academic achievement, i.e the kind of thing which makes for an easily justified hire. Imagine having hired Dr House except without a degree, or a slightly below average doctor with an average grade degree. Now one is plain better, but the other will never get you fired. But thats a whole other problem.

> Is it the kind of society we want to create where you are essentially "locked in" to your parents' social position?

No, that is why interviews should look more like the ones Google performs and less like "please describe yourself, hobbies and life experiences"

Yes, I agree. For all the complaints, I think it is a much fairer system than the alternative.

> The study simulates the other things being equal scenario by eliminating all other avenues for the hiring manager to judge.

I think the issue here is that if the hiring manager only has one variable on which to judge, of course that variable will be used. Whatever that variable may be.

Here is the speech, which is a proxy for class, which is a delicate topic. What if the candidates only differed in the color of their shirt?

I think the issue is not so much to see whether certain criteria matter "all else being equal". It probably does, and the fewer different criteria there are, the more a particular one will matter.

What's more interesting to me, though, is how much a given criteria matters compared to others. Especially criteria I cannot influence (say my race) versus others I can (say my coding level or even speech). And this kind of study cannot show that.

Should we all show up to interviews wearing lime shirts if some study showed that's what a bunch of hiring managers preferred when they had nothing else on which to judge? And how important is that compared to, say, knowing how to code a FizzBuzz? Given limited resources, should I invest in learning to code or in finding the greenest shirt possible?

> education

I.e. something that does matter.

I notice with wry amusement that my own speech patterns vary depending on who I'm talking to. For an obvious example, using simpler words and a higher pitch when talking to very young children. But I use different vocabularies when talking to different groups of adults, too. This is all quite unconscious. I bet we all do this.

More dramatically, https://en.wikipedia.org/wiki/Code-switching

This is how research works, you isolate variables. By OP’s logic science is never testing for anything “real.”

I’m not surprised that HN rushes to discredit the idea that hiring is not a perfect meritocracy, but this is kinda silly.

But this is also not how isolating variables works. If you give someone no information, and require them to make a decision based only on speech patterns, that does not actually tell you very much about how they would make a decision if they had lots of information including speech patterns. Compare:

a) only speech

b) lots of complex skills-based information that is (verified in a separate experiment) equivalent on balance, plus also speech

It's quite possible that you could see a large difference in decisions in (a) without any detectable difference in (b).

For example, imagine in this study hiring managers are (perhaps subconsciously) estimating education level from speech. If they already have lots of details about candidate's education, that "screens out" the rough proxy of their speech patterns.

Isolating this variable allows one to study how good hiring managers are at estimating base probabilities for education/competence given social class (news alert: they are not independent).

It says nothing about what actually matters: do hiring managers with the ability to properly interview "equally qualified" candidates from different social backgrounds do so effectively (i.e. offer them similar compensation), or do they still rely on the base rates (i.e. bias), and how big is that effect?

> I’m not surprised that HN rushes to discredit the idea that hiring is not a perfect meritocracy, but this is kinda silly.

I think this is a straw man argument, unless you can point to more than one person here (now I see none) arguing hiring is a perfect meritocracy.

I see lots of people questioning the methodology and conclusions drawn, but I don't see anyone claiming hiring is perfectly meritocratic.

On the one hand, if we need to perfectly understand the problem before starting to solve it, we'll be forever stuck in analysis. On the other hand, if we are incorrect in our initial analysis, we risk wasting time and other resources on the wrong solutions, reducing resources available for other solutions further down the road. It's very important to distinguish constructive criticism from animosity.

Questioning the validity of the isolation, and conclusions drawn from the result is how science works.

Well, the experimenters could have combined their corpus of job candidates describing themselves with a bunch of recordings of actors giving the same descriptions in different accents.

Then each hiring manager could have been presented with a different subset of that pool, with both resume and recording.

A multivariate regression would then reveal the relative impacts of class, being an actor, and the different self-descriptions.

>I’m not surprised that HN rushes to discredit the idea that hiring is not a perfect meritocracy

I'm surprised you evidently haven't seen the thousands of threads where HN predominantly does nothing but complain about how hiring is not a perfect meritocracy.

> ... Please. You're not getting that software engineering job by being an artist with no prior experience who flashes high-class speech for 10 seconds...

You make it sound like that kind of thing is impossible. It's NOT. Class signals, in many cases, are absolutely critical to landing a job. You give an example of "an artist with no experience" grifting into a SW engineering job-- that's a bit extreme. What is common, however, are corporate HR/hiring-managers and even entire professional networks that dismiss otherwise qualified people based on class signals.

You people are thinking of software engineering, of course because this is HN. But keep in mind that for some people, that's just "the help". There's a whole world of business jobs and networks where even showing up with the wrong suit will get you instantly, silently, and permanently dismissed unless you have some incredibly strong connections to that group. I know this anecdotally because I grew up in my father's tailor shop!

This isn't a shocking or unbelievable finding.

It's been well known for a very long time that for some the education you receive at Yale or Harvard is more about how to be a member of that class, to learn the manners, speech, and dress, that will win you the job at companies playing by those rules. The "Gentleman's C" you earn is less consequential.

Ironically, HN frequently touts the benefits of "just talk to someone about their work" over the impartial whiteboard interview.

I far prefer the Software Engineering industry's focus on design and coding interviews. Impartial. Unrealistic. But fair.

Software coding interviews can be extremely subjective. I have seen someone rate similar responses vary differently based on their overall assessment of the candidate. It’s also extremely difficult to be equivalently helpful when answering questions.

Even deciding what’s a fair assessment is difficult. Should you give someone more slack if they haven’t used a language in 2 years, but remember most things well enough?

Fair enough but "just talk to them" is clearly worse.

Unfortunately once you move past the lower working level software/engineering jobs, either vertically to team lead positions or horizontally to product management or marketing or sales, whiteboard interviews are not that helpful, as soft skills becomes as important if not more important than technical skills for success at work.

Not only that, but whiteboarding very rarely tests for skills actually needed at the job. As you progress in your career, things like "I have held down the following software development jobs for X years...." are going to be far better ways to judge technical capabilities that are actually relevant to the job than 30-60 minutes of whiteboarding.

Many many different industries have jobs that require technical skills -- not tech as in technolgy -- and they get away just fine with managing to hire people. Nurses need to be able to nurse, a highly technical skillset; no one is having nurses give fake shots on test dummies during interviews. They use credentials and experience, and accept the risk that a hire might not pan out.

The fact that we want senior level developers to have to refresh/cram before interviews on how to implement graph search algs and hash tables just so they can pass some artificial whiteboarding section is a bit crazy. Their actual job experience over the past N years should be what counts.

Now, conversations about how one would solve technical problems of a sort actually faced on the job are totally on point, and really should be a part of any job interview in any field.

edit: Really, what we want as engineering types is for there to exist a way to gauge ability in a purely objective way. But there isn't one; it doesn't exist. Furthermore, you are building a team of people. IMO the focus should really be on fit and soft skills regardless. Make sure they meet a minimum bar of technical ability, then work to verify whether they'll be a productive, beneficial teammate.

Credentialing for software engineers (such as licensing and continuing education for nurses) has its own set of problems, both in terms of gate-keeping but more importantly in the type of work being entirely different.

There’s a spread in performance from the +1σ to the -1σ nurse, but both will be functional in a nursing role, so checking credentials is a mostly sufficient evaluation of occupational ability.

The spread in software ability is so much larger that the -1σ candidates (the least-effective ~15% of people who have ever worked in computer programming jobs) are worth trying to select against in a way the least effective ~15% of nurses probably are not. (I’m not in medicine so I’m not sure; apologies if the same effect is at play there as well.)

> The spread in software ability is so much larger that the -1σ candidates (the least-effective ~15% of people who have ever worked in computer programming jobs) are worth trying to select against in a way the least effective ~15% of nurses probably are not.

This isn’t an argument against credentialling since it is literally the direct effect of not having credentialling in software. Credentialing cuts most-to-all of what would be the low tail of the profession off before it ever enters. Nurses and lawyers who can’t do their profession’s equivalent of fizzbuzz mostly wash out of the exams to enter the profession (or earlier), they don’t show up in job interviews, except where the inability is produced by some kind of post-entry decline in capacity. And even then, decertification is a real possibility, which further purges the low end of the distribution.

So, yeah, a -1σ software developer (or at least software developer job applicant) is worse compared to basic expectations than a -1σ nurse, because a nurse as below expectations as a -1σ software developer is would be compelled to find a different profession because of credentialling requirements.

You make a good point about an idealized credentialing mechanism cutting off the left tail of the distribution, but the spread in performance will still be much higher in credential-holding SWE than nursing and, because of the high leverage, still worth selecting for.

You can only be so much better at nursing than the average qualified nurse. How many more patients will the best nurse save? You can be several multiples (whether it’s 10 or not, I won’t get into here, but I think it’s at least 3) more effective than the average in SWE.

> You can only be so much better at nursing than the average qualified nurse

I...disagree rather strongly that this, to the extent it is true, differentiates SWE from nursing. Comparing workers with identical responsibilities, the same is true in SWE, in both cases there are large impact multiples possible within the field, but they come from roles with greater responsibility; in nursing, because different roles require different credentials, this is reflected in the fact that a more competent nurse will be more likely to take on a more responsible role within the same credentialed category, and also be more likely to acquire greater credentials over time compared to a less competent nurse. “Nursing” isn’t a binary credential, but a whole set, starting with a basically heirarchical set: CNA -> LVN/LPN -> RN and beyond that various Advanced Practice Nursing credentials.

SWEs likewise have wide variety of scopes of practice and responsibility, but because we have no structure in the profession, they are assigned ad hoc and even with common titles there isn’t consistency in substantive roles, skills, or qualification. But, again, that’s a product of the absence of structure and credentialling, not an argument for it.

Whatever the entry-level credential is, SWEs as a set will have a spread of productivity/capability that’s wider than the entry-level nurse.

Whatever the highest-level credential is, SWEs will have a spread that’s wider than the highest-level nursing credential (and likely wider than the ratio in the entry-level SWE range as well).

> Whatever the entry-level credential is, SWEs as a set will have a spread of productivity/capability that’s wider than the entry-level nurse.

I see, and you have presented, no reason whatsoever to believe that this is true.

In the right position, excellent SWEs have enormous leverage. A median SWE might maintain some line-of-business app for an insurance company, and an excellent one might create software that is used by an entire industry (say, Redis just as an example).

By comparison, given the physical and time constraints of the job, an excellent nurse might be able to spread their expertise to 2x as many patients as a median? 3x? I think even 3x is a stretch. Maybe they can make a life-saving catch in the hardest 5% of cases (definitionally, very rare opportunity to apply this level of skill)?

Knowledge workers have a scale effect that others don't. Software stuff has a scale effect.

This is why a top PI or scientist can literally save hundreds of millions (as Norman Borlaug did) but the best surgeon can save mere thousands in their life at best using surgery¹. There is a scale effect.

Likewise, SWEs can write code that can apply to all people.

Same with the top public policy people, for instance. It's the length of the lever and where the fulcrum is placed.

No nurse has that lever.

¹ I hate having to handle the edge-casers, but scientists can come up with new surgical procedures (as Werner Forssmann did) and that is way for them to use a long lever.

Who is the Jon Carmack of nursing? How many more patients did they treat in a shift than the median nurse?

(Jon would have started with entry-level credentials at some point.) Substitute with Dennis Ritchie, Linus Torvalds, Bjarne S, Peter Norvig, James Gosling, Brenden Eich, rtm, rms, or whomever your idea of someone obviously on the right tail of the distribution is.

I'm sure there are famous and outstandingly accomplished nurses, just as there as there are for SWEs. My claim is that there are likely more of the latter and more continuous distribution of them.

> The solution was to allow background checks again.

I'm not sure this learning shows what you think it does, nor that the 'solution' is really a solution. Employers don't want to hire black people because they fear they might be criminals, and background checks allay this fear, allowing them to hire a subset of black people they feel are ok? To me this confirms that both background checks and hiring managers have a tendency to be racially biased, and the 'solution' is simply accepting of that bias.

Another perspective is that a lot of bias isn't "I don't like X people", but rather "I feel X people are disproportionately Y, and I don't like Y". Objective data that this X person isn't Y seems like a good way to (1) reduce the harm of the bias in the short term and (2) hopefully diminish the bias over time, given enough objective data to fix their biased priors via observation.

If fixing people's biases via objective data isn't the solution, is the solution figuring out which people are unbiased and only hiring sufficiently unbiased people? What if evidence suggests that there aren't enough sufficiently unbiased people in the world? Is unsubstantiated rhetoric preferable to objective data in breaking down people's biases?

Of course the ideal solution to bias is to just not be biased in the first place, but that's not an implementable solution.

> Another perspective is that a lot of bias isn't "I don't like X people", but rather "I feel X people are disproportionately Y, and I don't like Y"

Of course. But the answer to this depends on: 1. the source of the belief that 'X people tend to be Y'. 2. The truth or otherwise of that belief. 3. The source of the dislike of Y. 4. Whether or not Y is a reasonable cause for dislike.

Suppose that I tell you that the most important political issue for me is the welfare of animals, and in particular, that animals raised for meat should be stunned before they are slaughtered. There are definitely some people for which this is a genuinely held and rationally arrived at belief, consistent with their other beliefs and actions. There are others for which this is a device for attacking certain religious communities, through opposition to halal and shechita. Isn't it correct that we should try and understand which is taking place before responding to it?

Presenting objective data to someone arguing an irrational position in bad faith does no good - the bad faith argument simply adapts.

Sure, but we usually group these sorts of disagreements as axiomatic disagreements, not bias.

Maybe I'm grossly naive, but I doubt that percentage wise there are many hiring managers who hold racial superiority as axiomatic.

My understanding is that hiring bias usually boils down to assumptions about aggregate intelligence, trustworthiness, and/or work ethic of groups of people. If someone thinks Swedes just aren't that smart, it's not a matter of axiomatic disagreement about if being smart is good or bad in an employee, it is something that objective data can fix. If someone is worried that black people are generally untrustworthy due to criminal pasts, that's something that they can be objectively shown to be untrue.

(Yes, I grew up in a time and place where racist/originist Sven and Ole jokes about dumb Swedes were common.)

This is far too simplistic. There are well studied phenomena by which people apply bias along the following lines (an arbitrary example chosen):

1. I believe a stereotype that Swedish people are more friendly than Scottish people. 2. I know this person is Swedish, so I will tend to judge their behavior as generally friendly when I would interpret the same behavior by a Scottish person as less friendly. 3. I tend to assume that Scottish people do better in this job than Swedish people, perhaps because I've seen more Scots and fewer Swedes succeed in the past. 4. I therefore believe that success in this job is anti-correlated with friendliness. 5. The person in front of me would perform poorly in this role - not because he is Swedish, but because he is too friendly.

Note that although in practise, I might reject a Swede and accept a Scot (who is exactly as qualified and exactly as friendly) for the same job, I don't have any dislike of Swedish people, nor do I seek to avoid hiring them. In fact I might personally prefer Swedes to Scots (because they are so friendly) and seek out their company, yet still behave as above! That's literally the meaning of unconscious in unconscious bias.

1. That's very interesting. Thanks for enlightening me.

2. I think the case of the background checks is, however, one of the simple cases. It looks like some managers have an irrational belief that they're interviewing people likely to have criminal convictions, but change their minds when background checks come back clean. Hopefully as more and more background checks come back clean, managers adjust their biases.

On a side note, there is something to be said for giving convicted criminals a second chance. I'm not sure if people with a single criminal conviction are that much more likely than someone with a clean record to commit a crime that harms their employer.

It's worse than that - the hiring managers were accurately identifying things that matter. The article isn't saying that the hiring managers were unfairly biased, but that they were somehow able to pick up on traits (their level of education, their level of professional success) that they'd otherwise care about.

It'd be more worrying if hiring managers picked up on thing that didn't matter (their parents' level of education, controlled for their own, their parents' income level, controlled for their own, where they grew up, again, controlled for other markers of professional success) but the article isn't saying that's what the study found.

To me it sounds like they were going to come to the same conclusion no matter what. If the hiring managers were unable to identify any meaningful signals and discern better credentials from the recordings, they could've spun this as: the hiring managers, unable to discern things that matter, picked up on irrelevant cues and discriminated against people with certain irrelevant characteristics (it's inevitable that some traits would be correlated with the hiring managers' selection). In this case, the hiring managers managed to discriminate in favor of better candidates - so they are just reframing better credentials as class.

It's quite common to infer social status from speech. Since the level of education used to correlate with social status, it made sense to have a preference. If that correlation still holds is hard to tell, but I wouldn't be surprised.

It's wild how people with different assumptions about the world can read the same thing, focus on the same part, and walk away with opposite conclusions.

I read that as them trying to control for other factors and get a clean signal to demonstrate their hypothesis as a testable, verifiable, and falsifiable phenomenon.

But some people apparently think that the process of isolation IS the confounding factor that produces the results.

This makes things difficult. If they don't do this people will accuse them of not controlling for everything but when they do do it then they say "well yeah, you're controlling for everything else, of course you'll get that".

The point from my reading was all things being equal except for the subtle verbal patterns of class, the verbal patterns are a real influencing factor in outcome

It’s simply invalid science.

It’s equivalent to having variables x, y, z, setting x = 0 and then exploring the correlation between y and z. That’s a fair experiment, but you simply cannot generalise the conclusion outside of the tested range for x (i.e. for x != 0).

No it's not.

It's like having x and claiming y and z might exist. You set it equal to zero and then you demonstrate the existence of the others.

Every study I've ever seen dealing with messy social sciences has some kind of fundamental counternarrative. That's why there's follow-up studies to try to demonstrate things in different ways.

Take the Milgram experiment. Some participants have claimed they knew the machine was fake but didn't want to mess up the experiment so they played along. Do we throw that out then? Everything real is complex, we aren't dealing with carefully defined clean room abstractions.

The Asch conformity test with the lines, many participants expressed extreme indifference to the arbitrary exercise after the confederates disagreed. They simply did whatever was required to complete the experiment. Do we toss that out?

All experiments are like this. Every single one. That's why you have to do multiple different kinds and have a preponderance of evidence from a diversity of approaches. That's the whole point

The opposite baseline belief, that mannerisms, poise and confidence have absolutely nothing to do with class signalling and that class absolutely does not reproduce itself through social institutions, seems to me to be the more implausible hypothesis here.

There'd have to be some new hypothesis to explain why it so clearly looks like going to Eton or Harvard and growing up in the Hamptons affords one preferences in the job market even when the prospect employer doesn't know such details.

Personally I think that's the one that should be on the defensive. A confounding factor there is the one that needs demonstration.

If I had to study the effect on fuel economy of having the top down on a convertible, I'd want to control for speed.

But if I controlled for speed by simply setting the speed to zero, people ought to question my work even if my results agree with their expectations :)

That's not this. It's a false equivalency.

You've just restated the same error as the thing I responded to

The thing you'd be "zeroing" in this situation would be to find a specified road and weather condition and then demonstrate the existence of a drag coefficient by letting the car coast down the same hill with the top on or off and seeing how far it goes before stopping and how long it takes to get there

In your analogy you'd be up against the assumption that fuel economy is universally constant. Here, you're trying to show that it's not constant and the shape of the vehicle is part of the reason why.

You do this by removing confounding variables to demonstrate whether a phenomena is real or just a confluence of other things and merely presenting a convincing illusion.

We would want to take engine combustion and driving habits out of the tests. We would be pushing the car off at neutral down the hill. You want those noise creating events to not disrupt your signal

We're trying to establish if something is potentially real.

Coincidentally there's been a slew of bogus fuel efficiency additive products that have measurable effects because the people using them habitually change their driving style creating a false positive. Grifters have successfully been hawking this driving placebo scam for 50 years because when you bring all the other stuff in it's hard to tell what's real and what's just in the head. It's easy to falsely attribute things based on the stories we like to tell ourselves.

Also to extend the car pushing thing to the counternarrative example someone could claim that it was merely the tires were warmed up the second run, thus being more inflated and that's what caused a time discrepancy so drag coefficient is still an unsupported idea. No matter what you do, there's always a hole somewhere. That's why it's gotta be more than one type of demonstration. To borrow a French idiom, it's all about cherchez le creneau and constructing new experiments.

I would fully expect that receiving a good education would provide benefits even if you didn’t tell anyone specifically where you went.

I daily use my education to my benefit and very rarely does it come up where I attended.

"It’s simply invalid science"

So the whole peer review comittee was asleep, but you are here to tell us

I don't think it's generally sound life advice to outsource thinking to others.

In addition, judging by the topic, this was a sociology department study. Social studies are hardly even science, and the practitioners aren't exactly renowned for their statistical skills. I'm much more confident about my Oxford Masters of Science degree in math.

Can you be an expert in all things that affect your life? Have you studied nutrition, medicine, law, finance, construction and soil stability, to the extent where you can distinguish a phony research paper?

All of us rely on others in society to do their part and to get it mostly right. So if you are going to declare research invalid purely based on your judgement, you better have a bullerproof argument.

As an exaggerated example, I know a bloke who is "thinking for himself" and has concluded that a medicine is dangerous because it contains clorine, and clorine is toxic. So he has a set of facts, and they are correct, but the conclusion is wrong because he does not understand the basics of chemistry, and that once that clorine atom is part of a larger molecule, it has completely different properties.

If you study "facts" without understanding the field these mistakes are easy to make, all the more so if you approach from the high horse of a 'i am clever and have a brilliant degree"

But I’m not doubting chemistry or physics. I’m doubting sociology (not a science), and even more I’m doubting math/stats (the basis of experimental science).

> The point from my reading was all things being equal except for the subtle verbal patterns of class, the verbal patterns are a real influencing factor in outcome

No, because as other people replying to you have stated, being able to observe "other things" is the crucial difference between an actual interview and what happened in this study. Inferences about variable X, depend on whether you can observe only Y, or also A,B,C,W,Z.

Here is what an experiment measuring what you believe this one does would look like:

- Find participants with a range of skills/experience and pair up participants from low social class with participants from high social class according to skill

- Allow hiring managers to conduct realistic interview processes and grade the candidates

- Assess the bias due to social class by looking at the differences in average rating for participants in each pair. If you have enough interviews/participants, you can also estimate the size of the bias from social class as a function of applicant skill

- Issues: you do need another metric to quantify "skill" that does not rely on hiring managers performing interviews (education level, GPA, written tests, etc.)

Here is what this experiment did:

- Allow hiring managers to infer social class

- Measure the expected skill given social class assigned by the managers with no other information present

- Issues: decisions from "no other information present" and "after conducting a rigorous interview and all other (job related) things being equal" are not comparable

Wow I was about to make this more or less exact comment before I read yours. Including, by the way, that last sentence.