It's far easier for rich students to game GPA, college essays and extracurriculars than it is for them to game the SAT.

For GPA, they can hire a private tutor in the subject they're struggling with. There are services out there that will basically write your english essay for you / do your math/science homework.

For college essays, they can hire college counselors to help them draft a compelling essay.

For extra curriculars, they can hire a private coach, etc.

However, there is very limited evidence that SAT coaching actually increases your SAT score. The NACAC did a study on this and they found that average gains of test-prep students is ~30 points (this was when the SAT was 2400 points).

A 30 point increase out of 2400 points is not material to college admissions.

Out of the college essay, gpa, extracurriculars, etc. the SAT is the least influenced by your socioeconomic status. There obviously is an influence, but removing the SAT means more reliance on even more skewed factors.

Here's the study -> https://files.eric.ed.gov/fulltext/ED505529.pdf

For anecdotal/lived experience, to your point:

The rich kids in my (public) school were the ones afforded not only tutors, but also just the insight into extra-curriculars. They were the ones who had parents who knew to sign them up for college courses at ~15-16 years old to ensure they could maximize their GPA (5.0 scale, so getting a 5.0 was literally impossible unless you did this). This created a competitive disadvantage for kids who just really didn't realize these resources existed (plus, having parents that would support it + take you to the local college to take additional courses).

Not to mention that, but there were sports our school didn't participate in, but the richer kids had levels of access to that looked great on college applications - rowing and lacrosse specifically come to mind.

So, essentially - rich kids have many easier ways to pad their college applications. It's not that they aren't working hard(er) - they are. But they have easily acccessible opportunities that middle-to-lower-class students (like myself) did not have access to.

--

Coming to the SAT, I'm not surprised by that study. I wasn't afforded the same opportunities as these richer kids, but my SAT score was highly competitive with them. I was ranked 50th in my class with a 4.55 GPA (my 4.0s were gym each year, and I think one or two electives that weren't AP), but my SAT score was a 2300, which was relatively similar to most of the hyper-performant, wealthier kids.

--

This is all super anecdotal. I was definitely upset by all of this at the time - but it didn't affect my life very negatively. I still was able to get into a great school, and have a great career now. But these disadvantages certainly persist against others, and re-adding standardized tests likely will help level the playing field in my mind.

> Not to mention that, but there were sports our school didn't participate in, but the richer kids had levels of access to that looked great on college applications - rowing and lacrosse specifically come to mind.

Exactly. I went to smallish rural school. We had soccer, basketball, baseball. Youth soccer in my area was a rigged game where only people who were enrolled in the coaches summer camp would make the varsity team. Basketball had 75 kids try out for 12 spots. Baseball is ruthlessly competitive, and while I was a really good little league player, I had no chance against kids who were in multiple travel leagues, etc. I probably played 50 baseball games from age 9-12. The best players played at least 500.

My cousins went to a fancy private school. Everyone played on a varsity team; it was how they did gym. My older cousin did fencing, the middle one did basketball, the younger twins played squash. The squash guys sold themselves as a package and ended up getting a couple of ivy league places fighting for them.

I think we have a similar outlook on this. For me, getting good SAT/ACT scores let me "punch higher", and got me into some really good schools that I would not have be admitted to. Ultimately, I went to a state school, but was able to parlay the competitive nature of things to get a better grant package.

> Baseball is ruthlessly competitive, and while I was a really good little league player, I had no chance against kids who were in multiple travel leagues, etc. I probably played 50 baseball games from age 9-12. The best players played at least 500.

Same thing drove me out of baseball after age 12 or so. I had more than a little natural talent and put in some time drilling with my parents, so I could keep up with the kids doing the traveling leagues and such until (a little before) then. After that it became clear that my parents and I were gonna have to devote hundreds more hours per year (plus not a small amount of money), realistically, for me to keep playing. The gap was just growing way too fast, otherwise. What was left were bad teams/leagues where few players were really trying, so that's no fun, and ones for which I couldn't make the cut. Someone who liked it but just wanted to put in a high-side-of-normal amount of time and effort for a youth sport, had no place.

As far as extra-curricular activities go, like sports, etc., those aren't really the point. The point is for the candidate to demonstrate that they can accomplish significant things other than academics.

This can be anything. For me, I didn't do sports, or any school extracurriculars. What I did do was run a small business (paper route), used the money to fund my hotrodding efforts, was an Eagle scout (back in the days when that was something), etc.

Basically, you just gotta find something non-trivial to do that demonstrates motivation.

I went to MIT 20 years ago. Plenty of people smarter than me failed out. MIT seems to do a pretty good job of screening out people who won't pick up the material fast enough. In my experience, the ones who failed out were the ones who were plenty smart but didn't adjust fast enough to having to work hard for the first time in their lives. If you get into MIT, you've probably gotten special treatment from teachers your whole life, and not really had to work hard before.

My high school had just shy of 4,000 students in 4 grades. My senior year, I took slightly over a "full course load" at the local state university, plus went to high school 1/4 time. Technically, that wasn't supposed to happen, but administrators look the other way for smart kids. I wasn't really competing against others in my grade. People asked if I was smarter than the girl a year ahead of me who went to Harvard. She was my competition. I'm sure something similar happened with a kid a year behind me.

I knew that at MIT, I'd probably just be an average student. However, I really underestimated how hard it is to learn to work hard when you've been able to coast through your first 18 years, despite taking honors courses at the nearby state university, etc. I think the SATs are probably generally pretty good at measuring how quickly students learn, but there's a certain grit it takes to succeed at MIT that the SATs don't cover at all.

Edit: I'm also an Eagle Scout, but I came through after it became significantly easier. It seemed to me that probably at least 10% of the men at MIT were Eagle Scouts. If nothing else, it shows an ability to stick with something for at least a few years, despite it being uncool for most of your peer group.

I, too, had a disastrous freshman year due to my attempts to laze through it like I had all through public school. Fortunately, I was able to change before I was forced out.

I also got my comeuppance about being "smart".

At the time, being an Eagle wasn't cool anymore, either, and I never talked about it. I was reluctant to even mention it here. Also, these days, it seems that being an Eagle is a project for dad, while the kid is along for the ride. My parents had zero involvement with scouting.

MIT has a ~95% graduation rate, so most students really do graduate. And for the 5% that don't it's unclear how many dropped out due to the workload vs dropped out to found a company, etc. MIT has tons of internal resources to help you if you're struggling.

The shock for entering freshman is very real. I really like the practice of making your first semester Pass/No Record so that there's less pressure to try and get an A, and if you do fail it won't even be on your transcript. Second semester still treats F as No Record as well.

There's a certain subtle ego disorder that creeps up on you slowly when you're used to regularly being introduced as the smartest person someone has met, and you let that slowly become part of your identity. The people I knew who failed out had too big of an ego to seek help, and even were afraid to work too hard, because that made them feel less smart. They didn't outright brag, but were used to others doing their bragging for them, and had a kind of false modesty about them.

> What I did do was run a small business (paper route), used the money to fund my hotrodding efforts, was an Eagle scout (back in the days when that was something), etc.

If I were a college admissions officer strapped for time, I'd let the app through on proof of "Eagle Scout" and ignore the other two.

The only easier bet would be seeing the words "I'm Hungarian" on an app for a secret world-saving advanced math project.

Intention is irrelevant, the outcome is the same. A parent driving a kid to lacrosse practice every Wednesdays and Fridays shows as much potential to accomplishments as a parent asking their kid to help them with their under the table car mechanic job every weekend. Yet I bet only the former is evaluated as significant. I wonder why that is.

> Yet I bet only the former is evaluated as significant.

Are you sure about that, especially for an engineering school like Caltech or MIT?

I didn't play lacrosse, football, row, track, baseball, swimming, yachting, nope nope nope.

And was your extra-curricular activity both broadly available to lower class students (in time, cost, culturally, etc.) and a significant contributor to your acceptance?

There was no cost to joining the Boy Scouts. All you had to do was show up. About half of the troop was poverty kids. You didn't have to buy the uniform, most of the scouts never bothered acquiring one.

Your bet is based on anything? The second story is a potential sob story that plays better, barring subjective classist biases counteracting. That only points to objective test scores being a better measure.

Nope, just a feeling which is further reinforced by other posts on this thread (i.e. confirmation bias) such as:

> Standardized testing was pretty much the only reason I and many other working class folks I know could get into good schools -- I was never going to do a million side activities, and my summers were spent working, not building my academic resume.

– https://news.ycombinator.com/item?id=30833870

> I grew up as a low-income minority in a single-parent household and I ended up getting into good schools pretty much only due to my high test scores, which has been a life changer. Other than test scores, I couldn't afford to do any fancy extra-curriculars.

– https://news.ycombinator.com/item?id=30835611

Of course this is (somewhat) testable. I could ask different collage admission boards to summarize anonymized admission records of students where extra-curricular actives was weighted favorably for admission. And then group these activities based on how accessible they are to various wealth classes before counting them up. I don’t know if that has been done, and I’m not in a position to do it my self... so the best I can do is make a bet.

It’s been done.

https://press.princeton.edu/books/hardcover/9780691141602/no...

Review of above book

https://www.mindingthecampus.org/2010/07/12/how_diversity_pu...

If I was in the admissions dept, I'd look favorably on an applicant who worked at a job.

I had a paper route, which was run as an independent business. I had a territory, I signed up people in it to subscribe, I delivered the papers, I collected the money, I paid for the newspapers the newspaper company dropped off. How much money I made was entirely up to how I operated. If I was sick or out of town, I had to find someone to cover for me.

There's also a lot of room to disagree as to whether playing sports counts as a meaningful accomplishment. Professional sports are pure entertainment, and succeeding even at that is extremely rare. The best argument for caring about it is that it's better than nothing, and it's something that ensures more average people have a chance to get to MIT too, even if they aren't all that intellectually minded.

Succeeding in sports means you have put out focused effort over a period of time to accomplish something that nobody made you do.

This is worth something.

I disagree completely. You could literally give zero effort, focused or not, and sit on the bench of a winning team. On top of that, your parents could have 100% made you do those things.

You could. You can just phone it in at work too, but most people don't. Sports are a place where kids figure out who they are. Not the only place, but an important one to many.

My son is 11 and loves baseball, I've coached a few times as well and it's been a great shared experience. There are definitely kids in Little League / Cal Ripkin who are there because mom & dad said so. But... I've gotten to see my son and a few of his teammates build friendships and mentor relationships with the kids ahead of and behind him that are difficult to do in a school setting.

It's a big deal. When a ten year old stops and is there to help teach an eight year old how to do something, etc those are valuable skills/processes/habits to build. They learn to lose and how to practice.

Part of the "package" a student brings to an application is how they apply those experiences. You can send a laundry list of things, or use your essay/interview to tie it together.

> something that nobody made you do

Uhh, that is definitely not a given

This rings true with my Gen Z high school experience as of ~7 years ago.

I don't really have evidence for this, but it's always felt like SAT/ACT coaching doesn't improve scores so much as get rid of some of the "dumb mistakes" that cost you score.

It's gonna help you when the test writer was playing some gotcha tricks with phrasing or whatever, but if you don't understand the material, even narrowing it down to a 50/50 isn't going to get you a good score, and if you truly don't understand the material you probably won't be able to eliminate half the answers anyway. And they are absolutely aware of the "answer b/c if you don't know" nugget, that's nothing special either.

Also "adaptive difficulty" systems where the system throws harder questions at you after successfully answering the easier ones are basically the "elo rating" of academics. Everyone hates elo but... it slots you into a very statistically accurate ranking. If you score highly on Level 600 questions but you are failing on the Level 700 questions, odds are good you are somewhere between 600 and 700. My understanding is that's what SAT/ACT were moving towards about 10 years ago, that it would be computerized and "everyone's test is personalized" by the elo system probing your exact knowledge level from a bank of questions with difficulty scores dynamically based on how "similar ranked" students performed on that question.

>> I don't really have evidence for this, but it's always felt like SAT/ACT coaching doesn't improve scores so much as get rid of some of the "dumb mistakes" that cost you score.

As i learned in college, the real "coaching" was rich parents getting rich kids more time on the SAT by getting psychiatric diagnostic classifications that give you more time.

Time has been the real challenge on SATs for most people beyond a certain score threshold, and money can buy time.

I finished the SAT early, and used the extra time to go back to the beginning and verify each answer. More time wouldn't have done much. If you can answer the questions, there's enough time to complete the test.

I understand your perspective-- my test memories were all breezing through tests with copious extra time... But as an educator I've noticed that there is a wide variation in the amount of time needed for a test between students. For some tasks it is nearly an order of magnitude.

The students who are quick and on the competitive math team finish something in 6-7 minutes and some other students are doing correct work but not quite done in 45 minutes. More practice doesn't seem to make them much quicker, either.

And this is in students without a formal diagnosis that allows them to spend extra time.

[There was one time I crashed and burned on a test and ran out of time... where I didn't memorize enough of a big table of identities for a trig test and ended up having to derive everything from scratch]

I hypothesize that the student who took 8 times longer isn't going to do so well at MIT.

A typical exam at Caltech would be 4 problems and 2 hours.

I never memorized the trig identities. I simply knew them from using them a lot. And having worked enough algebra/trig problems, you can just see the answer in your head as you read the problem. (This turns out to be a big timesaver at Caltech, where every course was a math course. When you're dealing with calculus, you really need to have moved past struggling with trig.)

At some point in the last 40 years, however, they've slipped my mind.

> I hypothesize that the student who took 8 times longer isn't going to do so well at MIT.

Sure. But the grandparent's point was: if you're the student taking 3x longer, your parents can buy you a disability diagnosis that gets you extra time.

> I never memorized the trig identities. I simply knew them from using them a lot.

Yah, a reasonable course would make this possible. My analytical trig class was pretty heavy on obscure identities, and the first exams I was like-- no big deal, I know how these are derived, I can figure these out as I need them... For the purpose of that class, nope.

>> I hypothesize that the student who took 8 times longer isn't going to do so well at MIT.

You are assuming the effects of wealth stop at the SAT.

Are you suggesting that wealthy people can bribe the profs to bestow better grades on their students?

Or the grad students who do the test grading?

BTW, Caltech's testing was done on the honor system. That meant no proctoring, and it was entirely up to the student to adhere to the time limits, and any other instructions on the test.

You didn't need wealth to cheat. Any student could, and with half a brain not get caught.

I recall one physics midterm which 2/3 of the sophomore class failed, including me. I suppose that precludes there being large scale cheating going on.

>> Are you suggesting that wealthy people can bribe the profs to bestow better grades on their students? >> Or the grad students who do the test grading?

Not bribe. Hire as tutors with $. This happened pretty regularly at my college (Cornell) where ex-grad TAs were hired as tutors. You could focus on just what you needed to study if you could afford to hire them.

Here in the US we just went thru four years with President Trump (https://en.wikipedia.org/wiki/Donald_Trump) who could not even communicate with clarity and rattled barely coherent ideas. He graduated from Wharton, the most prestigious finance program in the US. He is just one case I thought of. Does everyone really think he was the most qualified candidate to be in the very small inbound class at Wharton? Does everyone thing he was actually qualified to graduate based on merit?

From my, albeit rather distant, recollection, if you desperately needed more time on the SAT you are probably already screwed.

For students in the category of "exam easy. finished the exam w/o any time issues", the whole sub-thread is irrelevant. You're going to ace the exam rich, or poor.

The sub-thread and discussion is about wealth bias for exam scores.

Coaching and studying improves SAT scores. People learn the type of questions they do poorly on and can study to improve. The SAT is a test you can study for.

The SAT is a test of your academic preparation, not Raven's progressive matrices. That you can study for it is not inherently a bad thing. Portions that are highly susceptible to coaching are bad, and that's why there aren't analogies any more.

If students learn the vocabulary and practice the math to do better on the test, at some point it just becomes the Key and Peele Heist sketch: "That's called a job!"

https://www.youtube.com/watch?v=jgYYOUC10aM

Agreed. I spent a bunch of time a few years ago doing home-coaching for our two teenagers as they rolled into SAT time. The way you get better at the math section is to genuinely fix whatever gaps might exist in your knowledge of algebra and geometry. As OP says, that "is not an inherently bad thing."

There's probably another 20 points that can be picked up by learning to read the questions very carefully -- so that you don't race to show how quickly you can spin-up an off-task answer that precisely matches the wrong question. Getting that right also "is not an inherently bad thing."

The verbal section is a bit more of a swamp, and there might be a larger element of gamesmanship there. But for schools like MIT, where math aptitude is the main event, I think keeping a math-focused role for the SAT can help a lot.

It identifies not just the elite-school wizards with lots of AP and math SAT 800s -- but also the teens from humbler public schools that didn't have an AP track, but whose 790s on the (pre-calc focused) math SATs speak to their ability to play at a higher level.

Apropos of analogies, I think the test-takers got rid of those because they can be ridiculously skewed to particular (affluent) cultures. For some people, it's obvious if yacht-to-dingy is akin to symphony-to-quartet. For people who grew up with less money, it is a total WTF moment.

I remember an analogy question that required knowledge of alcoholic drink formulations. I was way under drinking age, and had no idea what went into a martini.

You might be under drinking age, but you should have years of practice making mummy and daddy martinis at the end of their work days if you want any chance of succeeding at $PRESTIGIOUS_OLD_INSTITUTION

My parents didn't drink.

Unfortunately for a lot of people it seems to be the Rick and Morty heist sketch at this point

The top link is a study that shows that the improvement is pretty marginal in practice.

It does miss something: in specific ethnic enclaves SAT coaching is much more effective, perhaps because of a culture of out-of-school schoolwork and teaching beyond SAT prep. Those enclaves aren't particularly wealthy either (if I recall correctly it was a Korean enclave). Even then we're talking 70 points -- not nothing, but also not a radical transformation.

> Even then we're talking 70 points -- not nothing, but also not a radical transformation.

That would depend on the baseline score. For instance, if it was a 1510 baseline and then went up +70, then it would be useful.

I think it's usually not improvement at the highest levels.

I wouldn't be surprised if those improvements were mostly moving below-average scores toward the average, by giving deprived students basic skills that their "education" didn't.

In my personal experience, about seventeen years ago, retaking the test raised my score some 90 points (iirc) out of 1600, excluding writing section.

the relevant question in that in that case is: what would your average score be with and without prep over say 10 tests

And there are books available to help with a lot of that. (And I actually agree with the point that doing some amount of test prep/sample tests is helpful. But it doesn't need to be super-expensive/time-consuming. I do understand that the playing field has probably upleveled over the decades but it's still probably as democratized as any such thing is.

My own anecdotal experience confirms that sat and act tests are very studiable. Honestly even more than average tests just because there is so much material available to study with.

I don't know many tests you can't study for especially tests that are run on an annual basis.

Yeah, I have found that it improves 3 things. (this is for GRE, which is similar)

1. Basics: If you don't know standard permutations and combinations then knowing those formulae off the top of your head is nice to have. The language portions in particular, take a lot of preparation for non-native speakers.

2. Speed: Giving a decent number of sample tests helps put you in game mode for the real thing. It also acquaints you to the manner in which questions are phrased and their intended meanings. (big deal for non-native speakers) Lastly, it helps ease anxiety.

3. Gotcha-proofing: Every examination has some familiar gotcha patterns. Some training helps in looking out for them helps.

The gotchas are answers that match common student mistakes.

What I'd do is solve the problem without looking at the answers so they wouldn't bias me. Then look for a match of mine with one of the answers.

I'd say one of the valuable things an SAT tutor could teach is an attitude: to take initiative, and reject resignment to failure.

The biggest difference I noticed in how I would take a test versus other people I tried to coach is that I viewed the test as a fun game like a challenging video game level. And those who struggled on the test viewed it as dreadful judgement being rendered on them.

It's like when you can tell someone is extremely self-conscious while dancing: Beyond teaching them any actual dance moves, you have to turn off the part of their brain which is blocking their natural mental resources for problem solving, and that's often the fear they are inadequate to the task, will disappoint their supporters, and that it will hurt their future prospects.

> My understanding is that's what SAT/ACT were moving towards about 10 years ago, that it would be computerized and "everyone's test is personalized" by the elo system probing your exact knowledge level from a bank of questions with difficulty scores dynamically based on how "similar ranked" students performed on that question.

That is indeed where they're moving. They've recently announced that the SAT will be transitioning to a digital, adaptive test in the next 1-2 years. [1]

Notably, the upcoming iteration of the test will only be semi-adaptive, adjusting the version of the second half of the test based on your performance on the first half, rather than adapting to your performance on a question-by-question basis.

I suspect overall this will be an improvement in the accuracy of the results. As it stands, for students with a strong math background, a majority of the math questions on the current test are far too easy and cloud their results on the rest. With the recent removal of the CollegeBoard's math subject tests, high-level math students have very few opportunities to demonstrate their knowledge using a standardized metric.

[1]: https://newsroom.collegeboard.org/digital-sat-brings-student...

> high-level math students have very few opportunities to demonstrate their knowledge using a standardized metric.

AP math and physics exams? Or IB?

True that these are not accessible to everyone though..

I think that is great. Once you get past a certain threshold, the test loses prediction value. Giving the kids in the top 5% a way to differentiate is great.

My kid didn't get to take an SAT/ACT due to Covid. They did end up with a substantially lower PSAT score due to one of those dumb mistakes. They missed one "easy" question on the math portion while nailing almost everything else. The difference between National Merit Scholar Finalist and a "really good score".

They're doing great at a top 50 university now but it probably was the difference between waitlist and admit at two top 15 schools.

> SAT/ACT coaching doesn't improve scores so much as get rid of some of the "dumb mistakes"

If so, that still doesn't imply that rich kids with access to private tutors will necessarily do better on these tests than poor kids - just that anybody with the motivation to read a test-prep book will.

I live in a very rich town, like super rich. I work in tech so am rich by the standards of general population, but by the standards of Silicon Valley.

My child did very well on the ACT. All she did was buy a $35 practice book, which almost anyone can do. Literally this was it. Furthermore, she doesn’t know of a single friend or acquaintance that hired a private tutor.

All the nay-sayers belly-ache about this but for most children, even the “rich” ones, this simply isn’t true. It’s most definitely not necessary.

Considering the weight of the SATs/ACTs you would figure a student would at least buy a cheap book and put in two hours per week for a few weeks in practice. In practice, the vast majority of students who do this do quite well. The improvement is dramatic.

Everyone wants to always blame “the system” which yes, has an influence, but nobody wants to put ANY responsibility on the student themselves.

As one of those private tutors that the ultra-rich hire, I can definitely support most of what you've said here.

The greatest service that private tutoring provides is structure, accountability, and guidance. A dedicated student working independently through quality practice materials (many of which are cheap or free) can absolutely attain most if not all of the beneficial outcomes of preparing with a tutor/service.

I think when people look at inequity in college admissions, standardized testing ends up being an easy, tangible target, but not a particularly important one. If you want to look at how wealth impacts standardized test scores, focusing on paid preparation programs is missing the larger picture.

Wealthy students have had literate parents who can afford books and have the time to read to them when they're toddlers. They've gone to safe schools where the teachers can focus on teaching rather than making sure the students are well fed. They are surrounded by adults who have gone to college and can serve as role models for positive academic behavior. They have friends who are all taking the same tests and applying to the same schools to provide emotional and thoughtful support.

The collegeboard markets the SAT as measuring preparedness for secondary education. These students have been preparing for college their entire lives; is it any wonder that they score higher?

Thank you. You absolutely validate my own observations.

I went to college. My parents did, as did my extended family on one side. I saw both sides and chose this one.

I picked a town where my children would be surrounded by an environment where cool was defined as “good at school”. I didn’t want external influences contradicting my influence.

My child is dedicated, but I taught that dedication.

A parent doesn’t have to be rich to understand the importance of a good education. My mother came from a dirt poor background.

People never say it’s an advantage to have parents that care about education. They always put the blame somewhere else. The parents worked too hard, whatever. For different traits, some parents are just better than others. Academics is just one trait. Is the most important? No, but it’s one that’s easy to measure.

I agree. I spent some time as a Kaplan tutor. The students in my experience fell into a few buckets:

1. They already know the material/are serious and would do fine without tutoring (maybe some small help here and there) 2. They kind of care, but need structure to study. They probably wouldn't study or study effectively without being in a class or having someone for accountability. 3. They don't care and their parents are paying for someone to babysit them to study

Most students were in camps 2 and 3.

IQ is quite heritible. This is especially so when upbringing is not neglectful. People who land very high paying tech roles are probably average higher in IQ than the general population.

Your daughter likely has the benefit of strong academic genetics. A child like that who puts in effort (i.e. works through a book) is going to do amazingly well. I'd also argue that families who value academics are also more likely to buy the books to do a couple hours per week in.

IQ also fluctuates a lot, but yes, she is a lot like me (reads, thinks math/science is interesting, etc).

She bought only one prep book, for $35, with her own money. I guarantee the very poorest can get that one book, even from the library.

Parents absolutely influence children, otherwise what’s the point of parenting? This is a good thing. I’m just saying the $$ part is way overblown.

The $250k to put a totally unqualified student through school would pay for a lot of test prep hours at some impoverished schools.

It would likely take about 10 contact hours (1h/week, 10 weeks), basically enough coaching so that the result is not artificially low through under-preparation.

Let’s say it costs $100/hr all-in to coach 10 kids. So for $1k you can get 10 applications from motivated, underprivileged kids whose SATs are representative of their ability.

For one $50k annual ride you could run this in 50 low income schools and get 500 underprivileged applicants and then actually admit some of them who might benefit.

Rather than refusing to test, and wasting those resources supporting some kid who obviously won’t hack it.

Do you honestly believe that if you were from a very poor town, poor by the standards of african american neighborhoods in Detroit, that giving your child a $35 practice book would have a similar outcome?

My mother was an orphan at 15 in Honduras. When she had me, she shared a 400 square foot house with six of her sisters.

Look up the crime stats for Honduras and try again.

Also, looking at the very worst situations is not a valid argument. By that rational, what about the poor girl who is kept a sex slave in her basement by her father for twenty years?

Do you think sending your African American example to SAT prep will make a difference? Coming from the poorest and sketchiest towns in America? Such a child would likely be better served by starting at a community college. After that, SATs are not accepted.

My experience with "10 Sample SATs" which was my only prep was about 500 points of improvement. IIRC I was about a 1050-1100 on my first, and my best V/M combo was 1450 (back in early 90s, so pre-recentering). I actually did about 1380, I had a so-so actual one.

It probably is the best bang for the buck, if you're already 1-2 standard deviations on general intelligence. Because the test is a game like admissions is a game, so if you're smart enough to see the game, it's easier and most effective to practice with that.

For the "normals", I have no idea if it will work. But we're specifically discussing MIT, who do NOT want normals, I'm not good enough. They want 2-3 standard deviations people.

MIT should be able to weight for socioeconomic and location/environment given the amount of information required for financial aid and "the internet".

If this becomes ubiquitous, I can see suburbanites getting ghetto apartments for the address to game the weights :-)

Those places are sadly lacking even in the most basic standards of education, so their potential ACT scores are frankly irrelevant as is admission to MIT. You gotta learn to crawl before you can walk, and walk before you can run.

Not only is this analogy wrong (there are plenty of animals that run the moment they are born) but it is also deeply insensitive.

The reason people do worse from underprivileged is not only the lack of quality education, but a fundamental difference in the quality of life. Everything from the diet, to available time, and noise pollution, all makes a difference on education level. Our education system relies heavily on outside help, and poorer neighborhoods often don’t have the time and energy to give their kids even the most basic help with their homework.

> Everything from the diet, to available time, and noise pollution, all makes a difference on education level. Our education system relies heavily on outside help, and poorer neighborhoods often don’t have the time and energy to give their kids even the most basic help with their homework.

Well said. Thanks for elaborating on my point.

The analogy is correct because we’re talking about people.

Parents are the primary educators. When they aren’t educating, or need to educate on too many other things, academics suffer. This is news to nobody. It would be highly suspicious if environment had no influence on educational outcomes.

As a middle class going poorer higschool student (dad lost his desk job, mom had to start working again to keep the house, tried moving to cheaper city and it didn't work out, had to apply for scholarships for stuff, never had money to go out weekends, etc, etc) I think the thing resourceful people overlook that "rich" kids have in higschool and "poor" kids don't, is a space to study. Just a personal quiet space where you can deploy your book or laptop and write some stuff maybe with headphones maybe not but definitely without your parents screaming or the TV blasting stuff about social protests or whatever.

In college most of my other "poor" friends that made it either also had a space like that or were taught by their parents to study in public libraries or other spaces designed for concentration.

You aren't rich if you have to work and you're relying on what a high school student tells you who doesn't know all the details of the lives of other students, just what they tell her. In my experience as a student, other students will keep their tutoring and hand-ups under wraps.

I talk to a lot of parents. Most are quite open with their tactics.

>I live in a very rich town, like super rich.

>My child did very well on the ACT. All she did was buy a $35 practice book, which almost anyone can do.

I don't intend to belittle anyone's accomplishments, but there's a lot more to high ACT scores than a $35 book, no?

Indeed, and primary among them are parents.

> All the nay-sayers belly-ache about this but for most children, even the “rich” ones, this simply isn’t true. It’s most definitely not necessary.

I'd rather trust studies than your anecdote. Just saying.

Of course. Show me a study that compares motivated self-study with classes.

Two SAT tutors have responded to the thread and supported me.

>It's far easier for rich students to game GPA, college essays and extracurriculars than it is for them to game the SAT.

Disagree. I worked for the Princeton Review while in college back in the day. We would outright guarantee 99th percentile for one on one tutoring. If you didn't get 99th percentile the course was refunded or you could take it again. For classroom tutoring we would guarantee some improvement of I believe 200 (out of 1600) with the same refund or take it again option. Candidly, no one would pay for a prep course for a 30 point gain. These course are expensive. Some of them hundreds of dollars per hour.

The cited research is pretty fundamentally flawed.

"Although extensive, the academic research base does have limitations. Most notably, few published studies have been conducted on students taking admission tests since 2000. Only two studies have been published on the effects for ACT scores, and no studies have been published since the 2005 change to the SAT, which added the Writ- ing section among other changes."

This position also doesn't pass the sniff test. GPA is accumulated over four years of study. SAT/ACT is a single test. You can't retroactively improve your GPA by doing better in the past. But you can dramatically improve your SAT/ACT results.

> If you didn't get 99th percentile the course was refunded or you could take it again

Plenty of other domains have study programs with similar guarantees where the program has little to no effect-- it's a simple economic calculation: Set your rates so you make a good profit even if the program has no effect (even if you allow retaking the course, students won't be willing to waste their time and yours forever).

It would be more informative if you knew the actual before/after performance for the program. Elsewhere this has been studied (see links in the thread) and the improvement wasn't that substantial.

> You can't retroactively improve your GPA by doing better in the past.

Indeed, which means that people who's families have been carefully shepherding their education since they were much younger have an _insurmountable_ advantage in the GPA game. For people people who don't come from highly educated families, they only learn about the GPA boosting games as they start thinking about college years to late to take full advantage of them.

>Plenty of other domains have study programs with similar guarantees where the program has little to no effect-- it's a simple economic calculation: Set your rates so you make a good profit even if the program has no effect (even if you allow retaking the course, students won't be willing to waste their time and yours forever).

They offer a refund or to retake the course. It is indeed a simple economic calculation. The company would quickly go bankrupt if a sizable percentage of students were refunding. The company itself, and the location I worked, had excellent reviews. Even today it has a 4.9 star rating on Google Maps. Quite a high rating for a service that people in this comment section proclaim has no impact.

Intuitively, how could it be correct that people pay thousands of dollars on these courses and receive zero or near to zero benefit from them? If this were true no one would take the courses, the courses would be rated poorly, and the underlying business would fail.

>Indeed, which means that people who's families have been carefully shepherding their education since they were much younger have an _insurmountable_ advantage in the GPA game.

Sure, but the context is "which is easier to game." To game the GPA takes four years of study and prep and you must start doing it at the beginning of grade 9. Tens of thousands of dollars and preparation+work over multiple years. To game the SAT or ACT you must spend a few thousand dollars during grade 11 or grade 12.

> To game the SAT or ACT you must spend a few thousand dollars during grade 11 or grade 12.

"Or check out a SAT book or two for free from the library.", says my partner from an extremely poor family who got into college on the basis of her perfect SAT score and whom never would have qualified to a prestigious school on a GPA basis. (And whom was also admitted to law school on the basis of a nearly perfect LSAT, which she studied for only with free and extremely low cost used materials)

> Intuitively, how could it be correct that people pay thousands of dollars on these courses

People, particularly those to whom a thousand dollars isn't a big deal, spend all kinds of money on speculative and outright ineffective treatments. Including mystical mumbojumbo, quack medical treatments, and products and services which accomplish nothing except contributing to the 'identity' they present to themselves and others. ("I am a parent who cares, look I spent $zillions getting Jr the best opportunities!").

> To game the GPA takes four years of study and prep and you must start doing it at the beginning of grade 9. Tens of thousands of dollars and preparation+work over multiple years. To game the SAT or ACT you must spend a few thousand dollars during grade 11 or grade 12.

I think on this point we're agreeing to a great extent but we're drawing opposite conclusions. I agree SAT improves with focused study, though I believe that improvement is available for free (other than time and knowing you should do it).

You seem to agree with me that it is very expensive to hyper-optimize GPAs, requiring costs and actions extremely early and on a sustained basis.

My conclusion from this is GPA optimization relatively more available to students with more affluent families, because it takes more time, more money, and requires it earlier and more speculatively. -- we don't have an option where you can't improve your performance with the input of time,money, care but we can choose metrics where the available improvement is available to more people.

> Intuitively, how could it be correct that people pay thousands of dollars on these courses and receive zero or near to zero benefit from them?

Because the people paying have a sample size of n=1?

Look, any test will be subject to random measurement noise. If you got a bad night's sleep, got lost on the way to the test, had an argument with your parents or broke up with your boy/girlfriend, you're probably going to do worse. Conversely, if you had a nutritious breakfast or take a version of the test with lots of questions similar to the ones you practiced on you'll do better. Some people get "lucky" (for lack of a better word), while some get "unlucky".

The thing is, unlucky people are far more likely to retake the test and pay for test prep. So people enrolled in you test prep class are NOT a random sample of high school students. The sample is biased to include more unlucky people and fewer lucky people.

People paying for your service don't know whether their child was unlucky or not.

I used the work "lucky" for a reason. It is less likely that those negative factors would affect the student both times they take the test than just one time. So someone who encountered a confluence of unlucky factor for the first test might score higher on the second test simply by having fewer unlucky factors--or even just being more familiar with the test, having taking it previously.

> If this were true no one would take the courses, the courses would be rated poorly, and the underlying business would fail.

Your course doesn't need to actually help anyone. Your customers just need to believe it helped them. How do they know whether you actually helped, or whether the improved test scores are down to luck?

Does your average customer have 20 grandchildren who they can randomly split into test and control groups? Or do they have an imperfect understanding of statistics?

Maybe there was another variable at play: language. SATs are in English.

If English is not a student's primary language and fluency improves as they advance academically, up to a plateau with age.

Not sure if this was properly controlled in SAT studies. https://www.ncbi.nlm.nih.gov/books/NBK11007/

Glad you mentioned this. I went to a strong wealthy public high school with a high asian population (~50%?). When I was there, a lot of my friends were 2nd generation immigrants and their parents still spoke their native language at home. Their kids (my friends) were perfectly fluent/native in English but didn't do as well in the "edge vocabulary" parts of the SAT and I always figured that was why (in comparison to me and my english-at-home parents).

> Candidly, no one would pay for a prep course for a 30 point gain.

My understanding of the research in this area is that the 30 point gain is if you compare people who take a prep course to people who put in an equivalent amount of time preparing on their own with the widely available (e.g. can check them out of the local public library in many cases) preparation materials.

The 200 point gain is if you compare people who did the prep to people who have not done any prep.

The set of people who can't make themselves put in the time without the structure of a course would benefit by a lot more than 30 points from the course, and would pay for it. And do.

I vaguely recall a question on the Duke University application about what assistance you received with your college application efforts. (I can’t remember if test prep was included specifically.) I’m sure that some people lie by omission, but maybe asking this question is better than not asking it.

This times 10. GPAs have been rendered close to useless, especially at identifying above average ability, due to grade inflation, and also extreme variability between schools. Same for valedictorian and other appellations.

yeah this is a really hard problem and I don't see a fix outside of standardized testing. Every school is individually incentivized to use every trick - grading out of 5.0, grading loosely, giving a bonus score for AP/IB courses "because of difficulty", etc and teachers are obviously very sympathetic to the future of their students and the impact that being a Grading Nazi could have. And parents are obviously incentivized to find the school that's going to make Little Billy look best (best educated is great but not sufficient, that's why we're discussing testing).

You need a uniform grading system, which means a uniform material and a uniform grading process, which is... standardized testing, or at least AP/IB courses.

i came from a rural high school that didn't have any ap/ib courses. i wonder how much that affected my college applications.

i still got to go to the college of my choice (fire up chips!) but i have to wonder -- if i was able to boost my gpa using ap/ib courses, would i have received more scholarship opportunities/better offers from other schools?

That assumes the AP classes would have boosted your GPA. If the harder class knocked you from an A to a B then it would have been a net negative, at least at my high school (AP counted as a 1.2 weighting, so an A in a regular class is 4.0 and a B in an AP class is 3.0 * 1.2 = 3.6).

Admissions officers will tell you they are aware of what programs schools have, and take that into account. If your school has no AP with GPA inflation your 3.x is the same as a 4.x at some bigger high school. How true that is idk.

that may be true for colleges looking at local feeder schools ("northwestern knows that my high school doesn't have grade inflation"), but I don't know how that idea scales nationwide or internationally. To steal an example, how does a college in Seattle know that a high school in Illinois has grade inflation or not? is that tracked anywhere centralized?

you could certainly look at past performance of students from that school but that turns into a "legacy system with more steps"...

Anecdotal, but it seems universities have solved this by figuring out which schools have grade inflation. I went to a gifted school in Chicago that was quite competitive and did not have grade inflation. 20% of the school went to Northwestern every year because they'd accept every B student.

That method has its own problems, though.

For one thing, if a school has grade inflation so bad that even an A+ from that school isn't enough to get into Yale - is that a problem?

For another example, if adjustment for grade inflation means Yale will ask for an A+ from Martin Luther King High, Detroit while they'll accept a B from Phillips Academy, Andover - is that a problem?

Well the thing is I went to an inner city public high school. It was much closer to "Martin Luther King High, Detroit" than it was to a prep school. Majority of students were below the poverty line, yet almost half were accepted to Northwestern every year, many with full rides.

I've met people from elite private day schools. Their education in a different world than 99.99% of public high schools, except maybe a few like Stuyvesant in NYC and Lowell in SF, or the fortunate few where 3/4 of the kids have parents who are doctors or college professors (why not both?).

Exactly. What teacher is going to fight for the B when the parents are complaining to the administration she’s keeping their precious angel from Harvard?

The SAT, ACT, and even IQ tests, were originally created in part to help identity promising students who weren't from upscale backgrounds.

I'm not 100% sure that the tests can't be coached, but certainly not like the "leadership", etc.

And even if they can't be raised by coaching, the scores can certainly be lowered by poor education and a chaotic living situation.

EDIT: Most people who can pay for coaching are already sending their kids to the kind of high schools that serve to get them ready, so they are close to their peak already.

Even things like summer public service, there are consultants who can tell you, based on your target school, the best one for that school, like is it better to work on a clinic project in Honduras, or teach basic literacy in Burkina Faso.

Never mind that the plane fare to get your youth group to Burkina Faso would pay the school fees for an entire village, with enough left over to pay 1/2 the teacher's salary for a year.

Being a poor student myself, the EC is way more expensive and challenging for poor families(can not afford those at all), comparatively, SAT/ACT is actually much easier, a few books and keep bugging teachers can carry a long way at extremely low-cost. Comparing to EC's cost(and time), SAT/ACT mentor(online or offline) is still fairly affordable.

Living in internet era, I am jealous that nowadays 'poor' students can find so much resources online, most for free, even MIT courses! All you need is an ordinary computer and maybe internet access, which are quite affordable for nearly all families in US.

I will vote up for SAT/ACT and vote down on those EC from a socioeconomic perspective if I have to pick one.

I took one of those expensive SAT prep courses and yes, I agree that those don't increase scores very much, the program I took was awful.

However, I totally disagree that rich people can't game the SAT. I used to be a moderator at /r/SAT and /r/ACT on reddit. All of the questions and answers for all of the exams, including subject tests are known and published online. Both SAT and ACT routinely reuse exams from prior years, and anyone who puts in enough time to study the old exams can do well on the exams. And rich people have the luxury of more time to study, because they don't have to work second jobs or cook for their families or clean the house after school and have more services that can save them time.

> And rich people have the luxury of more time to study, because they don't have to work second jobs or cook for their families or clean the house after school and have more services that can save them time.

Unlike tricks for getting into university like studying after school for classes, hiring private tutors, or taking extracurriculars like lacrosse or rowing, which are great levelers equally accessible to the rich and the poor.

> And rich people have the luxury of more time to study

Those rich students, they cheat by studying harder!

Btw, I don't actually think being rich correlates with better academic achievements. It's better to be in the middle, not rich, not poor. To keep motivated.

Nah, there is a straight linear correlation between parental income and sat score.

https://static01.nyt.com/images/2009/08/27/business/economy/...

Did you bother to read the letter? This is directly addressed:

"This may seem like a counterintuitive claim to some, given the widespread understanding that performance on the SAT/ACT is correlated with socioeconomic status. Research indeed shows some correlation, but unfortunately, research also shows correlations hold for just about every other factor admissions officers can consider, including essays, grades, access to advanced coursework (as well as opportunities to actually take notionally available coursework), and letters of recommendations, among others. Meanwhile, research has shown widespread testing can identify subaltern students who would be missed by these other measures."

I'm very aware and don't disagree with you, I'm responding to the comment above this one that said there's no correlation at all.

What's great is /r/SAT and /r/ACT are available to basically everyone, even with a very slow internet connection. Extracurriculars, not so much.

Why do you say rich kids have more time? I grew up in an underprivileged area and I very much disagree that poorer kids are getting their free time hammered.

Thanks for the interesting link. However, based on the paper, it seems like a "30 point increase out of 2400 points" could be significant. The study says:

"A survey of NACAC-member colleges unexpectedly revealed that in a substantial minority of cases, colleges report either that they use a cut-off test score in the admission process or that a small increase in test score could have a significant impact on an applicant’s chances of being admitted." (p. 2)

The paper later notes:

"These results indicate that in some cases more than one third of postsecondary institutions agreed that a score increase on the SAT-M of 20 points, or a score increase on the SAT-CR of 10 points, would 'significantly improve student’s likelihood of admission.' This proportion tends to rise as the base level of the SAT score before the 20 or 10 point score improvement rises. This is especially true for the more selective institutions. At lower scores on the SAT scale, a small score increase does the most to improve a student’s chances of admission at less selective institutions; at higher scores, the same increase appears to have an equally large or even larger impact at more selective institutions." (p. 19)

The graphs on pages 18 and 19 give more detail.

The paper also notes that "The College Board gives a specific example of a use that should be avoided: 'Making decisions about otherwise qualified students based only on small differences in test scores'." So it appears that up to a third of the institutions surveyed are not following this guideline. It would be interesting to know who they are.

I agree with your comment about rich kids gaming GPA, essays, and extracurriculars. Daniel Markovits addresses many of these points in "The Meritocracy Trap". Since I don't see how you can prevent gaming GPA, essays, or extracurriculars, given the alternatives, you're probably right that the tests may be better in this respect.

I used to take the ACT for people, they'd pay me in beer (which I wasn't old enough to buy for myself). You'd get ID'd at the entrance, but nobody kept track of whether the name on your test was the name on your ID, so you'd just take each other's tests.

It might be harder now, I don't know.

But they also wouldn't be old enough to buy you beer.

Unless you are suggesting adults would buy you beer for you to take the ACT on behalf of a minor they knew. Which seems odd.

He’s old. You used to be able to buy beer at 18.

This country used to be a lot less uptight.

This was in 2004. I was 17, they were in their mid 20's.

I know that one was training to be a dental hygienist, or at least wanted to be training for that. The other few didn't share as much, but they all knew each other so maybe it was the same thing?

I highly doubt I've harmed anyone by enabling their hygienist to get where she was without knowing the formula for the volume of a cone. As far as I'm concerned the gumption necessary to hack your way in is worth just as much as the gumption needed to pass authentically.

And 18 year olds aren't taking the ACT. Or if they are, there are other reasons why they aren't getting into certain schools and they aren't as concerned with getting good ACT scores.

Not to mention, the drinking age being 18 was for a window of about 14 years in the 70s and 80s. (Unless he's from Louisiana)

Non-adults are capable of getting alcohol, without buying directly from a store.

Typically they know somebody who knows an adult that will to the transaction with the store and provide the id.

Otherwise, some stores accept good fake ids, or squint to believe that the person buying actually matches the picture on the card

In which case, he would be able to get beer in the same manner.

The issue is that the people he claims are getting him alcohol would be in the same situation as him.

No this was my older co-workers. My academics were fresh because I was still in high school, theirs was rusty because they had graduated several years prior without bothering to take the ACT.

My instant moral judgement on your having taken tests for others dissipated fully upon learning that they paid you in beer.

I will smile all day thinking about this.

Kids who aren't coached on the test strategy do worse. My high school did no SAT prep, and my parents weren't really aware of it. I was lucky in that I had an AP History teacher whose goal was for every kid to get a "5", that meant incorporating test strategy into the flow.

You needed to know stuff as the ante, but knowing the magic bullshit that would give you a good essay score was the key to get the top score. I increased my score ~120 points from taking the PSAT blind in 10th grade to the SAT because I understood at that point that strategy was key and found out about it.

All of this stuff is a red herring though. The nut of the controversy is that standardized tests correlate to IQ. IQ, rightly or wrongly, is perceived to be culturally biased.

Anecdotally I think it raised my scores by maybe 100 points. Not coaching so much as just doing practice tests to learn that, especially for the reading, the questions were actually pretty dumb. Lots of them are asking for the most basic insights. This is surprisingly non obvious or at least was not to me at the time. Many questions were filled with "traps" of answers that felt more insightful and more broadly relevant; but less relevant to the specific passages being questioned on.

I'm not so sure about that. About 20 years ago, counselors advised taking the SAT/ACT only once, since your score wouldn't really change.

I took the SAT several times. Each time my score went up significantly. My high school ended up creating a new award category for "greatest score increase", or something to that effect. I believe it was ~200 points.

I'd taken a prep course being offered by a local instructor. However, the biggest benefit was from dedicated self-study (Kaplan books, as I recall).

> A 30 point increase out of 2400 points is not material to college admissions.

Not so sure about that. Beyond the fact that that number is an average, the question is from where to where. So many kids get perfect 2400 scores that going from 2370 to 2400 might be the difference of getting eliminated from competitive admission pools altogether. Whereas nobody will care about you going from 1850 to 1880.

P.s. I am dating myself a bit with the 2400 range, which seems to have changed at some point. Transform accordingly :-)

Are there really that many kids getting perfect scores?

I got a pretty low/average score, but took the test early in junior year so I hadn't taken some of the more advanced math courses yet. I never took it again since I got into everywhere I applied to (didn't apply to ivy league, obviously). Seemed like most other kids I knew did similarly with the smartest kids maybe 150 points higher (2400 time-frame). Nobody I know got a perfect score, or even close to it.

Edit: man, after talking about this I want to see what my score was exactly. No way am I paying $30 for an archived score though. I want to say it was only 1200/1600 (the schools only wanted 2 of the sections). But I'm not sure I trust my memory for something so inconsequential from that long ago.

Extra edit: found my old score report. It's worse than I thought. The writing was 570 (74th percentile) and math was 510 (47th percentile). I'm a lot dumber than I remember.

I guess not exactly. But still looks pretty crowded at the top end: https://www.prepscholar.com/sat/s/colleges/Harvard-SAT-score...

Looks like it's 1% between 1550-1600. I couldn't find stats for an actual perfect score. Saying it's crowded I guess is ok, but is a matter of perspective. Like the top 1% of income earners saying their yacht club is crowded. Maybe true, but only for a very small number of people who could choose to go somewhere else if they actually wanted to.

http://go.collegewise.com/how-many-people-get-a-perfect-sat-...

Yeah no the study was definitely not about test prep getting people from 2370 to 2400 lmao. There is no test prep service in the world that will claim they can get you from a 2370 - 2400.

The mean SAT score is ~1600, so it's a 30 point increase for students scoring in that range.

If you're already capable of getting a 2350+, that means you know everything and it's just down to variance and not making a silly mistake.

A perfect 2400 score is actually really rare. From a stat in 2009, the collegeboard reported that 1 student out of every 5,000 taking the SAT gets a 2400.

I took it back when it was out of 1600 and missed a perfect by one question; however I don't think I was exceptionally brilliant or anything.

The SAT does not operate in the way the LSAT or some other computerized tests work where it keeps giving you harder and harder questions until you start getting them wrong.

I suspect more people could get a 2400 if those who get really close bother retaking it.

> So many kids get perfect 2400 scores

"So many" being ~500 out of a population of ~7 million or so

But test prep is marketed to people who are average or below average. Saying you can gain hundreds of points is clearly misleading/deceptive advertising.

I personally went up 300 points with tutoring, and that was when it was out of 1600.

n=1

I went up 240 points by taking the test an additional time and a grade later, without any tutoring in between.

Also n=1.

I have a family friend who does consulting for rich kids w/ college admissions.

You basically have someone who's on the board of an elite university coaching kids on their essay, clueing them into extra-curriculars, etc. They get paid pretty well.

I also had a dinner conversation from a lady with two children in Ivy league universities who said she emphasized with the parents who went to jail for bribing schools to get their childrens admitted and would do it herself if her children couldn't get in the school. She also personally knows one of the people doing jail time for bribery.

I had basically no adult academic/university guidance growing up. I just liked reading books in the library and studying things I liked. I was able to receive a scholarship to my university through my SAT scores. I'm not sure how I would square up in the current academic environment when I see the sheer amount of parent involvement in the application process. I also went to a smaller university where the level of tactics and skullduggery is limited.

> The NACAC did a study on this and they found that average gains of test-prep students is ~30 points (this was when the SAT was 2400 points).

> A 30 point increase out of 2400 points is not material to college admissions.

There must be some subset of students who gain much more from "test prep" than others? Even if its benefits for the average student are marginal, maybe there is a certain type of student for whom it is much more beneficial?

Not American so never did the SAT, but I honestly think I would have done much better in high school if I had one-on-one private tutoring. I struggled with focus and one-on-one attention helps keep me focused. Our son is similar – he's gifted and demonstrates his giftedness when the teacher focuses on him one-on-one, but then the teacher has to go spend time on the rest of the class, and as soon as that happens he stops doing any work.

>However, there is very limited evidence that SAT coaching actually increases your SAT score. The NACAC did a study on this and they found that average gains of test-prep students is ~30 points (this was when the SAT was 2400 points).

IIRC SAT's equivalent of Matura in Poland, so I'll be talking about my case

I've been taking advanced math exam and I had some time + some money (like 10% of minimal wage) during winter break and I decided to buy 3 lessons on analytical geometry cuz I've been terrible at geometry, but since that was analytical, then I've seen a chance to get into that

I've attended those 3 lessons, did some exercises and guess what

on official exam there actually was a task from analytical geometry and I managed to do it and receive full points, which basically increased my score by 10 percentage points (that's a lot, I'd say)

Saying that 10% of minimal wage spent was equal to 10 perc. points is naive, but you get the point

What if I were attending those for whole year? 2? 3? hard to say.

The SAT doesn't testing anything as high level as that.

> The NACAC did a study on this and they found that average gains of test-prep students is ~30 points (this was when the SAT was 2400 points).

When I took the SAT it was only 1600 (pre 2400), and SAT prep did in fact help scores significantly.

Back than, the test was designed not for scholastic aptitude (as it's name suggests) but instead to guarantee a standard distribution of scores.

It's been a long time since I cared about the SAT's so I assume once the word got out that the test could be gamed, the people behind it updated it.

> It's far easier for rich students to game GPA, college essays and extracurriculars than it is for them to game the SAT.

Rich students don't need to game anything. They simply get admitted because their family name is on a building.

>Rich students don't need to game anything. They simply get admitted because their family name is on a building.

The mean parental income for Ivy League students is 170k, which is above middle class, but not Bezos-level rich.

What is considered to be rich is a huge spectrum. The difference between 7 figure rich vs. 9 figure rich..is up to a factor of 1000. Those whose parents can donate enough to be commemorated on a building, is an outlier even for the rich. Unless your parents are dynastically rich, being rich is not that much of an advantage for admissions.

> instead to guarantee a standard distribution of scores

Which is precisely what you want in an assessment test. Any assessment test.

You only want a normal distribution if the quality under assessment is normally distributed, but you do want a test where the worst candidate does better than chance, and exactly one candidate gets a perfect score. That's an ideal which is only approximated, but it is the ideal.

For context I graduated high school in the late nighties (I took the paper/scantron test).

At the time, the SAT was purported to provide a score that predict ability to perform academically at higher learning institutions.

Along with other factors such as GPA, and participation in extra curricular activities, a school could reasonably determine how well a student would do.

In practice, the normal distribution for scores correlated with the distribution of college performance. It was a reasonable predictor of success, but it did penalize students from certain backgrounds.

Because the test was devised by psychologists and statisticians, uncovering the pattern to the types of questions and the expected answered allowed test prep people to devise tricks to improve scores beyond the expected deviation.

Your first post claims it isn't a test of scholastic aptitude, and then this one says that it does predict scholastic success, and what could reliably predict scholastic success other than a test of scholastic aptitude?

Sure, a big donation by the student's dad, but that's a known quantity. I took the same SAT you did if it matters.

Which certain backgrounds are you referring to? I'd ask you for the references to show the supposed boost that test prep gives to SAT scores, but then I'd have to find the papers that fail to reproduce it...

This was from 2020: https://www.brookings.edu/blog/up-front/2020/12/01/sat-math-...

Finding data from 1998 online is much harder.

I wonder how much taking the SAT multiple times plays a role. Personally I took it at least three times, and my score improved each time (don't remember by how much).

Which is also tied to socioeconomic status. If you can pay, your score will go up the more you take it, according to my experience, and the college board.

If you can't pay, you take it the one time it's offered for free if your school offers that. Then you get what you get.

I think we should learn from the gaokao and only test once a year. The fact that you can pay for more tests (and pay more for "score choice") is the most unjust part of the whole thing.

Counterpoint, you don’t want being sick or having a bad day to ruin your chances at what you’ve been working for for years. Also, being under pressure is generally not good for people’s ability to reason calmly - another reason not to make it so high stakes. So I really hope what you’re suggesting doesn’t come to pass.

We should probably make the SAT nearly free, though, if that price actually keeps people who’d otherwise go to college from taking it more than once.

> being under pressure is generally not good for people’s ability to reason calmly - another reason not to make it so high stakes

Q: Could be that being able to "reason calmly under pressure" is something that a future employer might well be interested in?

It could be, depending on the field you choose, but that’s not particularly relevant to what college you’ll do well at.

A: Should your entire future job prospects be dictated by something that an abstract employer in the future might want? Or should the test we use try to get at what is really important; actual knowledge and skill?

Anecdotally I improved 600 to 400 points from test prep, depending on what you count as my "first SAT test" and my last SAT test.

It's been decades, but back when I was in high school rich kids paid people to fake their identity to take the tests (with fake IDs). I heard rumors that it was a dozen kids in my graduating class, and witnessed 2 myself. 1 was caught.

Does anyone know if they have better checks for this now?

I am all for reinstating the SATs, but it’s a stretch to say that it’s not impacted by socioeconomics. While test prep may have limited value, a lifetime of wealth provides more educational opportunities.

As a counterargument, a fake ID is all that a rich person needs to put a smart kid in their place at the testing center. Those places are usually huge, nobody knows anyone, and if you flash a legit looking ID, you will have no trouble sitting the test. GPA and such require effort over years to game (and maybe the kid actually learns something from all that tutoring, who knows).

How common is this? I would guess not very.

Either way, if you cheated your way into MIT, expect to fail out. The hand holding stops there.

MIT is one of the more rigorous schools but I expect its still pretty easy to avoid failing out.

Do you speak from experience? I attended it for grad school and not undergrad, but there was the expectation that you stand on your own. I was in one of the rare programs that was a terminal masters and you had to reapply for the PhD, and only 50% were accepted to the PhD.

I think there probably aren't too many failing out of undergrad simply because they do a great job of filtering in the first place. But I know people who attended that certainly struggled... one who came a C student and still want on to a great medical school and a successful career. There isn't grade inflation.

> There isn't grade inflation.

That depends on your definition of grade inflation. I think most of my undergrad classes at MIT had a median grade somewhere in the B range, maybe B-. Edit: I know some people consider a non-inflated grade curve to be C-centered.

I came to MIT with more than a year's worth of credits from the U of MN, including 6 trimesters of honors level math[0] (multivariable calc, linear algebra, diff. eq.). I had all As, except a B in my Intro to World Politics class. My senior year of HS, I was actually taking a bit over a "full course load" at the U of MN, plus 1/4 time at my HS.

I could sleepwalk through nearly straight A's at a pretty well regarded school's honors program. I was a B/C student at MIT. I like to think that a lot of it was that some "wise" uperclassmen had sat me down my freshman year and explained that once you had a degree from MIT, nobody would ask for your GPA. (The were wrong, BTW. Work for those grades.) I taught myself most of a CS degree while earning a degree in Mechanical Engineering. However, I was also too slow to put my ego in check and admit to myself that I really needed to work hard.

[0] https://cse.umn.edu/mathcep/about-umtymp

Sounds like you’re effectively saying there wasn’t grade inflation. You were an A student elsewhere and then become a B/C student at MIT despite working hard. That’s my point — now imagine you were only an A student because you were rich and somehow swindled those good grades in high school. Imagine what would happen at MIT.

Note that Harvard undergrad has something like an A- average. That’s grade inflation.

I certainly agree with your broader point: nobody is handed a degree from MIT. If they have a degree, they've put in the work and have a good grasp of the subject matter. (Also, MIT doesn't give out honorary degrees.)

I also attended it for grad school only. Nobody was even close to failing out. I don't think MIT did a lot of grade inflation and generally regard it as a very strong academic institution.

That's not incompatible with weaker students being able to pass by with lower grades. Getting bare minimum grades just isn't that difficult.

edit: teammwork also seemed to be strongly encouraged. It wasn't a place that wanted you to fail. It wanted to help you succeed, which is by all means a good thing.

But saying you got into MIT, plus the connections there, probably would still help

If you fail out, it doesn’t look so good, and everyone will know. I wouldn’t expect many connections to last if you’re not someone people respect.

Well, Trump did it, that's one data point. Also, he did fine at U Penn, an Ivy, so why should we assume MIT is so awesome that another rich kid couldn't cheat his way through there?

Technically MIT isn't an ivy league school. But either way, what makes you think all Ivy's are the same?

I'm just saying that U Penn, being an Ivy, has as much "reputation" as MIT, CalTech, etc. Until someone pipes up with some sort of proof that it is actually better (which I doubt it is), then why should U Penn's reputation allow a Trump to go through, but at MIT such a thing could never occur? I'm not seeing it frankly.

MIT is absolutely harder and more rigorous than UPenn (and, for that matter, HYP). If you compare the GPAs of people from those schools and e.g. MCAT scores, MIT students exhibit a much stronger positive correlation.

This is pretty common knowledge, in the same way people know that Berkeley and CalTech have tougher classes than Stanford.

As an MIT grad with many friends at other Ivies, I can tell you that it is more rigorous than pretty much all of them. Princeton is probably the closest.

The man is functionally illiterate so it's likely someone else did it in his name.

Sure. So, 2 things.

1) The solution to that is to improve security measures. Not to remove the SAT entirely.

2) It is significantly harder to find someone who can score well on the SAT and is willing to take the test for you compared to gaming GPA, extra curriculars, etc.

I've heard of tons of instances of people hiring homework-help services, their dad paying $5k to get their extra-curricular club going, private tennis lessons, etc.

But I've never personally heard of someone paying someone else to take the SAT for them.

I'm sure it occasionally happens, but it's a lot harder to pull off compared to manipulating GPA, extracurriculars, college essay.

> It is significantly harder to find someone who can score well on the SAT and is willing to take the test for you compared to gaming GPA, extra curriculars, etc.

Idk about that. I found people to pay me for taking the ACT for them through my alma mater's subreddit. Top schools are full of people who got 35/36, I'm sure there are plenty of other people who scored there and would be willing to take the standardized test for 10k too. In fact, in some ways it's easier to find someone to take the test for you than it is to find someone to boost your extracurriculars because you can structure the payout around the score obtained. I got 10k for a 36, 7k for a 35... no guarantees with tutors and coaches.

If the parents/students are willing to cheat on the SAT, they're probably going to have no issue manufacturing extracurriculars. The schools for the most part aren't auditing run of the mill activities (student org leadership, fundraisers, mission trips, local awards), and a lot of local newspapers basically let you write your own articles for them so cheaters can build up documentation if they're really motivated.

Some years ago in high school others offered me significant amounts of money to take the SAT for them. I didn't do it, but based on the security arrangements at the time I'm pretty sure I could have done it without getting caught. So I have to assume there has been some of that cheating going on.

> I have to assume there has been some of that cheating going on.

It's an industry.

https://www.reuters.com/investigates/special-report/college-...

> So much for that common, popular notion that standardized tests do not predict anything of value.

To be fair, I don't think the debate was ever about the quality or predictive value of the tests. There is a small, but well-organized and vocal subset of the population that hates the idea of excellence and differentiation. They want, and have been quite successful in, the replacement of standards of excellence with vaguely defined (defined by them, of course) buzzwords like "equity" and "diversity".

I've pushed back against standardized testing at certain points of my life, and I don't think this comment even remotely summarizes my views.

If anything, I would say that my views are the opposite -- homogenization creates a lack of differentiation around skill and aptitude based on questionable science (and sometimes outright pseudoscience) and often leads to an oversimplification of human intelligence in general. It always feels very strange to me that people trying to compress aptitude into a single number say that they're defending differentiation or diversity of talent.

MIT's findings here don't really change my view of the value of SATs, although the findings are interesting and I think they're worth looking into further. I'm not sure "they're more predictive than GPAs" is the glowing recommendation that SAT proponents think it is. You can agree or disagree with me on that point, I'm not here to debate the entire idea of testing or IQ or whatever -- I just want to point out the above comment is a pretty big oversimplification and (in my mind) a borderline complete misrepresentation (I assume unintentionally) of what people like me believe. I can only speak for myself though, maybe there are people out there who do hate the idea of excellence.

Well, what you have written just feels like a more favorable to your side explanation of the same thing.

Colleges are not trying to compress aptitude into a single number. It’s even worse. They are trying to compress aptitude into a single Boolean variable, you are either admitted or not. That’s it. And it seems that subject tests and general aptitude tests are very good indicators of college fit. I don’t know what system you envision, but alternatives I have seen always seem far worse.

I'm not sure I understand what you mean. GP writes:

> There is a small, but well-organized and vocal subset of the population that hates the idea of excellence and differentiation.

I don't see how that applies to my comment above, and I don't see how saying:

> They are trying to compress aptitude into a single Boolean variable, you are either admitted or not. That’s it.

is doing anything other than backing up what I said. At the point where you are dividing a subset of the population into binary "in or out" groups, you are in fact advocating for homogenization, for less differentiation between students, and for fewer levels/categories of excellence or exceptionalism.

I'm not here to tell you that's wrong, you do whatever you want. MIT is trying to decide who gets into their specific college, fine. But if you're arguing that the point of SATs is to make a binary determination about students, then it's just strictly inaccurate to say that it's the SAT critics who are all trying to cut down tall poppies.

You conflate vertical differentiation with horizontal differentiation. Horizontal differentiation is what is usually understood as “diversity” and considered good among certain groups of people. Vertical differentiation is what is usually understood as “hierarchy” and considered bad among those groups of people.

MIT like many American universities does only general admission and that’s indeed would be considered weird in other countries, but it seems like a whole nother issue.

> You conflate vertical differentiation with horizontal differentiation.

A binary admissions model reduces both. That's not to say a binary admissions model is wrong, but it does reduce vertical differentiation. Of course compressing an integer value into a binary result reduces differentiation, a boolean represents fewer states than a number.

To go a step further, even if that wasn't the case, vertical and horizontal differentiation still can't ever be completely decoupled from each other. Horizontal differentiation allows for greater vertical differentiation by allowing people to vertically differentiate based on their strengths rather than on a questionably representative average of all of their qualities. And I don't think that's a solely Progressive or Left-wing idea, it's a big part of the reasoning behind why economic specialization leads to more advanced societies.

So what kind of system do you envision? I am a bit confused what you are arguing for.

See https://news.ycombinator.com/item?id=30834517, but I'm here clarifying what my criticisms of the SAT are and what I think its weaknesses are -- and pointing out that my criticism of the SAT is the exact opposite of what tharne says it is. I'm not here to rework the entire admissions process.

I don't have a single-comment answer to replacing the entire SAT and reworking the entire college admissions process, and it's feasible that the SAT might still be preferable to pure GPAs in the meantime. But I don't think saying that requires us to pretend that compressing skillsets into an objectively less granular/descriptive metric is a good thing or that it's somehow increasing our understanding of student skillsets. Saying that MIT might be right to accept SAT scores doesn't mean we need to pretend that the SAT doesn't have very serious flaws. Certainly it doesn't require me to pretend that every argument against SATs are arguments against meritocracy, I think that's just objectively wrong.

Ideally we would have standardized metrics that were more granular, and ideally we would at least have an SAT that was administered differently and more regularly so that they were optimized less for formal test taking skills. But there are a lot of barriers in front of that.

----

I also don't have a single-comment answer for what to replace Github repos with during hiring interviews, or how to make whiteboard coding tests more accurate, and I have criticisms about them too. The answer might be that there isn't an easy single number that represents meritocracy, and we might be fooling ourselves pretending that there is, and it might just be wishful thinking in the first place to pretend that there is a version of admissions processes for colleges that isn't fiendishly difficult and complicated and multifaceted.

When people criticize whiteboard interviews on here, it's reasonable to ask if there's a better system, but I rarely see people saying, "you're only criticizing whiteboard interviews because you hate meritocratic job placements." No, I have criticisms of these systems because they're not good representations of talent.

> If anything, I would say that my views are the opposite -- homogenization creates a lack of differentiation around skill and aptitude based on questionable science

If that were true, you'd expect countries like South Korea, Japan, and German to perform poorly in science and engineering, among other things.

Diversity may be a worthy goal for societal reasons, but it certainly is not a perquisite for excellence, seeing as there are many highly successful countries that are very homogeneous.

> If that were true, you'd expect countries like South Korea, Japan, and German to perform poorly in science and engineering, among other things.

It's wild to me that someone can have the view that the existence of other countries settles the debate over whether or not our school systems encourage well-rounded/successful students given that comparisons to more homogenized schooling environments like China is still one of the more contentious high-level debates about educational quality we have today. Again, I'm not here to convince you one way or another, but that is not a debate that I think most of society considers settled.

> Diversity may be a worthy goal for societal reasons, but it certainly is not a perquisite for excellence

If that's the argument you want to make, then fine, go for it. But then don't say that you're opposing a group that "hates the idea of excellence and differentiation." You are arguing for removing differentiation between different kinds of intelligence and skillsets and compressing that spectrum into an objectively less descriptive metric.

Make up your mind whether I'm arguing for more diversity and more differentiation between people or for less of it.

What's a better alternative in your view?

I'm not completely sure. I think MIT's conclusions might be correct, they might be preferable to GPAs. I also think there might be other alternatives that aren't easy to implement, that require either a restructuring of how we do school or a better distribution of resources than we currently have.

One conclusion that MIT hints at (although it doesn't say it outright) is that SATs might be a better indicator of success across economic levels in part because it's harder to buy a better SAT score with money. Looking at things like extracurricular activity runs into many of the same problems as looking at Github repos during hiring processes -- a lot of people don't have time to do a bunch of extracurricular activities, and access to those extracurricular activities is likely highly correlated with socioeconomic status. It might be difficult to move in that direction when access to school resources varies so much between areas.

I do think the SAT could be improved -- I think one really easy way would be to change how it's administered so that it optimizes less for formal test-taking skill. The really good thing about the SAT is that it's a less school-specific measure than GPA. So a better alternative might be a version of the SAT that kept a standardized metric but that either widened its scope significantly or was administered differently.

I also want to put forward the idea that admissions might just be really hard, period, and there might not be an easy way to assess potential, and trying to figure out the easiest way to do it might be like asking, "what's the best way to teach a child to play an instrument in a single day?"

----

One really important point that I want to get across: there is a difference between a measure being good and a measure being "the least terrible option we have at the moment" -- and confusing the two can cause real harm.

At the top of this thread I see the quote, "so much for that common, popular notion that standardized tests do not predict anything of value." And if that's somebody's attitude, then they're never going to find a better option because the whole thing is being approached through the lens of "see, we were right, this is a good metric."

I think a lot of criticism of standardized testing, IQ, coding tests for hiring, etc... is not necessarily trying to destroy everything, it's just trying to point out that many of these measures are really bad and they shouldn't be treated with the respect they're often given. I think that someone can very easily both have the position, "yeah, MIT probably should use SAT scores alongside GPAs" and the position, "people place way too much confidence in these things as an indicator of success."

Unsure if I buy this.

I'm definitely not the person you describe, but the idea of standardised testing being equivalent across all factors just strikes me as being fundamentally untrue.

Personally I am very lucky to test well; and I definitely buy the notion that people who test well in SATs may go on to do better in University, but the reasons are probably the same: freedom from worry about financial circumstances will affect grades. 10 times in every 10.

I grew up poor and I achieved some of the highest scores state wide in my country's standardised tests as a child (we get tested at ~8,10,12,14). A lot of my peers at my school were from social housing. My assessment is that their biggest issue wasn't money but their homelife. Parents who didn't value education, or even a basic respect for rules/authority. The kids were wild because their parents were kind of wild themselves. Money wouldn't fix scores for these kids.

If you wish to make a political correct stance, I wouldn't go the money route. I'd say that these kids are victims of intergenerational poverty cycles.

Same. My family was below the US poverty line, but my parents were college educated and most of the extended family placed tremendous emphasis on education, academic performance, and college prep. I always get very annoyed with modern discourse that reduces all successes, even staying out of prison, to family income and nothing else. Most of the people I went to school with were from poor or working class families, and I guess a “normal” proportion went to college, and a “normal” proportion were “smart kids.” Based on my observations, a large factor that I never see discussed is religion. Although I’m an atheist, I think the religiosity of the communities I grew up in was a highly effective mitigator of common social ills.

I think the benefit of religion is that a religious mother/father is less likely to be off on 3-day meth binge compared to a non-religious one. There's a social network to help support people. The social network also encourages a reduction/removal of typical vices that are going to affect a families children (alcohol, drugs, etc).

I agree, and it's not the job of MIT to fix these kids.

>but the idea of standardised testing being equivalent across all factors just strikes me as being fundamentally untrue.

What's your take on MIT's stance?

our ability to accurately predict student academic success at MIT 02 Our research shows this predictive validity holds even when you control for socioeconomic factors that correlate with testing.

Not GP, but you should approach this using Bayes' Theorem just like anything else. If one study from MIT causes you to completely flip on any of your beliefs, you need to rethink how you form these kinds of opinions.

MIT's conclusions should cause you to adjust your priors by a certain amount, but they should not cause you to completely flip by themselves -- particularly if you're not in the camp that thinks literally every decision MIT makes is correct by virtue of it being MIT.

If you wouldn't have looked at MIT's original plan of abandoning SAT scores as proof that they didn't matter, you probably also shouldn't look at them picking up SAT scores again as proof that they do matter. MIT's conclusions should lead you to update your priors by some amount dependent on how much you trust you currently have in the accuracy of college admissions processes when they assess student qualifications and outcomes.

----

My personal take on this is that I do absolutely buy that SAT scores could be a leveling factor between kids from different socioeconomic backgrounds and that they could be a better metric than GPA for determining admission. But of course, that's a pretty low barrier of entry to clear, GPA scores are probably close to meaningless when compared across schools. It seems to me that there's a lot of room here for SAT scores to be simultaneously mostly meaningless and at the same time also a reliably better predictor of school success than GPAs.

It's also important to ask what exactly MIT is measuring -- what does it mean by academic success and how much does that definition overlap with "fits in when placed in an environment optimized for people who are good at standardized testing?" And again, even if they are kind of circular or if they're measuring the wrong things, it's still plausible that they're more reliable than GPAs; it's a low bar to clear.

My take is exactly what I said.

The same factors that lead to success for SATs can lead to further academic success.

I believe that MIT is probably right, in fact, I'm quite certain of it. Many people will drop out of university or perform poorly than their peers for socio-economic reasons, the person working while studying will probably do worse than the person who just studies.

MIT wants the most graduates and especially the most successful graduates, so the institution is right to do this, but I do still think it's more inhumane than I'm personally comfortable with -- but this is part of why I live in Europe where university students in general are seen as an investment by the state and not so much a business to be optimised.

> this is part of why I live in Europe where university students in general are seen as an investment by the state and not so much a business to be optimised

In this specific case, though, I don't think these two things are in conflict at all. By selecting the best candidates on the basis of merit, MIT is doing what's best for both MIT as well as the broader society.

We all benefit from living in a country that produces top-tier scientists and engineers, and MIT benefits from being a place that is known for producing top-tier scientists and engineers.

Funny that you bring up Europe. As far as I know European countries don’t rely on extracurriculars and other nebulous measures as much as US colleges do.

What's "inhumane" about trying to select those who will benefit most from your program?

Doesn't most of Europe also rely on standardized testing for university admissions? My country definitely does so, and has for decades, both ore and post communist times. I also know France has the famous Bacalaureat at the end of high school.

> Doesn't most of Europe also rely on standardized testing for university admissions?

They sure do. So does India. In fact, a lot of other countries rely on testing a whole lot more than the U.S. which has interviews, essays, sports, teacher recommendations, etc.

We have to read the sentence very carefully. It's saying that regardless of socioeconomic factors, the number correlates with graduate success rate. This seems like a very easy "duh". The way I read that is "if a student gets in the 99th percentile regardless of whether they grow up rich or poor, they are likely to do well at MIT". This doesn't talk about acceptance rates based on socioeconomic factors.

The point in question is whether the students in a lower socioeconomic situation even has a chance to get into MIT.

It's not saying regardless, it's says controlled for. A subtle distinction, but the former is a raw comparator and latter is an adjustment, which implies even with a bias that still doesn't account for substantive change.

I was hoping someone had more insight into process or metrics on it.

Your own post has an implicit accusation that lower socioeconomic situations preclude high testing, which has all kinds of implications about the quality of education and living standards as a prior for acumen. I suppose my own bias shows in drawing that conclusion though. As Henry said though,

"If I had seven peasants, I could make seven lords. But if I had seven lords, I could not make ONE Holbein."

>To be fair, I don't think the debate was ever about the quality or predictive value of the tests.

It is. The common argument is that GPAs are as predictive as SATs. MIT says it is not. I think the problem is you only need average ability to a good GPA, but a top 1-5% SAT score confers a higher ceiling of ability. MIT wants to admit exceptional students, not just average or above average ones.

The debate is about the quality and predictive value of the tests. Opponents claimed that the tests had a cultural bias so students from some backgrounds would do better than others, that students who had a good education before university would be better prepared, and that studying for tests or taking tests repeatedly has been shown to improve scores but is only accessible to people who can afford it. These are all claims that the tests are not good at predicting aptitude.

The arguments against these tests are, of course, awful. Objective tests are the best way we know of to remove human bias. Aptitude tests (basically IQ tests) are the best way we know of to measure someone's natural ability (determined in early childhood) with little influence from their experience. Since their arguments make so little sense, it is reasonable to wonder about the psychology of opponents of standardized testing. But their arguments are, at least on the surface, about predictive value.

> it is reasonable to wonder about the psychology of opponents of standardized testing

It is, at its core, a fear that testing largely reproduces the status quo. If one accepts the idea that there is an intellectual elite who constitute the highest strata of society, and that their gifts are innate and heritable rather than trained, it follows that social mobility is pretty much dead. It is a bleak vision.

Personally I think there are different problems that are much bigger and woollier which keep people from non-elite backgrounds down, regardless of test outcomes. The structure of the education sector and employment more widely. Expectations about life and the distribution of rewards etc. We rarely have good quality, nonpartisan discussions about these things which I think pushes people to take views which are instrumental rather than informed.

>it follows that social mobility is pretty much dead. It is a bleak vision.

I have always found the idea of social mobility depressing. It assumes that we will always have a hierarchy, with some people who are powerful and prestigious and others who are poor and always feel inadequate. It assumes that we will always have an underclass but at least people can leave it.

The kind of social mobility that SAT has some influence on is not really about "power and prestige", which I also think of as generally pathological dynamics. It's literally about how competent and professional you want to be, and how well you can perform your work duties. It's social mobility within the 'working' class, not really away from it.

Yes. The old saying among Labour party socialists in the UK was "rise with your class, not above it". They were in favour of a high floor on living standards and a low ceiling on wealth. It isn't a stretch to think that a more even playing field would be a better substitute for mobility.

The MIT article literally says "diversity" is one of their goals. Seems like you're arguing against yourself.

> The MIT article literally says "diversity" is one of their goals. Seems like you're arguing against yourself.

Except no one goes to MIT because it's "diverse", whatever that even means anymore. They go there because it is one of the best schools in the world.

It is actually possible that some people go to MIT because it has more diversity [1] than its very close competitor, Caltech. [2] But it's true that the first filter for these students is undoubtedly world-leading-technical-program.

1: https://www.collegefactual.com/colleges/massachusetts-instit...

2: https://www.collegefactual.com/colleges/california-institute...

We're talking about the school's goals in forming a class, not the applicants' goals. Most schools that are among the "best in the world" find they can weigh multiple factors to decide who to admit, and there's no single magic number that does that job for them.

Is the current push back against coding tests in job hiring perhaps similar to this push back against the SAT?

> Is the current push back against coding tests in job hiring

Is there such a pushback? As in, is the percentage of the workforce refusing to take such tests increasing?

It very much depends on the style of coding test - personally, I'm more than happy to do take-home style tests where I prepare something in a matter of a few hours, but I can't stand "leetcode" interviews or anything where I'm pressured to produce in 30 minutes or less; perhaps that's because that's typically not how I work in the real world and in my experience, they do a really poor job of demonstrating my skill set and experience.

I have terminated interviews before they even got started because of poor interview loop design from employers.

I recently failed a CS coding test. I was asked to solve a problem in 10m. I solved it in 20m and was rejected. I came up with a solution and communicate it right from the start. My solution was totally clear and readable. I just needed time to warm up and attentive to my code. I love CS. I love solving problems and reading books about Algorithm and Data Structure. I implemented them from scratch as a hobby. But the interviewer guy is not caring about that and said process is process. I felt disappointed at first but felt lucky after that since I wouldn't want to work with those people in the future.

10 mins per problem sounds extreme except for something that can be answered in no more than 5 lines of python (no code golf of course). Even then its signal-to-noise ratio (from an interviewer's perspective) can't possibly be too high. Most places would ask you to solve a moderately nontrivial problem in 30-50 minutes

From the other part of the table - we'd lose more candidates if we did take-homes. People in general prefer to study once and use that knowledge for multiple companies at once, you can't optimize take-homes like that.

I don't know if there's a _rising_ pushback but you definitely do hear a not-small amount of complaining about coding interviews on HN.

Sure, but all in all, you just need 100 frequent HN users hating on these interviews to fill all related threads with such complaints, and there are dozens of millions of SWE worldwide. Big tech companies still hire mostly based on this type of interviews and they've been growing a lot lately, meaning that enough people apply to them. Leetcode and the likes have also democratized the process for many candidates.

If anything, I'd say that the proportion of the workforce willing to submit themselves to these tests is increasing, but that's just a guess.

I refuse now and didn't two years ago. Anecdotal, I realise. But I think lots of people have decided that the message heavily test oriented recruitment processes send out indicates a bad work culture and sense of entitlement from employers. I subscribe to this view and vote with my feet.

The SAT has been demonstrated to be effective at predicting success in university. We have almost no evidence about the computer industry's hiring practices. It is completely unscientific. Interviews operate on folklore, not statistics.

This is something your HR department should be very concerned about. If the questions you ask during your interview are not useful in finding a good candidate why are you asking. This isn't just about time either, interviews have some strong laws around them so asking the wrong question could get you in court.

I know when we wanted to do a coding test they told use we need to spend 6 months of giving everyone a coding test, have it independently graded by someone not involved in the hiring process. Then after people have worked here for 6 months we examine our actual results from those we hired and see if the tests at all predicted something useful. (or something like that - there is room in the scientific process for some variation)

The bar below which HR has to be worried is not "we've scientifically determined that our interview questions lead to good on-the-job performance". There has to be some reasonable sense in which you could argue the interview filters for good candidates, but no one is requiring you run studies.

Google once did a retrospective study and found that interview scores for people we ended up hiring were not correlated at all with people's on-the-job performance. I'm pretty sure nothing really changed as a result of this. I think it's a combination of the industry, especially FAANG, being kind of "stuck" on these kinds of interviews, and a lack of clearly better alternatives (I think there are better alternatives but it's not like I can point to studies backing me up).

> I know when we wanted to do a coding test they told use we need to spend 6 months of giving everyone a coding test, have it independently graded by someone not involved in the hiring process. Then after people have worked here for 6 months we examine our actual results from those we hired and see if the tests at all predicted something useful.

This is interesting but also way heavier weight than anything I've ever heard of. OOC where do you work? (Like vague description of kind of company, if you're not comfortable sharing the specific name).

> Google once did a retrospective study and found that interview scores for people we ended up hiring were not correlated at all with people's on-the-job performance.

This sounds like an unsound result. If you select based on a criteria the correlation with the criteria is usually diminished and sometimes even reversed in the selected sub-population.

Like if you select only very strong people to move furniture then measure their performance. Because they're all strong, you won't observe that weak people are bad at it-- plus you'll still have some people who were otherwise inferior candidates who were only selected because they were very strong, resulting in a reverse result. But if you dropped the strength test you'd get many unsuitable hires (and suddenly find strength was strongly correlated to performance in the people you hired).

This is actually confirmed with real world data on this for professional football with player weight and professional basketball with player height.

For Offensive Linemen in the NFL, there is no correlation between weight (which range from 300-360 pounds) and overall performance. A "heavy" 350 pound player is not more likely to do better than a "light" 310 player. But nobody who weighs a mere 250 pounds could realistically make the cut or perform well at the highest level.

For basketball players there is no correlation between height and performance, and there are several standouts examples of players below six feet so there's no cutoff. But if you compare the distribution of the subpopulation versus the general population, you'll see an extremely strong height bias.

> This sounds like an unsound result. If you select based on a criteria the correlation with the criteria is usually diminished and sometimes even reversed in the selected sub-population.

Yeah that's very true and I think was part of why they maybe didn't react to it too much. What you really want is to find the people you rejected and see how well they're doing, but we don't have that data.

Still though, naively I think I would have thought that someone who gets great marks across the board should be able to be more successful at Google than someone who barely squeezes by, and I do think it's kinda telling that that's not the case. But I'm maybe just injecting my own biases around the interview process.

edit: This reminds me a lot of this informal study that found that verbal and math scores on SATs were inversely correlated, which seemed surprising, until people realized they were only ever looking at samples all from a single school. Since people at any given school generally probably had ~similar SAT scores (if they were lower they wouldn't have gotten in, if they were higher they would have gone to a more selective school), the variation you see within a given school will be inverse (the higher you do on math, the lower you must have had to do on verbal to have gotten the "target" score for that school).

At google's scale, if they had an alternative basis for hiring people they could judge candidates by both and hire randomly use one method or the other method to make some of their hires, then compare their performance over time and at least say if there is a significant difference or not.

But as you note, the lack of obvious good alternatives is an issue... and we can't pretend that there isn't an enormous difference among candidates. If we though that unfiltered candidates were broadly similar then "hire at random, dismiss after N months based on performance" would be a great criteria, but I don't think anyone who has done much interviewing thinks that would be remotely viable.

(Though perhaps the differences between candidates are less than we might assume based on interviewing since interviewees should be worse than employment pool in general, since bad candidates interview more due to leaving jobs more often and taking longer to get hired)

>If we though that unfiltered candidates were broadly similar then "hire at random, dismiss after N months based on performance" would be a great criteria, but I don't think anyone who has done much interviewing thinks that would be remotely viable.

I know a fair number of companies that do essentially that. They hire contractors for 6 months, at the end of 6 months the good ones are offered a full time position. The contractor company probably does some form of interview, but they are more interested in their 6 months of overhead from the contractor than quality candidates.

> since bad candidates interview more due to leaving jobs more often and taking longer to get hired

But there are also great people who interview badly.

> OOC where do you work

Big tractor.

It's similar but it also brings in a challenging problem: coding tests costs candidates far more than it costs employers in terms of time. I am currently interviewing and two of the companies I am otherwise excited for sent take-home tests that just exhaust me, especially after a long day of otherwise productive work. I've got 12 years of experience under my belt but somehow great references and a killer resume aren't enough to convince them I can find a security vulnerability.

I do coding tests for the first interview. Nothing hard, just enough to do basic data modeling and writing a unit test. I also time cap to under an hour, and the internet is available as a resource.

This filters out most people.

I respectfully disagree. Any decent interviewer spends the same man-hour as the candidate does.

It's more about labor market dynamics and supply versus demand. If there are plenty of developers available to hire then employers will insert extra hurdles in the process to filter out weak candidates (with the understanding that there will be some "false negatives"). But when the labor market is tight then employers will take a chance on any candidate who seems minimally competent because they need to fill the req.

I think that is more about the disconnect between coding tests and the actual day to day work and skills required to do the job. Example: I could have a high level of competency in software engineering and also not care how a mouse gets out of a bucket.

Those "mouse getting out of a blender" brain-teasers or whatever are pretty unheard of at this point I think. Most people complain about coding questions, generally leetcode-style questions I think.

> So much for that common, popular notion that standardized tests do not predict anything of value.

To people with a particular agenda -- that society will be improved if equal outcomes are mandated -- I suppose standardized tests aren't "valuable" to them.

I am very grateful for the SAT. I wasn't a good student. I was unable to do any work outside of school because of a bad home (my brother did hours long "hand-clapping/stimming" and chanting rituals, my mother drank, and my father was violent). But I did well on the SAT -- enough to get a national merit scholarship and a scholarship from Hofstra University where I got my BA in Math in the early 80s.

If it wasn't for the SAT I don't know what I would have done. Standardized tests are the only answer, and I was very upset when I read of schools getting rid of using them for admission.

SATs are also a great predictor of a person's ability to complete college. That was one of their original uses. So without SATs, you'll get more people doing poorly, and more people with no degrees and a lot of debt. And if these people are members of what's considered an "under-represented minority" then there will be even more remedial action required elsewhere to fix the problem of the high failure rate (like giving them degrees anyway, etc).

I guess the issue in their analysis is that they are trying to predict MIT grade success, which is possibly another flawed metric. If say exams at MIT resembled SAT, it seems more logical that you'd find a correlation between SAT success and MIT academic success.

What would be a more interesting measure of real value is to study for academic innovation and invention (valuable to society), as well as future success on the job market or at new business ventures (valuable to the student and economy).

I'm making the assumption here that since we have limited educational resources, we'd want to provide the best education to those most likely to advance an academic field through new discovery or insight or invention, as well as those who'd best innovate or provide for existing business and services to society.

And I'm curious if we've ever had any study looking into that? Or if this one did?

Just an interesting anecdote on the predictive power of standardized tests:

When I took the ACT in 2006, I scored around a 24 or 26 for composite score. Not very good. I didn't prep nor study for it because I was pretty apathetic about school and did my best to coast on whatever natural talent I could muster. Since my score wasn't very good, I retook it several weeks later. About halfway through I realized I was given the exact same question set as the first time I took it. I had of course not studied nor prepped for the second time, being the apathetic teenager I was. However this time I score around 32 or 34 or so. I don't remember exactly.

What was different? Why did my score go up about 8 points? Better mood that day? Which score was the "real" one that best represented my abilities?

Did I eat a better breakfast beforehand? I recognized some of the questions but of course I never expected to see the same questions a second time so I didn't prep for that. I didn't prep for anything about it.

I wonder how many kid's college admissions results are ultimately because of one bad or good day? I suppose the other lesson is if you get a bad score, try again. You only need one good score.

MIT and Caltech have always held a 'number-oriented universities' perception in my mind. This is in contrast to a 'prestige-oriented or identity-oriented' perception that I hold for Ivies.

It is nice to see MIT do justice to those priors.

What's surprising though, is that APs and similar exams are not enough. In the UK, I though they essentially looked at A Level results, which are much more representative of what you'll actually study at uni. But I guess both SAT/ACTs & APs must be a better measure that just APs. I just remember fucking hating studying for the SATs though. So boring. SAT IIs were somewhat fun to study for though. In France for instance, they mostly just look at the baccalaureat to get into prep schools / first year at uni. Then exams to get into engr/business/vet schools are actually very interesting topics and very close to what you'll actually study. Same with exams at the end of the first year of med school (which you get into right after 12th grade, unlike in the US where it's after your bachelors).

That being said, they seem to have backed up their numbers, and MIT knows how to count, so they must be right! I just always hoped SAT/ACTs weren't that conclusive so that we didn't have to go through them anymore and could focus on the funner AP/A Level stuff :)

tbh I thought APs were generally more difficult than actual classes at a high-level university (USAFA). And my high school's regular courses (granted a fantastic high school) were actually much more difficult than a state school's courses.

That said, outside of admissions, I don't think I got academic value out of them. They were hard for the sake of being hard. I'd rather have taken the SAT or ACT any day.

(also apropos of nothing but I don't think much of the writing section on the SAT either, which was a hot topic 15 or so years ago... a huge amount is dependent on the graders, and it's fundamentally a "blackboard programming" type scenario where the student is separated from basic resources like word processing and graded on the resulting product... that's not how you would actually work in an academic setting.)

> And my high school's regular courses (granted a fantastic high school) were actually much more difficult than a state school's courses.

I went to a barely-known state university and was very surprised when some of my intro-level gen ed requirement classes mostly covered material I'd already seen in, and with a similar level of rigor to, junior high school. And my junior high and high schools were nothing special at all—at the higher end of performance in the state (so far as those measures are helpful, anyway) but just regular public schools in a state with overall mediocre-bordering-on-poor schools.

If I'd known that the first couple years of college weren't going to be harder than high school, and would have a lower total time commitment, hell, I'd have probably tried to go the drop out -> GED -> start college at 16 or 17 route. I wasn't gonna get into top-tier universities, anyway.

The boringness could actually be a big part of the effectiveness. Efficient study habits and ability to work through boredom certainly help with some undergrad classes. The test would have some predictive power even if it's just measuring those.

Also it's hard to standardize something that is not boring.

AP scores and SAT II's are highly subject to the quality of instruction. I had several teachers who treated it as a more advanced class than honors, but felt no need to teach to the rubric for the test specifically.

I aced the SAT and ACT but had a decent number of mediocre AP scores because I was seeing the material for the first time when I opened the test. Got to college and after a single 45 minute lecture covering the gap material, I'm pretty sure I could have scored a 5. Ended up making for several easy A's freshman year.

There are a lot of highschools where AP classes aren't really available, or are taught with varying degrees of rigor.

I practiced the math part several time to make sure I had it down pat, never the writing part though. Reading those long essays is a chore. I think the reading/verbal part is less coachable than the math part.

>What's surprising though, is that APs and similar exams are not enough.

That isn't what they said. They said that access to those tests is not universal. Students from high schools that don't offer AP classes would have a hard time taking AP exams. This would exclude people from rural or impoverished areas.

This is why the SAT and ACT are useful: they are meant to be aptitude tests. They are IQ tests in disguise. If properly designed, they will measure intelligence with minimal influence from education or cultural background. Theoretically something like these tests could be administered to elementary school students and still be useful for predicting success in college a decade later.

> They said that access to those tests is not universal. Students from high schools that don't offer AP classes would have a hard time taking AP exams.

Yeah, I wish they'd just flat out told me "we expect AP courses" before I applied for MIT back in the day. Would have saved me a lot of hassle that just resulted in "sorry, we wanted AP credits" in the end.

I passed the AP calc exam without a class. But that had a lot more to do with motivation and interest and a sense of entitlement than with aptitude. I wish everyone had my sense of entitlement, but they don't, and classes do seem to make a passable substitute.

It's important to keep in mind that the context here is the SAT/ACT versus the currently available alternatives. They're not saying that the SAT/ACT is good, but simply that it's better than other options. Some of those options, like alternative standard tests, may be significantly better, but in the world as it is today, it doesn't matter if they are, because lower-income candidates don't have access to them.

The current discussion I thought focuses around the idea that higher scores should just about guarantee acceptance due some fantasy notion of objective merit, and here they pretty much say that's not the case here:

> To be clear, performance on standardized tests is not the central focus of our holistic admissions process. We do not prefer people with perfect scores; indeed, despite what some people infer from our statistics, we do not consider an applicant’s scores at all beyond the point where preparedness has been established as part of a multifactor analysis

Seems like semantics - maybe they don't explicitly prefer perfect scores but it sounds like they're filtering out using a minimum bar, and I'd bet it isn't too far off perfect.

> So much for that common, popular notion that standardized tests do not predict anything of value.

It was always an example of people refusing to believe something because it would be nicer if it weren't true.

The University of California commissioned a study to determine the predictive value of SATs for college success. A strong positive correlation was found. The test requirement was dropped anyway.

The problem with testing isn't that it's inaccurate, but that it's a poor tool for the problem that it's being used to solve. You can't express "likelihood of success in college" in one number. Looking back on my time in college, the idea that my success could be predicted by my knowledge of geometry and vocabulary words is laughable.

If there was a test that could accurately predict someone's chances of "success", be it SAT or IQ test, it would be used by everyone for everything. Billion dollar companies would be giving CEOs a version of that test before hiring them. A near perfect SAT score is noteworthy personal trivia, but other than that it loses all meaning as soon as someone steps on campus.

Schools with strong brands use tests because they have way more applicants than they can properly review. Standardized testing is made necessary by scale, not predictive accuracy. I'm sure that anyone who has worked in college admissions for years has a very clear picture of what a successful student looks like. But there doesn't seem to be a way to quantify and codify that knowledge. And there likely wouldn't be time to apply it to tens of thousands of people in a few months anyway. So, standardized testing is what we have until someone comes up with something better.

> the idea that my success could be predicted by my knowledge of geometry and vocabulary words is laughable.

You really should look into the g-factor research. This isn't about knowledge, but rather performance. It turns out that your performance on geometry and vocabulary tests is highly correlated with your performance on tests in such disparate fields as Classics and Music:

https://en.wikipedia.org/wiki/G_factor_(psychometrics)#Cogni...

The crazy result in general intelligence is that your performance in all these areas is highly correlated, and incredibly correlated with career success:

> "Research indicates that tests of g are the best single predictors of job performance, with an average validity coefficient of .55 across several meta-analyses of studies based on supervisor ratings and job samples."

Many years ago--business grad school--one of the professors (forecasting?) had done research into GMATs and various post-school success metrics. Like it or not, the standardized test scores were a better predictor (by far) than anything else.

In terms of gaming scores on standardized tests generally, yes people with more money can take prep classes and the like, but there are also test prep books available--presumably even from the library--that probably get you a lot of the way there.

> common, popular notion that standardized tests do not predict anything of value

It's backed up by research, for example: https://news.uchicago.edu/story/test-scores-dont-stack-gpas-...

I don't doubt that MIT's study showed otherwise for their needs. But UChicago is also a top-tier school and is not requiring standardized tests, for educational not political reasons.

Don't underestimate the value of the kind of conscientiousness it takes to do well on these tests! I found grit much more useful than cleverness when I was an academic researcher.

I wonder why we can't hire more teachers to grade word problems in tests like SAT, like Asian countries do. Those problems are much harder to game or cram.