As someone who got a perfect score on the GRE Quant section, I generally agree with your criticism of that test.
However, I think you don't necessarily make the best argument about preferring GRE Quant vs SAT Math.
There are large differences in the groups taking the tests. First of all, way more kids take SAT than GRE. About 2mn kids take SAT per year, while about 300k or so take GRE. That means that if about 1% of kids get a perfect score on SAT Math, that's about 20k kids. If 9% of kids who take GRE get a perfect GRE Quant score, that's about 27k. Assuming just that these are the same groups of people (ignoring that they don't take it the same year), then it's really not all that different at identifying the best people per year at math in the full population. But you would also want to take into account that the people taking the GRE are most of the way through their college experience and thinking about grad school. Much different than the population taking the SATs.
The problem with the Putnam test is on the other end of things. It is too hard, especially if you gave it to everyone. If you gave it to everyone, then the median isn't 2, it's 0. So it would only be useful for the top programs trying to get the top people.
Interestingly enough, the GRE purposefully restricts the score at the top end because of strong Asian scorers. See Section 2.2:
I think the idea is to limit the advantage that comes from grinding test prep. So the trick would be coming up with a Quantitative test that is more correlated with IQ at the high levels and such that test prep wouldn't help you too much.
“Interestingly enough, the GRE purposefully restricts the score at the top end because of strong Asian scorers.”
I don’t think you are making the persuasive argument you think you are making.
Your point re: gaming the system is fair enough. But your identical point suggesting hard work and effort (“grinding”) being bad… is not.
I suspect I am not the only one who would say that selecting only for IQ as opposed to the union of IQ and diligence would be far from an optimal thing.
I was about to reply that although IQ is easy to test for and is a broadly useful advantage, high levels of diligence, while also broadly useful, are harder to test for unless they've already been invested in an unhealthily narrow field. Aiming all your diligence at a metric for years in order to make your diligence legible is quite doable - but not usually good for you.
However, then I realized that these tests don't exist to incentivize behaviors that are good for people. They don't even exist to identify the smartest and most diligent people. They exist to identify the people who will perform best in graduate programs and the careers that follow from them.
Are graduate programs and careers that require graduate degrees generally known for accommodating healthy well-roundedness? They are not.
So in a sense, perhaps it would be a good thing for these tests to identify the people who have a precedent of aiming all their diligence at a metric for years and not turning aside for something else. Because that's what they'll most likely be in for in the following decades as well.
I think IQ and conscientiousness are good qualities, but it's tricky when the test is measuring both, especially when some subgroups are incentivized particularly strongly to invest in test prep.
As someone who got a perfect score on the GRE Quant section, I generally agree with your criticism of that test.
However, I think you don't necessarily make the best argument about preferring GRE Quant vs SAT Math.
There are large differences in the groups taking the tests. First of all, way more kids take SAT than GRE. About 2mn kids take SAT per year, while about 300k or so take GRE. That means that if about 1% of kids get a perfect score on SAT Math, that's about 20k kids. If 9% of kids who take GRE get a perfect GRE Quant score, that's about 27k. Assuming just that these are the same groups of people (ignoring that they don't take it the same year), then it's really not all that different at identifying the best people per year at math in the full population. But you would also want to take into account that the people taking the GRE are most of the way through their college experience and thinking about grad school. Much different than the population taking the SATs.
The problem with the Putnam test is on the other end of things. It is too hard, especially if you gave it to everyone. If you gave it to everyone, then the median isn't 2, it's 0. So it would only be useful for the top programs trying to get the top people.
Interestingly enough, the GRE purposefully restricts the score at the top end because of strong Asian scorers. See Section 2.2:
https://www.ets.org/s/research/pdf/gre_compendium.pdf
I think the idea is to limit the advantage that comes from grinding test prep. So the trick would be coming up with a Quantitative test that is more correlated with IQ at the high levels and such that test prep wouldn't help you too much.
“Interestingly enough, the GRE purposefully restricts the score at the top end because of strong Asian scorers.”
I don’t think you are making the persuasive argument you think you are making.
Your point re: gaming the system is fair enough. But your identical point suggesting hard work and effort (“grinding”) being bad… is not.
I suspect I am not the only one who would say that selecting only for IQ as opposed to the union of IQ and diligence would be far from an optimal thing.
I was about to reply that although IQ is easy to test for and is a broadly useful advantage, high levels of diligence, while also broadly useful, are harder to test for unless they've already been invested in an unhealthily narrow field. Aiming all your diligence at a metric for years in order to make your diligence legible is quite doable - but not usually good for you.
However, then I realized that these tests don't exist to incentivize behaviors that are good for people. They don't even exist to identify the smartest and most diligent people. They exist to identify the people who will perform best in graduate programs and the careers that follow from them.
Are graduate programs and careers that require graduate degrees generally known for accommodating healthy well-roundedness? They are not.
So in a sense, perhaps it would be a good thing for these tests to identify the people who have a precedent of aiming all their diligence at a metric for years and not turning aside for something else. Because that's what they'll most likely be in for in the following decades as well.
I think IQ and conscientiousness are good qualities, but it's tricky when the test is measuring both, especially when some subgroups are incentivized particularly strongly to invest in test prep.
Well, with you logic, what test other than an IQ test is worthwhile??