Posts tagged ‘SAT’

SAT Scores Are Bad Education Evaluation Data

I am happy to see the public school system coming in for much-deserved criticism.  I don't have anything to add to this article that I have not already said about schools many times.  But I want to make one complaint about a chart used in the blog post:

sat-scores

 

SAT scores are a terrible metric for measuring academic performance over time.

First, I am not at all convinced that the test scoring does not shift over time (no WAY my son had a higher score than me, LOL).

But perhaps the most important problem is that all students don't take the SAT -- it is a choice.  Shifts in the mix of kids taking the test -- for example, if over time more kids get interested in college so that more marginal academic kids take the test -- then the scores are going to move solely based on mix shifts.  Making this more complicated, there is at least one competitive test (the ACT) which enjoys more popularity in some states than others, so the SAT will represent an incomplete and shifting geographic mix of the US.  Finally, as students have gotten smarter about this whole process**, they gravitate to the ACT or the SAT based on differing capabilities, since they test in different ways.

To me, all this makes SAT scores barely more scientific than an Internet poll.

** If you have not had a college-bound student recently, you will have to trust me on this, but parents can spend an astounding amount of time trying to out-think this stuff.  And that is here in flyover country.  Apparently private school parents on the East Coast can be absurd (up to and including hiring consultants for 6 figures).  A few years ago it was in vogue to try to find your kid a unique avocation.  Violin was passe -- I knew kids playing xylophone and the bagpipes.  A friend of mine at a high profile DC private school used to have fun with other parents telling them his son was a national champion at falconry, the craziest thing he could make up on the spur of the moment at a cocktail party.  Other parents would sigh enviously, wishing they had thought of that one for their kid.

Accountability to Forecasts of Doom

Activists are always making exaggerated statements on current problems and extrapolate these into forecasts of doom.  One thing activists really, really hate is when people come back later and hold them accountable for these forecasts.  You can see it as NASA officials squirm and fire off condescension at skeptics who have the temerity to actually check their global warming forecasts against actual temperatures.

If I had a newspaper, I'd have a special regular feature where I dig back 10-20 years in my archives to find such forecasts of doom and check them against reality  (actually, if I had a paper, I would not allow activist's press releases to show up virtually unedited as "news" stories, but that is another matter).  Heck, I could have a regular feature just reality-checking old Paul Ehrlich forecasts.

Well, I don't have a newspaper, but I do have a blog, and this is a new feature I am working on.  I am still trying to play with various search engines and news libraries (such as the NY Times) to see if I can come up with some kind of query format that efficiently digs up such predictions that are at least 10 years old.  I am still a little stumped on this, but I am working on it.

But, as a sort of beta-test of the feature, one such comparison fell into my lap today.  I remember my feminist wife reading a book published in 1994 called "Failing at Fairness."  This work was a big, big deal at the time.  Media such as the NY Times fawned on it.  I will let a 1994 review on the Society for Women Engineers' site summarize the book:

Failing at Fairness: How American Schools Cheat Girls eloquently describes the results of years of research into sexism in schools. The study began as an examination of gender bias in textbooks, and evolved into a decade of painstaking classroom observation uncovering a "hidden curriculum" in classroom interaction.   Authors Myra and David Sadker present a compelling tale of gender bias in education at all levels.

Taken at face value, the book more than proves the point of the subtitle: our schools cheat girls out of an education equal to that received by boys. The authors do an excellent job of pointing out some of the more subtle ways of favoring boys over girls. However, so many descriptions of incidents of sexism -- blatant, subtle, by old teachers, young teachers, male teachers, female teachers, and even by one of the Sadkers' own "trained" researchers -- are included that it can seem like overkill at
times. In addition, the wealth of statistics can be overwhelming, and yes, even slightly depressing.

One of the more horrifying aspects of Failing at Fairness is the discussion about standardized tests, their historical deliberate design as culturally biased for exclusionary purposes, and the dive in the scores received by girls as they progress through their education.

Current standardized test administrators claim to be more sensitive to cultural prejudices in today's tests, although minority students still score less than white students (at least on the SAT). Also, the book states quite plainly, "Regardless of ethnic or racial background, all American girls share a common bond: a gender gap in test performance that leaves them behind the boys." The prevailing opinion of the discussion group is that the tests are still exclusionary; they are not measuring achievement, but are rather reflecting the way students are taught.

I don't doubt that they found their share of anecdotal issues.  I am sure I could find them today.  But their overall premise that girls were getting hosed by primary education and that standardized tests were structured to exclude girls from college education made no sense even at the time the book was published:

male_female_jobs

The chart is from Mark Perry, and he shows a similar picture for bachelor's degrees, where women blew past men in 1981, and in PHDs, where women passed men in 2006.  People would laugh at this book today, as most discussion is about under-performance of boys.

I don't know the authors, but I would interpret this as the classic inability of activists to declare victory.  I am fairly certain that their hypothesis was far more correct in 1969 than in 1994.  But society really went through a step-change in the 1970s vis a vis attitudes about females.  The previous generation of women's activists did great work to make these issues plain and help lead change in societal attitudes.

But activists have a really hard time declaring victory.  From a quite personal standpoint, declaring victory as an activist is exactly the same as walking into your boss and telling him that the company really doesn't need your job position.  Money, prestige, academic advancement, and attention, and (self-esteem, for certain types of people) are all tied to there being a major problem.  If there is no longer a big problem, then all this stuff goes away.

Increase Ivy League Capacity

There have been a number of articles of late about college admissions and Asians.  For example, my alma mater Princeton is getting sued by a young man who says the school's admissions standards are discriminatory against Asians  (he was forced to go to Yale instead, which in my mind represents substantial pain and suffering).  David Bernstein at Volokh also had this:

Liming Luo is a high school senior who is both a math prodigy and received a perfect 2,400 score on her SATs.  New York Magazine
asked Katherine Cohen, CEO and founder of IvyWise, a school-admissions
consulting company, about her [and other students'] prospects for
admission to MIT, the college of her choice. The answer:

Her perfect SAT score is truly outstanding but not a free ticket.
She is applying to many technical colleges, so she will be competing
against a lot of other high-achieving math/science kids (and a lot of other Asian students in particular). While she may be admitted to MIT early, I am not convinced she's a shoo-in"”I'd want to see more evidence that she's giving back to the community.

I don't know enough to comment on the Asian issues, but I wanted to make a couple of other points.  First, Bernstein is probably correct in wondering why there is such a focus on "giving back to the community" for an 18-year old girl who appears to be a math genius.  But his question is naive.  I can say from experience that everything on an application for college may be negotiable (e.g. good athletics allows for lower SAT scores) except for community service.  That has become inviolable.  Every college prep school I know have elaborate programs nowadays to make sure their kids get lots of community service hours.  My son, at the age of eleven, missed on his first shot at National Junior Honor Society because he only had about 20 hours of community service.  I can tell you that for college-bound high school kids, community service is longer about volunteering and giving back but about grimly checking off one of the most important boxes for college applications.

My other thought is that you don't have to be Asian to worry nowadays that near-perfect SAT's and grades are not enough to get one into the Ivy League.  As you can see here, placing in the 99th percentile on SAT's only gives one a 1 in 5 shot at getting in to Princeton.  The other thing you can see is that top Ivy's are being honest when they say they want more than just good grades -- you can see at Princeton and Harvard that moving from 91st to 99th percentile on SAT's does little to improve a person's prospect of getting in.  (On the Asian discrimination issue, that means that more than half of the kids in the top 1 percentile of SAT's will get turned down by Princeton, and some of these will be Asians.  Whether that is discrimination or just brutally tough admissions is hard to say).

Which leads me to my main point -- the Ivy League needs to find a way to increase capacity.  The number of kids that are "ivy-ready" has exploded over the last decades, but the class sizes at Ivy schools have remained flat.    For years I have been campaigning at Princeton for this, and I am happy to see they are increasing the class size, but only by a small amount.  Princeton has an endowment larger than the GNP of most countries.  To date, it has spent that money both well and poorly.  Well, because Princeton is one of just a handful of schools that guarantee that if you get in, they will make sure you can pay for it, and they do it with grants, leaving every student debt free at graduation.  Poorly, because they have been overly focused on increasingly what I call the "educational intensity" or the amount of physical plant and equipment and stuff per student.  In this latter case, we have got to be near the limit of spending an incremental $10 million to increase the education quality by .01%.  We should instead be looking for ways to offer this very high quality of education to more people, since so many more are qualified today.

By the way, one of the reasons Ivy League schools don't take my advice is because of the faculty.  The very first thing that the faculty wants is more endowed chairs, more equipment, more office space, etc.  The very last thing most faculty wants is more students that would force them to actually teach more rather than publish and do research.

Postscript:  OK, I will make one comment about the Asian kids thing.  I don't know if what Ivy admissions offices are doing is discriminatory or not.  But I do know that among the white parents of college-bound high school students that I know, there is real undercurrent of anti-Asian resentment.  I can't tell you how often I hear stuff like "Oh of course he does well, he's Asian" or "I don't know if my kid can get into X, all the Asian kids get the spots."  Its a strange, resentful sort of racism I see all the time from parents who would never be caught dead uttering anything untoward about blacks.  There is this funny feeling I get in some of these conversations that it's OK to dislike Asians in a way that would never be perceived as OK for blacks.

 

Taliban at Yale, and Advice for Princeton

Everyone seems worked up about Yale admitting an official of the Taliban as a student.  While I find the guy in question pretty bankrupt, I'm not sure I am very excited about starting down the path of vetting potential college applicants against some political extremism standard.  I am sure there are any number of Ivy League freshmen whose beliefs I would find horrifying, but I don't feel the need to start culling them out.  I do find it odd that Yale would have recruited this guy like he was some kind of rock star, and celebrated his choice of Yale as if he was some prize. 

As I have written to my Alma mater Princeton on any number of occasions, I think that Ivy League schools are making a huge mistake which is tangentially related to Yale's Taliban student.  If the University of Texas had accepted him as one of 10,000 or so in their freshman class, there would not be so much outcry.  But this is an Ivy League school, with 20,000 or more kids competing for 1500 freshman spots.  Every parent tends to think, "so my kid with straight A's and a 1350 SAT and 200 hours of community service got turned down at Yale so a misogynist fascist with a 4th grade education can attend?"

Instead of arguing about admitting one less Taliban guy, I urge Ivy League schools to find a way to bring their higher quality of education to many more people.  Princeton, Harvard, and Yale each have endowments over $10 billion each, and they use this money every year to increase the education intensity to the same 1500 people per class.  Every time I go back to visit campus, I see more buildings, equipment, facilities, professors for the same 1500 folks.  Enough!  At some point there has got to be a diminishing return.  It is time for someone in the Ivy League to take the leadership to redefine their mission away from the current facilities arms race with the other Ivy's and towards a mission to broaden their reach in the country.  Instead of yet more molecular biology equipment for the same 1500 people per class, lets find a way to bring a Princeton education to, say, 6000 people a class.  Lets quadruple the size of the Ivy League.

Of course, the Ivy League conservatives (which means, in this context, everyone who graduated before this year and all of the faculty) fear this change.  The last thing the faculty, who we know to be in charge of the asylum from the whole Sommers affair, want is to have more students to teach -- they want the toys.  And alumni fear that somehow the "essential essence" of the university might be lost, though everyone made that same argument when these schools went coed and few today would argue to reverse this decision.   Administrators argue that the freshman pool would be diluted, sort of like the argument about pitching in baseball after expansion.  But one only has to look at admissions numbers to see that quadrupling the freshman class size would cause the Ivy's to lower their standards to... about where they were when I got in!  (If your SAT scores are in the 98th percentile you still have only a 10% chance of getting into Princeton or Harvard.)  The fact is that the pool of high school students in the upper echelons and Ivy-ready has grown tremendously in the past few years, causing Ivy's to narrow their admissions qualifications to near ridiculous levels, with average SAT scores in the stratosphere, hundreds of hours of community service, multiple sports letters, and consultant-aided choices of special activities to differentiate students from the crowd (e.g. bagpipes or falconry).

I understand that this is difficult -- just the issue of physical space is daunting.  But these are the leading Universities in the world.  Surely there is enough brainpower to figure it out if the mission is accepted.  The University of California has of late been doing a lot of interesting things to bring college education to the masses, and dealing with the fact that the number of people who can afford the cost and time of a college degree has increased exponentially.  I think the Ivy League needs to work through the same exercise at the top end of the bell curve.  They need to address a similar near exponential expansion in the number of students who are "Ivy-ready."

Getting Into Ivy League Schools

Since I went to two Ivy League Schools (Princeton undergrad, Harvard MBA), I get asked by parents a lot about how to get their kids into an Ivy League school.  My answer is the same one that I think many of my friends from college give:  "I'm not sure I could have gotten into Princeton if I did it today, rather than 20 years ago".  While the number of bright, qualified students seems to have gone up tenfold over the last decades, the number of admissions spots at Ivy League schools has hardly changed, and few new schools have emerged as Ivy League equivalents (if not in fact, at least in the perceptions of the public).

I have recently discovered this really nice blog by Kurt Johnson, who recently got accepted to attend Wharton business school next year.  He has several good posts about school rankings and admissions, including this one here.  The curves showing that only about 20% of applicants in the top 1 percentile of test scores get into Princeton is scary.  Yes, I had good SAT scores, somewhere in the 1500's  (I would never have believed at the time I would have forgotten the number, but I seem to have).  At the time, that was pretty much a layup for getting into the Ivy League, though I had some decent sports and activities as well.  Now, the odds are I wouldn't make it.

Today, parents are downright crazed in trying to figure out what it takes to get in.  For example, any of the 11 year olds at our elementary school do community service, which I guess is fine though it seems to be driven more by setting up early resume wins rather than saving the world.  Things like piano and violin are out:  Parents are pushing their kids into more unique, differentiated instruments like bagpipes or the xylophone.  My old college roommate, whose kids go to a college prep school in DC, joked that he planned to send the other school parents into a jealous hysteria by telling them his kids were competing in falconry.

Kurt also makes a good point about one of my pet peeves of performance measurement:  that is, measuring a process based on inputs rather than outputs.  You see this all the time, for example, when the department of homeland security talks.  They say things like we have xx thousand agents making xx checks with xx equipment blah blah.  Yes, but are we safer?

Postscript: By the way, after reading Kurt's work, he is basically going to Wharton for a piece of paper.  He already appears to be at least as thoughtful an analyst of business issues as most poeple I know with Ivy League MBA's.  OK, this is a bit unfair.  I learned a lot that was useful in my first year of busienss school, then I entertained myself in the second year with a lot of material that was interesting but I never used much.  My MBA was sort of a 1-year technical degree with an extra year in "business liberal arts".  I have talked to lawyers that say the same thing about law school.