The Viability of Standarized Testing

I may not get to a full posting before Monday, but I will throw some ideas out. First, my experience is with entrance tests - SAT, ACT, GMAT, GRE, LSAT - and I will confine my comments to them alone. Second, I will modify my earlier exuberant language to say that standardized entrance tests are relatively better than other forms of assessment, and, I think they are pretty accurate as indicators of raw ability. Third, most of these entrance tests (particularly the graduate level ones) are only secondarily about content; the higher difficulty questions are really testing the student's ability to reason deductively and apply general principles to unexpected situations. Most of my teaching consists of critical reasoning skills. I know that sounds strange, so I will try to post some examples next week.

There are certainly going to be flaws in any standardized model, but I do support the basic idea of standardized testing. We live in a society of strangers today. Because of industrialization, geographic mobility, and the resulting atomization of traditional forms of community, we spend most of our lives interacting with strangers at work, school, and in our neighborhoods.

This is different (even if only in degree) from the past. Imagine with me a Puritan community in Massachusetts during the mid to late-17th century. The township is small enough so that pretty much everybody in town knows everyone else. A majority of the inhabitants have lived in the town for many years and a signficant minority have been born and raised there (at least, relative to today). Almost every villager is a member of the local, established parish. In this town you would personally know the abilities or skills of each potential laborer. So, for example, if you were a shopkeeper you'd know which kids were adept with numbers and would make good assistants. You would not need a formal, standardized testing procedure in this community.

But today we no longer live, work, or go to school among neighbors (in the old sense). Because we interact with the unknown we need some way of measuring the reliability and skills of people with whom we have no acquaintance. If we do not develop some standard of comparison a community of strangers cannot, or at least cannot efficiently, comingle and interact. Communities are formed by bonds of trust. How do you trust a stranger?

For business transactions we've seen the evolution of contract law. In an intimate society, your word can be your bond. The community acts as an arbitrator and can enforce the verbal contract by shunning the transgressive party. In a society of strangers we need written contracts. That is the cost of doing business today. Without strict legal guidelines and our judicial system, our modern society would be impossible. The development of standardized testing in education is a similar response to our modern society of strangers.

If you are a college seeking the best and the brightest students, how do you determine whether a student is da bomb or a bust? They have to have some way of comparing students to each other. This criterion must be standardized, applicable to all students; you cannot make a comparison without a consistent measuring stick. This is the basic flaw behind using GPA as your main measure of student performance. Every school grades differently. Some schools are guilty of rampant grade inflation. A GPA of 4.0 at one school might be equivalent to a 3.0 at another. With thousands of different grading standards, colleges and graduate schools have no hope of using GPA as a consistent measure.

Thus the introduction of the SAT, ACT, LSAT, GRE, etc. They attempt to, however imperfectly, fulfill a legitimate need.

I think I can add something here. The standardized tests are definitely in favor of the quick thinkers. Standardized tests indicate future testing ability and raw cleverness more than diligence, perseverance, etc. So, these tests only give part of the picture, but they do contribute an important part. Standardized tests are primarily aimed at the critical thinking skills of students. They are not designed to test how much content the student has accumulated (except for the subject specific tests), but how well the student is able to apply familiar ideas to unfamiliar situations. That's why the math caps at about Algebra II rather than going through Calculus, which is usually available to high schoolers. Here's an example of a quantitative question from a GMAT (business school):

The genius of this question type - data sufficiency - is that it pretends to be a math question while being nothing but pure logic. The student, after reading the question stem, must determine what information he/she needs to answer it, then evaluate the statements to see if they provide the information. Students who do unnecessary math, such as actually finding out the total number of games played by the two teams, are punished by the strict time limitations of the GMAT.

(For anyone who was wondering, the answer is C)

What you are arguing would matter if Robert Schaeffer was actually citing data. But all that Schaeffer gives as evidence is his own opinion:

Schaeffer's opinion isn't worth a hill of beans as evidence unless he is an unbiased expert. This is why expert witnesses in court are required to have some proof of expertise and must not have ties to the party for which they are testifying. Thus I believe that it is perfectly legitimate to note his position as a professional spinner for an anti-testing advocacy group. All we are given is his opinion, and as I've noted, that opinion is suspect because it is biased.

This is misdirection. The question posed by this forum is the viability of standardized testing in general, not specifically whether or not homeschoolers should be required to take such tests. These two questions are distinct though both are indeed worthy of discussion.

I do. Look at the second page:

Indeed, they are more likely to require that homeschoolers take extra standardized tests above and beyond what non-homeschoolers have to take. This is logical. The less independent verification the college can get, the more of an emphasis they are likely to place on standardized testing.

You have a gift for making things sound as unappealing as possible.(-; But what a standardized test tests for is not the person but what they know and how they think. Knowledge can be standardized. Either you know when Columbus sailed the ocean blue or you don't. Beyond just measuring what you know, a well designed test can determine your ability to think in certain ways. Logical thought, analytical ability, and comprehension are measurable.

Still, you are right to note that standardized testing is imperfect, that there are exceptions, and that success at test-taking is no guarantee of future success. That being said, the flaws of standardized testing are not grounds for denying any and all merit.

They cohosted a conference back in May and they seem joined at the hip in anti-testing articles that I've seen. http://www.fairtest.org/fairtest-nea-state-assessment-reform-conference

You're talking about two different things here. The whole point of a standardized test is to assess student performance more accurately than professional educators can. This is precisely why the NEA hates standardized testing...it reveals how bad a job many professional educators have done educating America's children.

And I propose that the rise of the ACT and SAT is precisely why colleges no longer require a GED. Simply put, standardized testing is a superior criterion to high school equivalency GEDs for colleges which are looking to judge the abilities of strangers. A GED is limited because not all GEDs are created equal. The advantage that a test like the ACT or the SAT has is precisely that every ACT is equal. Since colleges have standardized tests to measure ability they no longer place as much value on GEDs and such. This is why I wrote the whole "bite the hand" comment. Standardized testing has made access easier for homeschoolers by and large, not harder.

But all that said, as you noted, "Yipee," we agree on the basic issue.

PS - Do you have a cheat sheet for all those fancy little emoticons I can borrow?

I think some broader factors should be considered here.

First, speaking objectively and not relative to some pragmatic goal (like getting into college), I've never read any respected educator who thinks well of objective tests

Take Jacques Barzun for example, a great model of a scholar, humanist (in the best sense), educator, and writer. Barzun wrote extensively on teaching and education, having served at Columbia for most of his life, including as Dean of the Graduate School. In numerous places, including his book Begin Here: The Forgotten Conditions of Teaching and Learning, he comes out explicitly against objective, multiple choice tests. He thinks they are worse than useless. Barzun, for people who know who he is, should mean a lot, but he's hardly alone. The point here is that multiple choice, objective tests are intrinisically unhelpful in the process of education. Their only value, and this is key, is with respect to people other than the student, and with respect to goal's other than education. Objective tests go against good education, as people like Barzun well know, but they fit with the standards of bureaucratized systems because they provide objective, shared assessments.

So, objective tests, of which standardized tests are instances, are not helpful for educating people nor are they evidence that someone is educated.

Second, the way we assess is provincial; many do not do it this way. When German exchange students, or students from many other countries (e.g. I know a guy from Latvia educated in the Russian system) come to my graduate school, the freak out because some classes have tests, which they simply do not use in their home system (the Latvian wrote essays even for mathematics exams; go figure - he's good at math, by the way). So, the whole idea of exams and tests used the way we do is not only recent but it's also far from universal. Indeed, anyone who knows anything about an Oxbridge education knows that, even today, practically everything rests on how well you do on your exams that you take at the end of you degree; those scores determine whether you get a First, Second, or Third Class degree.

Third, even given their value for bureaucrats, standarized, objective tests are still of highly dubious value for the people, ultimately, for whom they are most relevant: the teachers/employers of the people who take the test. I have yet to meet a respected academic or scholar in my field who thinks well of GRE scores; in fact, one of the leading scholars in the history of philosophy told me in correspondence that he found the GRE's worthless for predicting student success. Again, this is not uncommon.

Fourth, if you think of all the really important means of asessments, like BAR and CPA exams, exams for people graduating for a degree from Oxford or Cambridge, the battery of tests use for comprehensive exams in academics, or the tests administered to people in the military/government jobs who are seeking to move up in rank, none of them center on (some don't use any) objective, multiple-chooise tests because what such tests assess is normally utterly worthless for gaining a sense of people's fitness at that level.

Fifth, a key question to ask about any test is: who is assessing the results? If only experts in the respective fields tested on are qualified to review and assess the test data, then you are likely to have a test that actually tells you something worthwhile; if any idiot can assess the test results, as is necessarily the case in objective, standarized tests, then you have a very good idea of how limited the data from that test is. This is why people like Barzun, by far among the historical majority, place the emphasis on essays. Objective tests cannot tell you any of the important things about a students' knowledge; just knowing when something happened is, truly, worthless if you don't know how the student thinks of that data. I am positive I know a great deal more about certain events than some people who may remember the dates of them better than I. My knowledge cannot be assessed on a standarized test; only an essay would adequately reflect it; their knowledge could be objectively assessed, and those are the types of students' favored by objective exams.

Finally, I don't think the solution is prior grade records. Educators know those are not very helpful because there are not uniform standards of grading, and grade-inflation is a problem that runs from the lowliest schools all the way into the Ivy League. As many graduate schools will point out on their websites, a 4.0 from PoeDunk U is worth far less than a 3.2 from the University of Chicago or a 3.5 from Princeton. That's why admissions commitees so value the only two generally reliable indicators of student preparation: writing samples and (to a lesser extent if the person is unknown) recommendation letters.

So, I think objective tests have little to any intrinsic value; indeed, Barzun and others would say they are positively harmful as we use them; and at the scale at which we use them, I don't know what sensible person could disagree - I know many teachers who are frustrated by the idiocy of having to "teach to the test," instead of teaching their subject as it should be taught they teach it according to some standarized test requirements.

Although the nature and standards of helpful assessment vary from field to field, the principles do not change much; look at great schools with proven and deserved reputations and you will see patterns, patterns that don't involve heave weighting (if they are used at all) of standarized tests. Oxford is a good example if you have never looked at the admissions process (it involves, among other things, formal examinations in you field as well as an interview that can essentially be an oral examination by an expert in the area to which you are applying).