What Should Be Done About Standardized Tests?

A Freakonomics Quorum
By Stephen J. Dubner

What should be done about the quality and quantity of standardized testing
in U.S. schools? We touched on the subject in Freakonomics, but only insofar
as the introduction of high-stakes testing altered the incentives at play -
including the incentives for some teachers, who were found to cheat in order
to cover up the poor performance of their students (which, obviously, also
indicates the poor performance of the teachers).

Personally, I used to love taking standardized tests. To me, they
represented the big ballgame that you spent all season preparing for,
practicing for; they were easily my strongest incentive for paying attention
during the school year. I realize, however, that this may not be a common
view. Tests have increasingly come to be seen as a ritualized burden that
encourages rote learning at the expense of good thinking.

So what should be done? We gathered a group of testing afficionados - W.
James Popham, Robert Zemsky, Thomas Toch, Monty Neill, and Gaston Caperton -
and put to them the following questions:

Should there be less standardized testing in the current school system, or
more? Should all schools, including colleges, institute exit exams?

Here are their responses. Many thanks to all of them for their
participation. I have to admit, I never saw the parallel between tests and
French fries before, but now that I've seen it, I won't soon forget it.

W. James Popham, author of The Truth About Testing: An Educator's Call to
Action and America's Failing Schools:

Standardized tests have much in common with French fries. Both of them
differ in composition as well as quality. French fries are available in
numerous incarnations, including straight, curly, skins-on, skins-off, and,
in recent years, with sweet potatoes. Regarding quality, of course, the
taste of French fries can range substantially - from sublime to soggy. It's
really the same with standardized tests.

Certain standardized tests (called achievement tests) are intended to show
us what skills or knowledge students have mastered. Other standardized tests
(called aptitude tests) are designed to predict how well test-takers will
perform in future settings, such as when they get to college. Some
standardized tests are designed to differentiate among test-takers so we can
say that Kevin scored at the 82nd percentile, while Melanie's performance
puts her at the 96th percentile. Some standardized tests are supposed to let
us know how well a particular group of students, such as those in a given
school, have been taught. But, just as is true with French fries,
standardized tests can vary dramatically in their quality. Some standardized
tests perform their measurement mission marvelously; others do a dismal job
of it.

Thus, if we're asked whether there should be more or fewer standardized
tests in our school system, the only defensible answer is, "It depends." It
depends on whether the right kinds of tests are being used and whether those
tests are good ones. Given the kinds and caliber of the standardized tests
currently being used in our schools, I come down on the "less" side of the
argument. But that's chiefly because the wrong sorts of standardized tests
are frequently being used. Take the No Child Left Behind Act, for instance,
a federal accountability law requiring scads of standardized tests to be
used in evaluating schools. Do you know that almost all of the standardized
tests now being employed to judge school quality are unable to distinguish
between well taught and badly taught students?

We surely don't need more of those sorts of misleading tests. But we
definitely do need more standardized tests that are sufficiently sensitive
to instructional quality, so we can accurately tell which schools are truly
successful and which ones aren't. Standardized tests can be written that
accurately measure a school's instructional effectiveness, yet also
stimulate teachers to do a better job of teaching.

Turning to the exit-exam question, all schools - kindergarten through
college - should employ exit exams allowing us to determine what students
have actually learned. We owe it to our students to make sure that they've
been properly taught. But when I hear, as I recently have, of a proposal for
colleges to start using end-of-course tests as exit exams, I become
altogether apprehensive. I was a college professor for more than 30 years,
and I assure you that most professors know no more about making exit exams
than they do about making French fries.

Robert Zemsky, professor and chair of the Learning Alliance at the
University of Pennsylvania, and former member of the Spellings Commission:

Discussing testing is roughly akin to planning a visit to the dentist - it's
all about remembered pain. No one really likes to be tested. And yet
high-stakes testing - already a key element in the reform of primary and
secondary education - has become a standard feature of the "let's reform
higher education" industry.

Testing raises a host of problematic questions. Who is being tested: the
student, or the teacher? What is being tested: what the student knows, or
what the student has learned? Should the tests focus on specific knowledge -
like the ability to read a complex text or solve a standard physics
problem - or should the test focus on more general attributes, like creative
thinking and problem solving? Can a test in which the test-taker - that is,
the student - does not have a direct stake in the outcome actually command
the test-taker to do his or her very best?

Then there are the questions of what to do with the results. I have
actually sat through an extended discussion of how we could use regression
analysis to parse out the contribution different teachers made to a group of
students' performance on a set of standardized tests. The answer was, yes it
was possible, and could in fact be used to award merit pay increases. But
nobody left the room feeling very comfortable that there would be any gain
in what we knew made for good teaching.

What we know - and what makes those of us in higher education particularly
leery of generalized tests designed to capture how well an institution
teaches attributes like creating thinking and problem solving - is that the
best predictor of how well a group of college students will do on such a
test is how well they did on the SAT or the ACT. Those instruments may not
be perfect, or even good at identifying scholastic aptitudes, but boy are
they good at telling us who the best test-takers are.

Thomas Toch, co-director of Education Sector, a Washington, D.C., think
tank:

There's a lot of standardized testing in public education. Elementary,
middle, and high school students are taking some 56 million reading, math,
and science tests this year just to comply with the demands of the No Child
Left Behind Act, and many states and school systems layer a lot of other
standardized tests on top of that.

This testing is valuable. Without it, parents, taxpayers, and policymakers
would have a tough time knowing how well schools were performing. That was
the case prior to the advent of the "standards movement" in public education
in the 1990s, when states began setting standards, testing students, and
publicizing the results. Students could fall through the cracks, and many
did, but educators didn't have strong incentives to help them because
without tests that measured students' performance against clear standards,
there was no way of holding teachers and principals accountable for their
students' success.

NCLB took the standards movement to its logical next step, requiring
standards and testing systems in every state, and creating consequences for
schools that failed to make adequate progress with specific groups of
students that public schools hadn't educated very well in the past - the
poor, students of color, English language learners, and the disabled. That
has been the law's most valuable contribution.

But we need much better tests. For a variety of reasons, including the
need to produce vast numbers of tests quickly and cheaply, the majority of
today's state-level standardized tests are multiple-choice measures of
mostly low-level skills, such as the recalling of facts in a reading
passage. They largely sidestep higher-level skills, such as having students
compare and contrast two reading passages, and the open-ended questions that
are best suited to measuring such skills. Roughly half of the nation's
students are taking tests under NCLB that are completely free of open-ended
questions.

This presents a problem, because when tests are high-stakes events, as
they are under NCLB (teachers and principals can eventually lose their jobs
if their students flunk NCLB tests for several consecutive years), educators
have a strong incentive to "teach to the test." In this case, that means
teaching low level skills at the expense of the more demanding material that
everyone says students need to master in today's complicated world.

Exit exams, which students must pass to graduate, make sense. "Social
promotion," or advancing unprepared students, has been commonplace in
schools and colleges for a long time.

But such tests pose tough questions. Two-thirds of the nation's public
high school students currently must pass exit exams in reading and math in
order to graduate. But the majority of the tests measure ninth- or
tenth-grade-level basic skills; passing them doesn't mean students are ready
for the workplace, much less prepared for college. Yet many state lawmakers
have been wary of setting the bar higher for fear of large numbers of
students failing.

But is it fair to give students what amounts to a counterfeit passport to
college or work? And do such tests spur high school teachers and principals
to aim high with their students? To both questions, the answer is, "No." In
most states today, high school exit tests serve the same role as the
standardized tests mandated by NCLB: they try to jack up the floor of
student achievement in the nation's schools. The best high school exit tests
would be end-of-course exams akin to the "comprehensive" exams that many
colleges and universities require students to pass in their majors before
graduation - tests, that is, that would raise the ceiling of student
achievement.

Monty Neill, executive director of FairTest:

The No Child Left Behind law has had one clear accomplishment: it has
given a black eye to education policies based on the overuse of standardized
testing.

NCLB's testing mandates have flooded American classrooms with millions of
additional tests. At the same time, the rate of learning improvement has
actually slowed, according to the National Assessment of Educational
Progress (NAEP).

A mounting pile of surveys and reports document the negative consequences
of testing overuse and abuse, as well as growing public opposition to the
test-and-punish approach. For more evidence, just listen to the roars of
approval when any of the presidential candidates criticizes the law. No
wonder more than 140 national education, civil rights, religious,
disability, parenting, and civic groups have called for its comprehensive
overhaul.

Having long tracked the misuse and abuse of such tests, FairTest predicted
a range of negative consequences from NCLB. Most have now been documented by
independent researchers. The problems are compounded by high school
graduation tests, and by pressure to score high on college admissions exams.

High-stakes testing has narrowed and dumbed down curricula; eliminated
time spent on untested subjects like social studies, art, and even recess;
turned classrooms into little more than test preparation centers; reduced
high school graduation rates; and driven good teachers from the profession.
Those are all reasons why FairTest and other experts advocate a sharp
reduction in public school standardized testing and a halt to exit exams.

One-size-fits-all testing schemes make even less sense for colleges and
universities. How could one exam ever accurately assess the learning of
students majoring in subjects as diverse as art history, biomedical
engineering, and political science?

As such, the politicians blindly mandating such exams are the ones outside
the mainstream, not assessment reformers like us. Indeed, the testing
industry's own standards state that no single exam should be used as the
sole or primary criterion to make high-stakes educational decisions such as
promotion, retention, graduation, college admission, or scholarship awards.

There are better ways to assess student learning. Classroom-based
information, such as grades, provides richer evidence of performance. High
school grade point average is a better predictor of college success than
either the SAT or the ACT.

The nation does need better assessments and more training for educators to
get the most out of them. FairTest has long promoted high-quality,
classroom-based assessments that can be used to improve student learning and
teaching. We also support the more than 760 colleges that do not require
admissions test scores for many or all of their applicants.

High quality assessment is an educational necessity. But high-stakes
standardized tests harm educational quality and promote inequity.

Gaston Caperton, president of The College Board:

The quantity of testing is less important than the quality of testing.
This is where the SAT excels. In an era of rampant grade inflation, the SAT
offers students the most level playing field available to demonstrate their
knowledge of core material. The SAT, in combination with the grade point
average, provides students, parents and admissions counselors with the best
predictor of academic success in college.

As for the question about exit exams, I think they largely exist already
in the form of final exams in various subject areas, both in high school and
in college. The SAT is unique in that it provides a focused look back at a
student's accomplishments in high school while offering a glimpse into that
student's potential in a college environment.