By Jay Mathews
Washington Post Staff Writer
Tuesday, December 4, 2007; 12:28 PM
On Dec. 13, The Washington Post will mark the 10th year of the Challenge
Index, my high school rating system, with our latest ranked list of all 185
public schools in the Washington area. Since 1998, Newsweek magazine also
has been publishing its national best high schools list using the same
method.
I am particularly excited this time because we have some competition. U.S.
News & World Report, at the urging of Andrew J. Rotherham, my friendly
adversary on this issue, has just published its own " America's Best High
Schools" list at usnews.com. I have long celebrated what I call the School
Rating Scoundrel's Club, composed of those of us who think that rating and
ranking -- despite their many critics -- are useful ways to help readers
figure out which schools are best for them. I admire the U.S. News college
rankings and am intrigued by its new high school list. It is strengthened by
Rotherham's commitment to improving schools, but it is also too complicated
for its own good.
The Challenge Index rates and ranks schools by just one number, the
college-level test participation rate, calculated by dividing the number of
Advanced Placement, International Baccalaureate or Cambridge tests --
college-level exams given in high school -- by the number of graduating
seniors. The U.S. News list mixes together several numbers. It looks for
schools in the 40 states for which it has data whose average state test
scores exceed statistical expectations and whose minority proficiency rates
exceed state averages for those groups. Schools that survive that initial
screening are then ranked based on a weighted formula that includes both AP
test participation and AP test scores.
The essential differences between the two ways of ranking reflect the
differences between Rotherham and me. Only 36, Rotherham has served as an
education adviser to President Bill Clinton, has founded two education
policy and research organizations and is a member of the Virginia Board of
Education, the youngest appointee to that board in modern times. He is a
policy maker. His high schools list is based on key factors in the policy
process: test scores, minority achievement and college readiness as measured
by AP participation and success. U.S. News and the statisticians at Standard
& Poor's, led by Paul Gazzerro, the director of analytical criteria for
School Evaluation Services, have compiled the list using a basic
policy-making tool--data collected each year by state governments.
At 62, I have been a journalist for 40 years. With editors and researchers
at The Post and Newsweek, I have relied on the basic reporting tools of
direct contact with principals, teachers, students and parents. Our focus is
not what works for policy makers but what is most useful for readers,
particularly parents, trying to judge the quality of their local schools and
others that might be available to them. I think college-level test
participation is the only available comparative factor that allows parents
to see how much value schools are adding to their children's lives. It is a
much better measure than average test scores, which largely reflect not how
good a school is but how wealthy and well-educated its parents are. AP, IB
and Cambridge test participation also has the virtue of being a factor that
smart school leaders can improve very quickly by removing the barriers to
taking such courses that still exist in most high schools. Allowing all
students to take AP and IB in turn creates incentives to improve preparation
for those courses and tests and enrich the teaching of all classes. Raising
the incomes and education levels of students' parents is not nearly so easy.
A look at the top 100 schools on Newsweek's list last May and on the U.S.
News list just released shows three significant differences. The U.S. News
list has eight of the most selective high schools in the country, with the
Thomas Jefferson High School of Science and Technology in Northern Virginia
at No. 1. The Newsweek list excludes such schools, placing them on a
separate "Public Elites" list because the Challenge Index is designed to
show which schools are challenging average students and doesn't work with
public magnet or charter schools that have few or no average students.
Secondly, several big urban schools with many low-income students and strong
AP programs appear on the Newsweek list but not on the U.S. News list
because of their low average state test scores. Third, about three dozen
schools with strong IB programs are on the Newsweek list but not the U.S.
News list, which only uses AP in its rankings.
I asked education experts with very diverse views what they thought of the
new high school ranking rivalry. Those opposed to the test-based rating
systems required by the federal No Child Left Behind law were, as you might
guess, not happy with either magazine's list. Author Alfie Kohn, whose
latest book is "The Homework Myth," said: "Just as many colleges are finally
beginning to opt out of U.S. News & World Report's infamous rankings,
pointing out the damage done by that list and the absurdity of its premises,
the magazine decides to extend this nonsense to high schools. The whole
enterprise is noxious; it's about winning, not learning--as though we were
talking about football teams. And one suspects the point is to sell
magazines, not to provide useful information.
"But even people who go in for this sort of thing ought to be appalled
because the magazine's criteria for success are about standardized tests,
which measure what matters least. A school with rising, or unexpectedly
high, scores often has become little more than a glorified test-prep center,
sacrificing recess, project-based learning, the arts and a host of other
things that help kids to become enthusiastic and proficient learners. This
list not only confuses meaningful learning with high test scores; it
actively contributes to a culture where schools give up the former to attain
the latter so they can become--God help us--'winners.' "
Inner-city educator and author Deborah Meier said she has been drawn to what
she calls "the Consumer Reports model" as an alternative to the Newsweek and
U.S. News rankings. The Consumer Reports people, she said, "do occasionally
rate someone as 'best buy'--but mostly they just provide information on a
wide range of stuff. . . . Mostly, it's not comparative--at least overall.
We are judging apples vs. oranges. I'm interested in wagons that are fairly
small, with four-wheel drive and good gas mileage. Then I get a chance,
after all, to drive it before I buy it and look for little things that
matter to me--the size of the numbers on the mileage meter, the location of
certain interior lights . . . The ranking game, the constant urge to compare
everything--who's better, etc.--is a bad and injurious habit. I have three
kids and I truly can't think of any way of ranking them that makes
sense--and surely not over time!"
Educational psychologist and author Gerald W. Bracey gave a brief friendly
nod to Challenge Index. He said it "is more 'democratic' because it rewards
those who encourage participation, not those who score high. Obviously,
there are few schools in the country which can compete with Thomas Jefferson
on achievement measures."
But the heart of Bracey's argument: "I don't think we should be ranking the
schools at all. In part, I just think it's unhealthy. In part, both methods
have such a narrow focus. It's like ranking an athlete or a team on a single
measure. Yes, they are ranked, but it's a competitive arena. It's like
ranking the Key School in Indianapolis on test scores. The school is formed
around Howard Gardner's theories of multiple intelligences, and many of the
things they do each day have nothing to do with test scores."
Four other experts who more or less support the national effort to raise
school achievement measured by test scores saw some advantages, but also
drawbacks, in high school rankings. Mike Riley, the outgoing Bellevue,
Wash., school superintendent, who is soon to join a College Board effort to
raise low-income student achievement, said what he liked about the new U.S.
News list is "it tries to consider other factors. A truly comprehensive view
of schools would be very helpful, not only for the purposes of a rating
system but also for its value in expanding the way everyone--the public,
parents, educators--think about what school 'success' means."
Riley had a problem, however, with the U.S. News screening method. "If a
school meets expectations on state testing--but doesn't exceed expectations
on this measure--it fails to pass through the gate and the number of kids
who pass a nationally recognized test (AP) is never considered. I wonder if
they have this backwards. A national measure puts all schools on the same
playing field while state tests are wildly uneven in terms of their
difficulty levels and knowledge and skills measured."
Gerald N. Tirozzi, executive director of the National Association of
Secondary School Principals, said his organization "is not a proponent of
school rankings. We recognize that the schools included on both the U.S.
News and World Report and Newsweek Challenge Index lists are deserving of
the honors they have received. But how can we truly compare one school to
another when each state has its own set of standards? Until we have national
standards, at the very least in the areas of reading and math, rankings of
this type do not hold nearly as much weight as they could if all schools
were on a level playing field."
Mel Riddile, a former National High School Principal of the Year who now
runs T.C. Williams High School in Alexandria, objected to the top U.S. News
ranking for Jefferson. "Comparing schools that can improve test scores by
sorting applications does nothing for schools like T.J. because they know
they are good. They should be. They are the all-star team. U.S. News is
terribly naive to construct a rating that compares results and fails to take
into account inputs and processes. This rating, while it seeks to find the
schools doing the best, only identifies schools that have the best. This
ranking system is like comparing the American League All-Star team to that
of the Baltimore Orioles who have not had a winning season for a decade and
saying that the all-star team is statistically superior. This type of
ranking does a disservice to efforts to reform or reinvent American high
schools, because it says to school boards, communities, principals and
teachers in diverse, open enrollment schools, that you can never be good
unless you stack the deck. . . . The best schools are those that do the most
with the students, all the students, who walk through their doors, not the
best qualified applicants."
Dick Reed, a very active high school parent in Fairfax County, who has
studied the AP and IB programs, said the U.S. News rankings were "a good
list, though I have quibbles," mostly having to do with the complex format.
"It would be almost impossible for an interested person to do their own
comparative scoring. If all that's wanted are the top schools, it's fine. If
I wanted to get my school in this list somehow, I would have almost no idea
how to do so. That's not true with the Challenge Index. To get higher on the
Challenge Index, I simply press my school to encourage more students to take
IB or AP classes. Nothing more need be done--nothing more can be done. With
the new list, there's far more involved and almost none of those things can
be affected by an individual."
Reed endorsed, however, the idea behind both lists: "People are naturally
competitive. Harness that instinct and use it to raise the performance of a
school."
There are limits to the usefulness of this competitive instinct, of course.
To see it taken to its hilarious extremes, check out the high schools the
Wall Street Journal ranked last Friday as the most likely to get your child
into eight selective colleges, including Harvard and Princeton. They were
almost all very expensive private schools, or very selective public ones,
including Jefferson (roughly 36th out of 40 schools.) The accompanying
articles mentioned that these schools have counselors who are experts on
gaming the admissions process, but did not note another explanation for
their success: They likely have the largest percentage of parents who are
alumni of the targeted colleges, thus getting extra legacy points for their
kids.
I hope there will be many more discussions of what really makes a good high
school. As a U.S. News statement explained, the new list was inspired in
part by an article Rotherham and research colleague Sara Mead wrote saying
it was wrong for the Newsweek list to include large urban schools that had
strong AP or IB programs but low average scores on state tests and high
dropout rates. They let me respond on Rotherham's Web site,
educationsector.org. I said those schools most needed and deserved the
encouragement of recognition in a national list. The Challenge Index grew
from my five years of writing a book about Garfield High School in East Los
Angeles, where AP teachers had changed hundreds of lives despite being in a
very large urban school with poor state test scores because of the steady
influx of low-income Hispanic students. Garfield proved those students could
perform very well on AP exams if given extra time and encouragement. The
list was designed in part to show other such inner-city schools they could
do the same for many of their students if they followed Garfield's example.
Rotherham argued that Garfield's achievements were admirable, but until
inner-city educators found a way to raise state test scores and lower
dropout rates for all students, they did not deserve to be recognized on a
national list. I said those were diseases for which there was yet no cure,
and those schools should not be denied recognition because they have not
solved problems that remain insoluble.
There are some possible solutions, of course, on which both Rotherham and I
agree. We have on each of our lists a few inner-city high schools like YES
in Houston and Preuss in San Diego, which have shown what small charter
schools intensely focused on raising low-income student achievement can do.
My hope is that both the Newsweek and U.S. News lists will evolve toward our
shared goal -- improving education for the many American children who are
not getting the teaching they deserve. We are both searching for schools to
celebrate, but in different ways. The U.S. News list reminds me of a big
government operation, with satellite data and computers, tracking schools
from 200 miles up. Those of us working on the Challenge Index at Newsweek
and The Post are on the ground, peeking into schools, calling or faxing or
e-mailing principals, teachers and counselors, collecting their data,
discussing their initiatives and asking them for their reactions to what we
are doing.
Rotherham, Gazzerro and the fine journalists at U.S. News are going to
discover that when they don't put a high-achieving school on their list,
many people will inquire about that. Alumni of Benjamin Banneker Academic
High School in the District--which along with 10 states was unable to
provide data to U.S. News--are already complaining about their school going
unmentioned. Fairfax County, the largest school district in Rotherham's
state, Virginia, has all 25 high schools (not counting Jefferson) on the
Newsweek list of 1,350 schools, the top 5 percent in the United States, but
only five, including Jefferson, on the U.S. News list. I suspect he will be
hearing from those left out, all part of the learning process for us
school-rating scoundrels.
Join me for a live chat online today at 2 p.m. at washingtonpost.com.
Posts
No comments:
Post a Comment