Contested Studies

Germany reached 13th place out of 57 countries tested in the results of the latest Program for International Student Assessment -- PISA -- study, officially published on Tuesday, Nov. 4.

But any joy over improvement in standing was damped when the test's creator at the Organization for Economic Cooperation and Development (OECD), Andreas Schleicher, last week told the press that the improvement was more likely due to changes in the way the test was structured, than to any actual improvement on the part of German students.

The discussion over Germany's school performance has been raging since 2000, when the country landed in the bottom third of the rankings of 32 countries, and continued through 2003, when Germany still rated 18th in a list of 40 countries.

Those rankings "certainly served to alert the public of just how different our perception of the system was compared with its stance in the real world," said Rainer Lehmann, a professor of education research at Berlin's Humboldt University.

Indeed, the country's mediocre ranking back in 2000 caught most Germans, who thought highly of their school system, off guard. The term PISA has since become a household word, reflecting the study's resonance in the media.

Debate over studies' relevance

The debate has raised questions about the general relevance of comparative educational testing, pitting those who see tests as a useful comparative tool against those who focus on the shortcomings of big, international studies.

Most experts agree that big international studies have an important place in education policy.

"The idea is that you have some natural variance between educational systems, and you try to capitalize upon this by looking at different influences," said Lehmann. "The goal is to learn from other systems."

Michael Becker-Mrotzeck, a professor in the department of language education at Cologne University, agreed.

"In principle, (these studies) tell us useful information about our school system," he said. "For instance, from the first and second PISA studies, Germans learned that in no other country does social status of the parents have such a big effect on the academic success of their children."

In addition, the earlier studies showed weakness in certain basic competencies, like reading, math and science. As a result, he said, schools have shifted focus -- with visible results.

In November an international test of elementary school competency showed that reading had "clearly improved."

"Our education system reacted, quickly," to the deficits uncovered by PISA, Becker-Mrotzeck said. This led to the development of a spate of learning materials, and a host of creative concepts, such as reading nights and author-readings in schools.

Germany coming out of anti-testing era

Unlike in the UK and the US, where educational testing has long been a buzzword and where standardized tests are routinely used to monitoring how much information is being passed from teachers to students, Germany is fairly new to the game.

After a spate of general tests in the early 70s showed similar weaknesses to those rediscovered by PISA in 2001, the country underwent an anti-testing backlash that lasted a good 20 years.

Only recently have policymakers reinstated mass comparative testing in schools. Today, German schoolchildren face four "general" exams over the course of their 13 year school career -- in 3rd, 8th, 10th and at the end of the 12th grade, Becker-Mrotzeck explained.

"It is different than in the US or England, where they might have such tests twice a year," said Becker-Mrozeck. "I don't think at that level Germany is in danger of doing what they call in America, 'teaching to the test.'"

Still, learning doesn't take place via testing, said Renate Hendricks, a Social Democrat parliamentarian for the state of North-Rhine Westphalia, who specializes in education policy.

"I think you can test what a person can do, but learning comes through motivation, curiosity, interest, and an emotional relationship to the teacher," among other things, Hendricks said.

Hendricks did acknowledge that comparative studies are useful for giving feedback aimed at improving schools. "But if politicians are incapable of using the information they get to make positive change, then no, these studies are not useful," she said.

Disparities in German system

She added that when the figures are broken down by social background, immense disparities in the German system are still evident in the newest PISA results.

"Now it is clear we have to do something," she said. "These politicians have to take off their rose-colored glasses. Or black glasses, or whatever they are wearing."

Meanwhile, still other critics say the whole structure of PISA -- indeed, the idea of widespread comparative international testing itself -- is flawed. The PISA study was also designed with "unrealistic goals," according to Joachim Wuttke, a Munich-based physicist, statistician and math teacher who has written two articles critiquing the methodology behind PISA and similar tests.

Tests weighted against Germany, says expert

"The ranking of states on a one-dimensional scale is not serious," he said. "You can construct tests that give different results, and this one is made in a way that shows Germany is mediocre."

Various factors can affect outcome. The type of school system a country has is just one potential factor, he said, adding that the spectrum of students tested also differed from country to country.

"Germany took great pains to get a representative sample of students, testing even in schools for special-needs students," he said. "Meanwhile, Finland excluded students with dyslexia from the test. You can't do an international comparison of reading difficulties when you exclude some students."

Ultimately, Wuttke said tests like PISA "don't test the system, they test the population." He pointed out that in highest-ranking Finland, there are almost no immigrants.

"If you compared results of all native students only, then Finland is no longer at the top of the list," he said.

Germany shocked out of stupor

One thing the PISA proponents seem to agree on is that the results from 2001 shocked the country into a back-to-basics frenzy.

"I see the main advantage of this test as having reoriented schools toward cognitive growth," said Humboldt University's Lehmann. "Every year you have a new set of educational goals seems to be defined --family education, sexual education, peace education, gender education ... but the core business seems to be forgotten."

But even there, Wuttke sees PISA as more of a lost opportunity than a gain.

"Reading, math and science are important subjects, but it shouldn't come at the expense of other things like sports and music," he said. "I happen to think they are just as important."