\n
![\"fusion_starter_logo\"](\"http:\/\/impact.ucdsehd.net\/wp-content\/uploads\/2015\/11\/fusion_starter_logo.png\"){kind=logo}
<\/a>\nBOULDER, CO (October 30, 2015)<\/b> \u2013 For 15 years, journalists, advocates and policymakers have cited scores on international tests, such as the Program for International Student Assessment (PISA), to conclude that American student achievement \u201clags woefully behind\u201d other nations, threatening our future and suggesting an urgent need for education reform.<\/p>\n
A brief published today by the National Education Policy Center at the University of Colorado Boulder explores such policy analyses and claims around PISA as well as a second prominent international test, the Trends in International Mathematics and Science Study (TIMSS).<\/p>\n
In International Test Score Comparisons and Educational Policy: A Review of the Critiques, Stanford education professor Martin Carnoy focuses on four main critiques of analyses that use average PISA scores as a comparative measure of student achievement.<\/p>\n
The ranking is misleading, Carnoy asserts, because:<\/p>\n
- \n
- Students in different countries have different levels of family\u2014not just school\u2014academic resources;<\/li>\n
- The larger gains reported on the TIMSS, which is adjusted for different levels of family academic resources, raise questions about the validity of using PISA results for international comparisons.<\/li>\n
- PISA test score error terms\u2014the difference between measured achievement and actual achievement\u2014are considerably larger than the testing agencies acknowledge, making the country rankings unstable.<\/li>\n
- The Shanghai educational system is held up as a model for the rest of the world on the basis of data on a subset of students that is not representative of the Shanghai student population as a whole.<\/li>\n<\/ul>\n
Professor Carnoy also assesses the underlying social meaning and education policy value of international comparisons. First, he describes problems with claims that average national math scores are accurate predictors of future economic growth. Second, he explains that using scoring data in this manner has limited use for establishing education policy, due to the lack of causal inference analysis. This is the well-known \u201ccorrelation is not causation\u201d problem.<\/p>\n
Third, there is a conflict of interest arising from the Organization for Economic Cooperation and Development (which administers the PISA) and its member governments acting as a testing agency while simultaneously serving as data analyst and interpreter of results for policy purposes.<\/p>\n
Fourth, Carnoy questions the usefulness of nation-level test score comparisons with regard to countries such as the United States with such diverse and complex education systems. The differences between states in the U.S. are so large that employing state-level test results over time to examine the impact of education policies would be more useful and interesting.<\/p>\n
Despite such compelling critiques of international testing, Carnoy concludes that these tests will not go away, nor will they stop being wrongly applied to shape educational policy. However, he says, \u201cthere are changes that could be made to reduce misuse.\u201d He concludes with five policy recommendations, including reporting test results by family academic resource subgroups of students with different levels of resources.<\/p>\n
In a companion report<\/a> released today by the Economic Policy Institute (EPI), Carnoy and co-authors Emma Garc\u00eda and Tatiana Khavenson provide detailed analyses explaining how and why comparisons using data at the level of U.S. states are more useful than comparing the U.S. with other countries for understanding and improving student performance.<\/p>\n
Find International Test Score Comparisons and Educational Policy: A Review of the Critiques<\/i> by Martin Carnoy on the web at: http:\/\/nepc.colorado.edu\/publication\/international-test-scores<\/a>.<\/p>\n
\nClick Here To View the Press Release on the NEPC Website<\/a><\/p>\n
Reference Publication:<\/b>
\nInternational Test Score Comparisons and Educational Policy: A Review of the Critiques
\nContact:<\/b>
\nWilliam J. Mathis, (802) 383-0058, Email<\/a>
\nMartin Carnoy, (650) 725-1254, Email<\/a><\/p>\n
\nThe mission of the National Education Policy Center<\/b> is to produce and disseminate high-quality, peer-reviewed research to inform education policy discussions. We are guided by the belief that the democratic governance of public education is strengthened when policies are based on sound evidence. For more information on NEPC, please visit http:\/\/nepc.colorado.edu\/<\/a>.<\/p>\n