Tag Archives: testing

Teaching time in the U.S.

In The Mismeasure of Teaching Time, Sam Abrams exposes the myth that teachers in the United States spend nearly twice as much time leading classes as teachers in many other OECD countries. Abrams, Director of the National Center for the Study of Privatization in Education at Teachers College, Columbia University, details basic contradictions in the U.S. figures reported by the OECD and repeated by many journalists and scholars. His analysis suggests that U.S. teachers still do spend more time teaching than their counterparts in other OECD countries, but only about 15 percent more. This is still an important difference, but, as Abrams argues, one that has overshadowed more significant differences: in particular, teacher pay and the structure of the school day. Soon after the study was published last week by the Center for Benefit-Cost Studies of Education (CBCSE), we had a chance to ask Abrams when he first noticed the problems with U.S. figures, how he determined what was going on, and what the implications are for educators and the general public (see also recent EdWeek coverage). We share his response below:  

After repeatedly seeing this misinformation about teaching hours in books and articles, I wrote to the OECD in January 2012 to inform them that the U.S. hours were way off and provided as evidence terms of teacher contracts from several major school districts.  I moreover explained that I had been the scheduler of a public high school in New York City for seven years as well as a teacher for many more. The people I contacted at the OECD conceded the U.S. hours appeared inflated and relayed this information to the U.S. representative to the OECD, who, in turn, I was told, stood by the numbers.  I decided at that point to save my argument for a book I was writing on educational privatization.  But as that book was taking longer than expected and as this myth was getting repeated on a regular basis in op-ed pieces, think tank studies, and books, I decided in October of 2014, after compiling more evidence, to address the problem directly with the National Center for Educational Statistics (NCES), the source of the U.S. figures.

Tom Snyder, the NCES Program Director for Annual Reports and Information, could not have been more helpful. When presented with my case, as I wrote in my study, Snyder undertook with his staff a review of their data and a week later reported that the United States had indeed been inadvertently overstating teaching time. Snyder’s openness and cooperation was a lesson to me that researchers can get a lot more assistance from authorities than I had thought.

I should add that if I had not been a teacher, I would not have been in much of a position to question the data.  This implicitly points to the divide between policy and practice. As a teacher, I had never heard of Education at a Glance and I didn’t read many books about education policy.  I read books about history and economics, as I was a teacher of history and economics, and I struggled to keep up with reading and grading my students’ papers.  It was only after I became a researcher and began to work on my book on educational privatization in 2008 that I started reading a lot of books about education policy.  With my background as a teacher, I could quickly see that some policy analysis, like this argument about teaching time, didn’t hold up.

The salient implication of this finding is that we’ve been tilting at windmills. Teaching time is a phantom problem.  U.S. teachers do teach more than their OECD counterparts but, as I explain in my study, only marginally more.  The real and telling differences between teaching in the United States and other OECD nations concern relative pay and the structure of the school day.  These problems have been obscured by the difference in teaching time because the alleged difference has been so dramatic.  For journalists and scholars, that dramatic difference has been impossible to ignore.  And they’ve understandably focused on it at the expense of these two other real and telling differences.

As I noted in my study, U.S. primary teachers, according to Education at a Glance, earn 67 percent as much as their college classmates while their OECD counterparts earn 85 percent; U.S. lower-secondary teachers earn 68 percent compared to 88 percent for their OECD counterparts; and U.S. upper-secondary teachers earn 70 percent in contrast to 92 percent for their OECD counterparts.  The data on pay appear quite reliable, as I explain in my study, because the method of collecting data on pay differs substantially from the method of collecting data on teaching time.

In absolute terms, U.S. teachers may make as much as their OECD counterparts but not in relative terms, because in other OECD nations, on average, bankers, lawyers, doctors, engineers, and management consultants make much less.  So, this is a social contract issue.  It isn’t so much that U.S. teachers don’t earn much. It’s that their college classmates who went into banking, law, medicine, engineering, and consulting make a lot more.  Teachers accordingly get priced out.  It’s thus hard to attract people to the profession and hard to retain them if they come aboard.  Fixing this problem constitutes a steep challenge.  It would necessitate raising not only teacher pay but also marginal income tax rates as well as tax rates on long-term capital gains.  But we at least need to look at this problem with our eyes wide open.

What’s not hard to fix is the structure of the school day.  In this regard, as I explain in my study, U.S. practice differs significantly from that of many other OECD nations. The architects of No Child Left Behind (NCLB) and Race to the Top (RTTT) wanted to close the achievement gap by identifying academic deficiencies through regular testing and by holding school administrators and teachers accountable for their students’ results. The goal of closing the achievement gap is clearly noble. But the high-stakes testing that defines NCLB and RTTT has had perverse consequences. In particular, it has reduced, as I documented in my study, time for play, art, music, and drama because school administrators have felt great pressure to pack more academic instruction and test prep into the school day in order to boost test results.

School administrators have had little choice.  Test results are ultimately relative.  If school administrators in one district reduce time for play, art, music, and drama to make more time for academic instruction and test prep, then school administrators in neighboring districts are hard pressed not to do the same.  We have a stadium effect.  If spectators in the first row at a basketball game rise to see a big play, then spectators in the next row must do the same. In a heartbeat, everyone in the arena is on their feet. The same holds for test prep and schools.  The result is a narrowed curriculum, intense pressure on students, teachers, and principals alike, and a tight day, lacking the breaks between classes necessary for students, teachers, and principals alike to regroup, reflect, and get some fresh air.

There has long been an assembly-line pace to the school day in the United States, with short breaks between classes and brief lunch periods. Raymond Callahan made that clear more than fifty years ago in his incisive book Education and the Cult of Efficiency, published in 1962. But with the high-stakes testing introduced by NCLB and RTTT, the pace has intensified.

As a coach for several years with the Ice Hockey in Harlem program in Central Park, I’ve come to learn that kids of all ages, from five to seventeen, crave and need play. We can’t get the kids off the ice at the end of practice. Kids need to improvise with their bodies and experience the joy of playing with their peers. At a certain point, as I write in my study, more academic time becomes counterproductive. Leaders of such major companies as SAS and Google, in this light, have understood that more work at the expense of relaxation and recreation likewise becomes counterproductive and have accordingly encouraged their employees to take breaks and designed their offices with relaxation and recreation in mind.

We could fix this, as I explain in my study, by getting rid of high-stakes testing.  We have had the National Assessment of Educational Progress (NAEP) since 1969 to test samples of students in fourth, eighth, and twelfth grades.  We need little more than that. If we could move on from high-stakes testing, then our schools would be a lot more like schools in many other OECD nations, and the lives of U.S. teachers would be a lot more manageable.

Quality Assurance in Chile (Different context, same issues?)

Chile’s education system has embraced school choice and market-based reforms like no other system in the world, but on my visit to Santiago I heard many of the same questions about what to do for failing schools and how to assure educational quality that I have heard in the US and in countries like Norway, the Netherlands, and Singapore. These questions focus on what to test and assess, how often to assess, and how to publish the results, what to do about persistently failing schools (or, put another way, the persistent failures of policies to help schools improve); and how to ensure that municipalities, districts, charter operators and other public and private “school owners” comply with legal regulations and reach acceptable standards of educational quality.

Debating Testing & Assessment

Chile’s current national test, SIMCE, developed in the 1980’s and 90’s in response to concerns of a number of different groups. Some saw SIMCE as a way to check that public resources were being used appropriately, in a context of decentralization of schools’ administration. Advocates for school choice, in particular, emphasized that to have a “free” marketplace, parents and students needed information on school performance to make decisions. Many educators also saw the development of the test as the way to provide teachers and schools with information so they could make improvements. (For a history of SIMCE, see Meckes & Carrasco Two decades of SIMCE: an overview of the National Assessment System in Chile”)

Enduring differences of opinion around these issues, however, have fueled a cycle of reviews, commissions, developments and adjustments in SIMCE and the Chilean approach to quality assurance. These reviews have included a Commission in 2003 that addressed questions like whether the tests were measuring the right things, whether the tests should be given every year, and whether they should be sample-based (like NAEP in the US or national testing in Finland) or census-based (given to all students at a particular level as are the tests required by the No Child Left Behind Act in the US).

A new Commission, established just this year by the Bachelet government, is revisiting many of these same questions. In this case, however, the review is fueled by some of the same concerns about the amount of testing that many are also raising in the United States. I was surprised to find, however, that even Chile does not require the same kind of annual testing in reading and mathematics from 3rd-8th grade that my children have experienced in New York. Currently, Chilean students are assessed in 2nd grade in reading, 4th grade in math and language, and every two years in science and history. A new test for 6th grade (at the end of what the Chilean’s call “basic education” has already been introduced, but with the concerns about too much testing the current government may stop it. This latest Commission is expected to share their findings with the government shortly. The Commission’s report will again consider how often national testing should take place, in what grades and subjects, what the results should be used for, and how public the results should be.

Making test results public & the “discovery” of persistently failing schools

Whether and how to make test results public remains one of the most contentious issues in Chile (as well as in many other countries including Norway, Denmark, and the Netherlands).  In the early 1990’s, SIMCE results were not published – even though the law governing SIMCE actually required it. However, demands to use the data for accountability and other purposes led to the publication of the results beginning in 1995. The data was used at that time in a number of ways including to provide rewards to some groups of schools and teachers and to identify schools that were not doing well.

In Chile the publication of tests results made visible the problem that some schools continued to get poor results year after year. Interestingly, at about the same time in the Netherlands, demands by newspapers to make public the results of school inspections revealed a similar pattern of repeatedly failing schools. In both countries, the recognition of persistently failing schools (which could also be seen as a problem of persistently failing policies) helped to highlight that information alone was not sufficient to lead parents and students to choose more successful schools, nor to equip teachers and schools to make improvements.   In response both Chile and the Netherlands eventually developed approaches to failing schools that included many elements reminiscent of the provisions for failing schools under NCLB in the US. These included the development of expectations for failing schools to improve performance, technical assistance to help them do so, and delivery to parents of information about the schools’ performance and their options to take their children elsewhere.

Ensuring Quality Assurance

In Chile, as in many other countries, quality assurance in general, has also become a focus for considerable policymaking over the past fifteen years. Following the “penguin riots” of 2006 (named for the black and white school uniforms worn by the protesting students), improving quality assurance was the main issue around which those in different places on the political spectrum agreed. Those agreements included a variety of changes in educational regulations that are still being put into place. Among these are the establishment of two new agencies (an Education Superintendency and an Education Quality Assurance Agency) so that the Ministry of Education does not oversee both school improvement and the assessment of the effectiveness improvement efforts (even Finland is in the process of creating new government agencies to deal with the division of responsibilities around quality assurance).

In Chile, the Education Superintendency will be responsible for ensuring compliance with legal regulations (Norway created a similar agency and “legal inspection” after years of having almost no way of knowing whether municipalities were complying with event the most basic educational requirements).

Meanwhile, the tasks of the Chilean Education Quality Assurance Agency will include evaluating schools using tests like SIMCE; assessing the extent to which schools reach established learning standards; and assessing schools on other quality indicators including academic selfesteem, school climate, and healthy life style. The Agency also will be responsible for using these assessments to classify schools into performance levels (High, Middle, Medium Low, and Insufficient). While the Chilean classification will draw on a variety of data, over 60% of the score will be based on students’ performance on the learning standards (similar to the emphasis of New York City’s Progress Report on test scores, up until just announced changes)

The results of this assessment process will determine how often Chilean schools will be inspected; guide the development of plans or contracts for improvement for low-performing schools; offer information parents’ can use in school selection; and, ultimately, could be used to withdraw the Ministry’s public recognition of the school (the Dutch created their own “risk-based” process to identify schools they felt needed additional oversight and support). Notably, however, with continuing debates and disagreements around the extent of testing and the process of closing failing schools in Chile, plans to classify schools beginning this year may not be fully carried out.

Thomas Hatch

 

Educational Testing in China, France, and Singapore

Reports in news publications focusing on Singapore, China, and France, show that prominent educational researchers and politicians are raising questions about educational testing, and even introducing reforms that drastically alter the testing landscape.  * Links are embedded as hyperlinks below.

China

According to a new document released by China’s Ministry of Education, the efforts of various education reform efforts over the past few decades has had no impact on China’s “tendency to evaluate education quality based simply on student test scores and school admissions rates.” In order to address the problems brought about by high-stakes testing, the Ministry of Education is taking steps to implement more serious reforms to change how schools are evaluated. Their new evaluation framework, which is called “Green Evaluation,” will end the use of test scores as the only measure of education quality and focus on five new indicators: moral development, academic development, psychological and physical health, development of interests and unique talents, and academic burdens.

France

Ed Alcock for The New York Times

Ed Alcock for The New York Times

As The New York Times recently reported in their article, “Rite of Passage for French Students Receives Poor Grade,” criticism of the baccalauréat in France is building. The weeklong national test is “better known simply as the ‘bac,’ the exhaustive finishing exam that has racked the nerves of France’s students since the time of Napoleon.” It is the “sole element considered in the awarding of high school diplomas,” but many critics such as Emmanuel Davidenkoff, the editor of the education magazine L’Étudiant, say the test does not evaluate the most relevant of students’ capabilities. “In France,” he says, “we evaluate essentially only hard knowledge, not all abilities.” The center-left government wants to reform the national school system, but is focused on primary schools. Education Minister Vincent Peillon believes that the tests will evolve as the country continues to reflect on it.

Singapore

Dr. Linda Darling-Hammond

Dr. Linda Darling-Hammond

The Sunday Times reported on a recent visit from Professor Linda Darling-Hammond of Stanford University, who was recently appointed a Visiting Professor to the National Institute of Education in Singapore. Darling-Hammond shared her impressions of the Primary School Leaving Exam (PSLE) on a visit to the country earlier this month. On the one hand, the examination system in general earned her praise as it places an emphasis on the testing of higher order thinking skills such as application, analysis, synthesis and evaluation. On the other hand, Darling-Hammond has reservations on how the results of a examinations taken at a particular stage (12 year olds) are used to determine the academic future of the children. She is of the opinion that a broader range of modes of assessment be used, including, for example, interviews and portfolios. While the latter approach is “less tidy … more time consuming,” positive results could be yielded from such a broader approach to testing.

South Korea

Controversy over abolition of NEAT English test among high school teachers

by DoKyoung Lee, Kookmin Ilbo (June 21, 2013)

*link in Korean

As part of a new university entrance system, NEAT (National English Ability Test) faces abolition even before its introduction as errors have been revealed in the electronic data processing system. Invested with 30 billion won (US$ 26 million), NEAT was designed to aim for practical and effective English education, an escape from traditional grammar drill training, and will replace the existing English exam starting in 2015; however, this date has been pushed back 2019 due unprepared school teachers and students, and concerns about the possibility of increasing private education expenses. While there are conflicting opinions on the issue among the schoolteachers who prepare students for this exam, the Ministry of Education has avoided further comment after mentioning that they will indicate their plan for NEAT when they make an announcement for new policies of revised university entrance system in coming August.

For more information:

Homegrown English tests in trouble

India

CBSE introduces Contemporary Comprehensive Education

The Times of India (June 6, 2013)

20453381.cmsThe Continuous Comprehensive Evaluation, introduced by the Central Board of Secondary Education (CBSE), was designed with the intention of reducing stress on students and evaluating them “in a more rounded way”; however, the system is being criticized for adding a layer of bureaucracy that reveals a distrust for schools and teachers, adding tremendous paperwork. Some say the process can be streamlined, and once members receive training the new process seems like less of a burden. Students seem to be heavily in favor of the system, which has turned the academic year into two semesters and, as one student said, “made even boring topics interesting through various projects.” Parents are also supportive, saying, “Students are not cramming notes like we had to. Their interpersonal and co-scholastic skills are better and they become better leaders.”

For more information:

A study on implementation of the Contemporary Comprehensive evaluation in upper primary schools of Kerala

Scan of Ed News: Testing

Since 1995, children have been required to sit literacy and numeracy tests in their last year of primary school. Photograph: Martin Godwin for the Guardian

Since 1995, children have been required to sit literacy and numeracy tests in their last year of primary school. Photograph: Martin Godwin for the Guardian

Over the past month, a number of reports indicate a variety of concerns about testing around the world.  For example, Israel’s new Education Minister, Shai Pironhas decided to introduce reforms that would de-emphasize testing in order to “promote learning.” Similarly, China is taking small steps towards allowing educators to have input in test design (as opposed to government only). In contrast, the British government is acting in opposition to educators and parents to fight off an unprecedented alliance of hundreds of students, schools, local councils, and teaching unions, who brought a legal challenge over last year’s GCSE English exam grades. UK teachers are also protesting primary literacy exams, which they say leave little time for art, music, and books, and make children feel like failures. Chile has announced sweeping changes to the country’s university entrance exam, which has received criticism for flaws and bias; however, the concern in this case was not raised by educators and parents, but by Pearson, a company that describes itself as a leading provider of test development, processing and scoring services to educational institutions, corporations and professional bodies around the world. Pearson’s analysis revealed significant flaws and bias in the design of the exam.

In Singapore, surveys recently revealed that many educators and parents feel that students experience too much testing and a report on a recent visit by Dr. Dennis Shirley highlighted his suggestion that  the task of student assessment be handed over to the teachers, so that they can design their own modes of testing.  While the Singapore government has proposed several initiatives aimed at strengthening efforts to help every student succeed, none yet include substantial modifications to testing. While it might seem that the decision made by five schools in the town of Alesund, Norway, to change the date of the midterm exams so that students could attend a Justin Bieber concert in Olso, was an effort to modify testing to meet the needs of the students, it was also one for which officials saw no alternative. As one principal explained, they expected Mr. Bieber’s show would lead to sparse classroom attendance. “We considered that this was a battle that we could not win this time,” he said.

Ireland

Ruairí Quinn calls for “inclusive debate” on education in Ireland

Donal Walsh, SchoolDays.ie (December 18, 2012)

Ireland: Google Images

In response to the recently released TIMSS and PIRLS scores, Ruairí Quinn, Ireland’s Education Minister, wants to reassess the amount of time students spend studying each subject. While Irish students performed at an above average level, the students of Northern Ireland achieved better results in mathematics. Quinn believes that the solution is to ensure a higher standard of knowledge amongst primary school teachers, and to increase the amount of time the students spend studying math and science. He said: “I have asked the National Council for Curriculum and Assessment to review the recommended time allocations for all subjects in the primary school.”

For more information:

What will happen in education in 2013?