Tag Archives: Assessment

Curriculum and assessment in African countries

This week, we conducted a scan of education news published in the past month from countries in Africa. These articles highlight efforts to increase access and quality of education through the implementation of national curricula and assessments and through initiatives focused on teacher recruitment, salaries, and training.

South Sudan recently launched its first national curriculum. Gurtong.net quoted Jonathan Veitch, UNICEF Country Representative, as saying…

“For now the curriculum is complete, textbooks must be designed and published, teachers need to be trained to implement this curriculum, and school managers, inspectors and supervisors require training to provide the required management and oversight….”

Reports from South Africa (recently ranked “almost dead last in math and science” on this year’s World Economic Forum Global Competitiveness report, as News24 noted) show that even with curriculum and assessments in place, educators need to see their worth in order for them to be useful for instruction. The Daily Maverick recently reported that both the teachers’ union and the Department of Basic Education agree that the current national assessments are not effective, and some teachers’ unions have already promised to “opt-out” of administering the current assessments.

Tensions between teachers and the national government in Kenya also reflect something of a “Catch-22.” In a recent World Bank report, concern was expressed that the quality of education in the country was alarmingly inadequate. On the one hand, many critics of the government, including many teachers, argue that the reasons include the government’s failure to comply with a court order to increase teacher salaries by 50-60%. In response, teachers are engaged in a formal, long-term strike to protest inadequate salary, which they would like to see rise to the levels of other professions. On the other hand, supporters of the government suggest that the teacher strikes are contributing to the problems because they result in irregular access to classrooms for most students. In a stalemate, the Education Ministry ordered schools to close as of September 21st.

According to All Africa, Cameroon’s Education Ministry is taking steps to try to “professionalize” teaching by bringing in Dutch consultants to help refine teacher training, as well as curriculum. According to Roeland Monasch, the CEO of the Dutch NGO Aflatoun, the solution is simple: “He assured that once teachers are well trained, students will do well in class.”

Deirdre Faughey

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Vietnam: Can one assessment meet the needs of all stakeholders?

20150707071925-edu-exam-questionVietnamese students surprised the world recently when it was revealed that they outperformed many other advanced countries on the PISA exam. According to a recent BBC article, “the country’s 15-year-olds scored higher in reading, maths and science than many developed countries, including the United States and the United Kingdom.”

News of this achievement has received a great deal of media attention, however in an effort to learn more about recent developments in the Vietnamese education system today we reached out to Duy Pham, former Deputy Director of the Center of Educational Measurement, of the Institute for Education Quality Assurance, at Vietnam National University, and curent doctoral student at the University of Massachusetts Amherst. Pham explained that while the PISA exam has caught the attention of an international audience, the people of Vietnam have been wrapped up in a dynamic debate around high school graduation exams and university entrance exams.

This year, the Vietnam Ministry of Education combined the high school graduation exam and the university entrance exam into one 4-day-long exam. This move came in response to concerns about student well-being and assessment validity. Many felt that students and families were suffering under the pressure of two separate exams. Also, private universities, which admit students who do not get selected for prestigious spots in public universities, felt that the old exam system was too challenging and resulted in the exclusion of too many students. These universities wanted to be able to admit a greater number of students but found that these students were not able to meet the high bar set by the old entrance exams. As Pham explained, “This year the pressure comes from many stakeholders, saying the university system blames the previous entrance exam of not being able to classify students in a way that allows them to select the right students.”

As the new exam was administered in the first week of July, there is no consensus on the new process. While the Ministry of Education has expressed satisfaction with the new system, educators, policymakers, and researchers are concerned that the new exams might be too difficult for the purposes of high school graduation, yet too easy for the purposes of university entrance. The question is how to find one assessment that meets the needs of all students and institutions.

Also, the question of pressure and fairness remains. Students can only take the new exam in one of the approximately 30 testing centers. These testing centers are located in big cities, which means that students from mountainous and rural areas need to travel with a parent or guardian and find accommodations for the duration of the exam. Under the old system, students could at least take the high school graduation exam in their own school settings.

Deirdre Faughey

For more information on this issue:

National exam for university, high school satisfies students – News VietNamNet http://buff.ly/1fQRVdu

One million students sit for national exams – News VietNamNet http://buff.ly/1fQRY9k

Volunteers swing into action for season of exams – News VietNamNet http://buff.ly/1fQS6FI

Vietnam school students and the exam of life in pictures http://buff.ly/1fQS8xe

Flashback: Volunteers devote themselves to helping Vietnam’s national exam contestants http://buff.ly/1OkqFiO

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Growth in national assessments?

With the implementation of new state tests (PARCC and Smarter Balanced) connected to the Common Core in the US, it has been hard to avoid concerns about the quality and extent of testing in the US in recent weeks (including in Delaware, Minnesota, Ohio, and California among others). At a recent seminar series from the Laboratory of International Assessment Studies, however, I also heard reports about the marked growth in the use of national assessments in many other countries. Although the new state tests in the US provide data on the performance of individual students and schools, the 2015 EFA Global Monitoring Report to be released in April, will include data on the number of national learning assessments (designed to provide information on system performance) conducted worldwide since 1990. As initially reported in “Improving, not over-hauling learning assessments post-2015,” that report will show that the number of countries administering national assessments has doubled in the last twenty-five years:

Before 2000, national tests were conducted in:

  • 49% of developed countries
  • 34% of developing countries
  • 6% of countries in transition

Between 2000-2013, national tests were conducted in:

  • 82% of developed countries
  • 65% of developing countries
  • 78% of countries in transition

The FHI 360 Education Policy and Data Center (EPDC) also released results of the National Learning Assessment Mapping Project (N-LAMP). As described in a blog post on the key findings, the project reviewed data on standardized exams and assessments administered at the national level from primary to upper secondary education in a sample of 125 countries from 6 regions of the world. They identified 403 national-level learning assessments from 105 countries. The majority of the assessments (55%) were low-stakes national large-scale student assessments (NLSA’s sample-based assessments used to monitor the performance of education systems like those documented in the Global Monitoring report). Most of the remaining assessments were high stakes exams (mandatory assessments required for completing a given level of schooling or gaining admission to the next level). Not surprisingly, the project found that almost all of the assessments focused on “Literacy & Communication” and “Numeracy & Maths,” with more than half also addressing “Science & Technology.” Nonetheless, other domains were represented with 73 assessments addressing “Social & Emotional” domains, 33 addressing “Physical Well-Being” and 11 addressing “Learning Approaches & Cognition.”

Also worth noting for those interested in international assessments, Teachers College Record has a new theme issue focusing on PISA, “Moving Beyond Country Rankings in International Assessments: The Case of PISA.”

Thomas Hatch

News scan: Germany and Ireland

This week, a scan of the news coming from Europe led us to put several links on Twitter; however, over the past year we’ve noticed more than one report on related topics. Here is a brief description of news coming out of Germany and Ireland. Next week, we will take a closer look at reports coming out of Central and South American countries.

Germany

According to a new study, Germany will not be able to meet ambitious education goals the country set for itself in 2008. Angela Merkel aimed to cut the dropout rate from 8% to 4%, but as of 2013 the rate stood at 5.7%. The German government is also struggling to reduce the number of young people (aged 20-29) who were without any professional qualification.  Interestingly, another report pointed out that there has been an ongoing Twitter debate (in German) about the country’s educational system, sparked by one girl who tweeted, “I am almost 18 and have no idea about taxes, rent or insurance. But, I can analyze a poem. In 4 languages.” The debate is raging over the purpose of an education and whether or not schools should prepare students for “life.”

 

Ireland

Teachers are protesting in Ireland because they disagree with government reforms that aim to move student evaluations away from standardized testing and towards a performance-based model, which would allow portfolios and other options. Teachers are concerned that the new assessments will force teachers to judge their own students, rather than advocate for them. They also object to the amount of time teachers will need to spend on the new assessments. Pasi Sahlsburg responded to the teachers’ plan to strike by saying that teachers need to take on more complex roles in order to boost the profession. In addition to seeing themselves differently, teachers need to see the students differently–and that’s what the alternative assessment model is all about. According to Sahlsburg, the situation in Ireland is “unique globally in many ways. Internationally it is more common that teachers are the ones that insist more freedom and autonomy in assessing and grading their students rather than the other way round.” In this case, an additional issue might be that teachers are wary of new, complex practices that they don’t have the capacity to carry out—practices that might seem unnecessary, particularly after Irish students just achieved test results that surpassed those the country set for the year 2020.

Deirdre Faughey

Assessment in Finland: Steering, Seeing, and Selection

In Finland, the notable lack of tests for accountability purposes receives considerable attention. In fact, when we talk to teachers, administrators, and policymakers here, the question “how do you know how well things are going in your …. (classroom, school, municipality…)?” elicits quizzical stares. It’s a question that doesn’t make much sense when the initial assumption is that things are going well. However, our discussions here over the last three weeks have highlighted a few other interesting aspects of the uses of assessment here.

Assessment for steering not accounting

The word “accountability” has been traced back to ancient “account-giving” and record-keeping practices, tracking how funds have been spent and ensuring those funds have been spent as intended. Correspondingly, in places like the US, tests have been used to hold teachers, school leaders, and schools “accountable” for their actions and to see if they have done what they are supposed to do. But rather than using assessments to look back to see what was done, in many ways, educators and system leaders in Finland use assessment to look forward and to see if people, classes and schools are headed in the right direction. Such an approach doesn’t require data on every single aspect of student, teacher, or school performance, but it does require paying attention to ensure that no one gets too far off course. It means a focus on looking for outliers and listening for signs of trouble, not checking on each individual or making sure everything is done a certain way or in a certain timeline. But such an approach also requires mechanisms (like the curriculum renewal process as we will argue in a later post) to support shared understanding of the goals and expected outcomes of the whole system and a wide range of supports to make sure that everyone can get where they are going. Of course, it also helps if the whole system seems to be moving in the right direction already.

Assessment by walking around

Given the focus on this kind of “steering” approach, questions about the data used to make decisions from an American seem odd. While we have only spoken to a small group of teachers and school leaders here, invariably, those we’ve met have explained that learning whether a class or a school is on the right path can be accomplished by regularly “walking around” (while our Finnish colleagues did not refer to it directly, a similar concept—management by wandering around—has been part of the literature in business for some time). That means getting around the classroom and the school; talking to students, teachers, staff, and parents; listening to needs for support; and being alert to any signs of trouble. Concerns that arise about particular classes, schools, programs, or practices (especially when they come from more than one source) can then trigger “a talk” with those involved and some further investigation. (Even at the national level, a policymaker we talked to said that they don’t need a lot of data to tell them that many Finnish teachers are not using the assessment criteria that are in the core curriculum because regular meetings with teachers make that clear all the time…)

Despite the benefits, however, such a personal approach leaves unspecified the basis for many important decisions. In fact, when we asked teachers how school leaders know what they are doing or how well they are doing, many weren’t sure. Similarly, school leaders often couldn’t tell us how their supervisors (municipal administrators) could determine whether or not they were effective leaders. This lack of clarity may become more problematic as at least some municipalities in Finland have begun piloting some ways of using bonuses and salary increments to reward some teachers. While it is not widely discussed, those we talked to in at least two different groups of schools reported that their school leaders could decide to give them small bonuses if the leader felt that they did a particularly good job with their students or were particularly engaged in professional activities like research or professional development. While teachers could make their own case and often came to mutual agreements, leaders and municipal administrators, not teachers, have the last word.

Assessment for screening, sampling and selection

Even with a focus on assessment “by walking around,” however, educators in Finland do make use of a variety of tests and assessments. In contrast to Norway (where students do not get any written marks and there is comparatively limited testing until 8th grade), teachers in Finnish primary schools regularly use assessments of their own design as well as tests and quizzes from the textbooks; students get a report card at the end of the year; and in some cases, high scoring students may be singled out for recognition and rewards (something that the Norwegians would find shocking). Finnish teachers use an array of diagnostic and screening tests extensively in the early grades in Finland to make sure that no students are falling behind, particularly in reading. For example in one municipality, primary school special education teachers administer a screening test in reading comprehension to all students at the end of 2nd and 4th grade across all schools (and many administer it at the end of every year). That information, however, is not used at the school or municipal level to “check” on who is and isn’t performing well, rather, it’s used to identify those students who will need extra help moving forward.

As many have reported, the National Board of Education in Finland also regularly gives tests to samples of students and schools that are used to look at national and regional performance in key subjects like Finnish and mathematics. While the National Board does not use that information for ranking (and can’t because all students and schools are not assessed), they do share school level information with the schools that participate and municipal level data with the municipalities involved. In addition, the National Board makes these sample assessments widely available for free so that any teacher, school, or municipality that wants to administer these tests can do so. As a consequence, even without national testing, Finnish schools and municipalities have government-paid for tools that are linked directly to the core curriculum that they can use to benchmark their performance against regional and national samples.

Despite this diagnostic emphasis, tests and assessments in Finland do have important consequences, however, even if they are not used to hold teachers and schools accountable directly. In particular, despite the emphasis on equity, in some municipalities, students can express a preference to attend a particular school and a students’ final exams and final grades at the end of basic education (9th grade) can have an influence on whether or not they get into their top choice upper secondary school. Thus, in some of the largest municipalities in particular, students with the highest grade point averages are likely to get into their first choice schools, while students with lower grade point averages may have to opt for less selective schools.

Furthermore, at the end of upper secondary school, students have to pass matriculation exams in several subjects, and their scores on those exams (in combination with the Universities’ own entrance exams) determine whether students can go on to university and which institutions and programs they can get into. In fact, the results of the matriculation exams are made public; and newspapers report on the highest performing students and rank the schools according to their students’ average scores (see “Lukiovertailu – Etelä-Tapiolan lukio Espoosta kärjessä” roughly: “Comparison of High Schools – South Tapiola in Espoo at the forefront of high schools”). In addition, even more information on the performance of vocational schools is made publicly available—including the numbers of graduates and the average time to completion—and that information is used by the government in decisions about funding.

As a consequence of the school choice options available and the selection practices of students, Sonja Kosunen and colleagues have argued that there is a kind of implicit tracking within the Finnish system that may have an impact on the equitable distribution of learning opportunities. (Nonetheless, as Jennifer Von Reis Saari has pointed out, in contrast to most countries like the US and Sweden the Finnish system is highly “permeable,” so that even students who choose a vocational track in high school can still end up studying advanced subjects and can still gain entrance to University programs.)

In the end, what we’ve learned makes it clear that teachers, school leaders, and policymakers in Finland have access to a robust set of assessments that are supported by a long tradition of work on assessment at institutions like the Centre for Educational Assessment at the University of Helsinki, the Centre of Learning Research at the University of Turku, and the Niilo Mäki institute, associated with the University of Jyvaskyla. Furthermore, those assessments are used for a variety of purposes that can have important consequences for students and schools. But at the same time, many teachers, school leaders and policymakers start with the assumptions that things are going (at least relatively) well, that they will know if things start to go off course, and that, if necessary, everyone will work together to get things back on track.

Thomas Hatch

India

Poor PISA score: Govt blames ‘disconnect ‘ with India

Anubhuti  Vishnoi,  Indian Express online (September 3, 2012)

Indian students participated in Program for International Student Assessment (PISA) for the first time in 2009. 16,000 students from 400 schools across the Indian states of Himachal Pradesh and Tamil Nadu took this test. Though Himachal Pradesh and Tamil Nadu are among the best performing states in India, the PISA scores of their students were dismally low, leading to much discussion in India. The Ministry of Human Resource Development (HRD), however, is arguing that these scores are not a reflection of the country’s schooling but of the disconnect between the test questions and India’s socio-cultural specificities, especially that of rural India. The Ministry will write to the Organization for Economic Cooperation Development to address this disconnect.