Tag Archives: AERA2015

Real-time Data for Real-time Use: Case Studies from Ghana and Mali

As part of a symposium focused on educational innovation around the world  at the annual conference of the American Educational Assocation in Chicago next week, we are sharing commentary papers from the participants.  Today’s contribution is from Radhika Iyengar, of the Earth Institute at Columbia University.

A common and most often used source of education data comes from household surveys. These household surveys are helpful to measure the “impact” of the education strategies and policies adopted by developing countries. The Multiple Indicator Cluster Survey (or MICS) are household surveys that various countries along with UN agencies (specially UNICEF) use to collect data on educational outcomes-such as net attendance rate, net intake-rate, primary school survival rate. These indicators are helpful to track progress towards the Millennium Development Goals (MDGs) indicators. The primary focus of the MDGs is to measure the progress towards universal primary education for both girls and boys. The indicators focus on issues like whether girls and boys alike attend primary school at the right age and are able to complete a full primary school cycle.

Despite their potential utility, household surveys like the MICS come with a set of challenges. The process of collecting household-level data using the surveys is very time intensive. Not all countries have the time and the resources to conduct these surveys on an annual basis. Even if they do, a full population census to calculate the denominator of indicators like Net Enrollment ratio could be a decade old. For the most part, the data is entered at the national level. The data is then cleaned, processed and sent back as aggregate numbers usually at the state level. District indicators are hard to find. The schools that patiently supplied this information and the households that took time to respond to those lengthy surveys never get to see the “end product.” The data appeared to vanish in thin air and what comes back in aggregate form is usually not useful for a school or a district to measure its progress. In fact, by the time the data is released, a student who was in Grade 2 may have dropped-out or may have proceeded to the next class without learning anything.

To address this lag in feedback, The Millennium Villages Project along with the Sustainable Engineering Lab have designed and implemented an android phone-based data collection system that collects real-time data. This data is then analyzed using faster back-end processing to provide feedback to the schools and the district education offices on a monthly-basis. This data collection, management and utilization system was developed after multiple years of practice based learning from the Health care system. The processes were tightened and improved upon and tis system is now being utilized for an integrated planning approach for the Education Sector as well as the Water Sector.

Let us first try to understand why is it important to collect all this education data in the first place. Figure 1 below shows that people’s perception don’t always reflect reality. The reality is that many children are not able to do Grade 2 level literacy and numeracy even when they are in Grade 4. This seems counter-intuitive to the general perception that things are going well in education.

Screen Shot 2015-04-13 at 9.37.38 AM

Figure 1. Satisfaction Results in Education (In East Africa)

Source: Pritchett (2013) . The Rebirth of Education: Schooling Ain’t Learning

However, we need quicker and useful data to make any difference. If time goes by and the people who supply this data don’t ever get to “see” the usefulness of collecting this data in the first place, these indicators may just become statistics.

In the Millennium Villages Project site at Tiby Mali, the data collected via the phone-based collection system showed an interesting geographic trend. Figure 2. presents the proportion of enrolled students attending observed classes at the time of observations for a particular month. In Tiby, Mali the data shows that the schools circled in green have a much lower student attendance than the schools circled in blue. It is surprising to see a clear geographical clustering of indicators based on the location of the schools. Why is it that the schools clustered near each other (in blue) have better attendance than the rest of the schools scattered (in green)? This map helped to form the basis of discussion with the District Education Offices. The discussions showed that the geography was a part of the problem. Schools are much more dispersed (in the north and south) than schools near the towns therefore distance to get to schools may be longer. The discussions also suggested some structural issues that promote teacher absenteeism. For instance in Mali full working days are Monday, Tuesday and Friday, with two school sessions per day but on Wednesday and Thursday schools are working with only a morning-shift. Absences are more frequent on Wednesdays and Thursdays because many teachers use those days to take care of their personal and administrative needs. The data helped to start a conversation in Mali on teacher attendance issues and made those issues much more visible and easy to understand. As a result of this early detection of the teacher absenteeism trend, government school inspectors have increased their school supervision and focus on the specific schools that showed repeated student attendance issues. The Mali example shows how stakeholders are able to use the data to assist in the functioning of the schools.

Screen Shot 2015-04-13 at 9.37.53 AMFigure 2. Proportion of enrolled students attending observed class at the time of the visit in Tiby, Mali.

Source: Millennium Villages Project Database.

Another very important indicator is that of student’s learning levels in basic literacy skills for numeracy and language. The data are collected on a monthly basis to measure if children are falling behind. The data are intended to help the school and the teachers to plan for remedial education based on understanding the gaps in the literacy measure. For instance, if a majority of the children tested are in the word recognition category, the teachers can focus on activities that are geared towards move children from the word recognition level to higher levels of readings – reading paragraphs and simple stories. The monthly data depicted on a map from Bonsaaso, Ghana suggested that the schools that lack full time teachers are also the schools where students lack basic reading skills. Surprisingly all these schools are grouped towards the south of the cluster.

Screen Shot 2015-04-13 at 9.38.13 AM

Figure 3. Reading Levels of children in Grade 3 in Bonsaaso, Ghana.

Source: Millennium Villages Project database

Further discussions with district education officials revealed that these schools are located in the remotest part of the cluster with many in the Amansie West District. The area lacks basic infrastructural amenities and teachers do not accept postings to those areas, hence lack of teachers is one of the characteristics in that area of the cluster and the district as a whole. Many of the teachers who accept postings to these areas are untrained. Since reviewing these data, multiple-stakeholder meetings have focused on improving the learning environment of these southern schools.

These case studies bring up several points. First, regarding the data collection and utilization process, frequently collected data with frequent feedback helps to make the data more useful. Also, the people who can in a position to act after seeing the data are best suited to collect the data. The closer they are to the issues, the better the data use is going to be. The second point centers on the use of technology to improve the functioning of the schools. Often the use of technology is limited to laptops used by students for learning and by teachers as teaching aids. However, the use of technology as a lever to improve education planning as a whole leaves much to be desired. “Real-time” data collection using efficient technology has more chances of being used. The time elapsed between data collection and feedback needs to be relatively short, since people forget what data was collected in the first place. We also need to keep in mind that different data users (policymakers, district officials, school members) often like to see different indicators. Therefore data displays need to be created at various levels.

However, efficient adoption of this technology-based solution for issues of data collection depends on many factors. Political will as well as local capacity to collect frequent data and disseminate the results is key. Stakeholder buy-in from a multi-sectoral perspective can help to gain insights from already existing practices from other sectors such as health. A democratic process that weighs the different data needs at the national, state and district levels is also critical in maximizing data use. This is critical since different stakeholders at various levels may have very different data uses. Who uses what data and how much capacity the system has to collect and process the data are iterative discussions. Despite relatively limited resources, innovation is still possible and can lead to quicker diagnosis and remediation. It is clear that education planning not only requires outcome indicators from the survey, but also process indicators from facility (school) inventories. A great step forward is the UN Secretariat on Data Revolution, which recognizes the importance of such facility mapping (see the case study on the Nigeria Information Management System). Real-time data leading to real-time use should be the data mantra for UN’s Post 2015 Agenda.

 

 

Bringing Effective Instructional Innovation to Scale through Social Movement in Mexico and Colombia

As part of a symposium focused on educational innovation around the world  at the annual conference of the American Educational Assocation in Chicago next week, we are sharing commentary papers from the participants.  Today’s contribution is from Santiago Rincón-Gallardo, Ontario Institute for Studies in Education. 

 

The Learning Community Project (LCP) in Mexico and Escuela Nueva in Colombia are examples of effective instructional innovation at scale. When I talk about instructional innovation I refer to pedagogical principles and practices that deliberately disrupt the traditional instructional culture and power relations of schooling. More specifically, an instructional innovation is one that radically redefines the instructional core by deliberately shifting the relationships between teachers and students in the presence of content.

Learning Community Project (LCP)

In the LCP, for example, students select their topics of study from the available collection of topics mastered by a tutor in the group, they develop individual lines of inquiry at their own pace, and are expected to demonstrate their learning in writing and in public presentations to the group and often to the larger community. Once they master a topic, they are expected to become tutors to other students – and even to adults in the group. The boundary between teachers and students becomes blurry, with teachers becoming students and students becoming teachers depending on who masters a particular topic and who is interested in learning it. Tutorial relationships are the key technology to encourage deep learning in these learning communities: the tutor and tutee engage in one-on-one dialogue to make evident what the tutee already knows about a topic, identify areas of struggle, and to crafts questions or point to additional materials so that the the tutee t can come up with her own answers.

Escuela Nueva

Escuela Nueva in Colombia transforms the conventional culture and structure of schooling into a learner-centered participatory model with a flexible promotion mechanism that allows students to move from one grade to another and complete academic units at their own pace. In these multi-grade schools, children work individually and in small groups using learning guides that are by design interactive and dialogue-based as well as learning materials available whenever needed in a “Learning Corner.” Students help each other when they struggle, and ask the teacher for suggestions or comments when necessary. Individual mastery and cooperation are seamlessly integrated into every classroom activity. Teachers constantly move from group to group, tailoring their one-on-one and group interventions to the emerging needs in the classroom. Community participation and a student school government are integrated in the everyday activities of the school, offering multiple opportunities to practice and master democratic behaviors and values.

Innovation, effectiveness, and spread

These are examples of instructional innovation in action. They have radically redefined the instructional core. But innovation per se is meaningless if it doesn’t deepen and improve student learning. Both the LCP and Escuela Nueva have demonstrated significant improvements in student performance on national standardized tests, even though standardized tests have not been their area of focus. Learning Community schools increased in 3 years the percentage of students achieving good and excellent levels in language and math at a faster pace than the national average (DGDGIE, 2012), whereas in the 1990s Escuela Nueva students – mostly from rural schools – consistently outperformed their better-off counterparts in urban schools (Psacharopoulos, Rojas & Velez 1992).

And maybe more importantly, these two models, have spread from a handful to thousands of schools at some point in time. In 2012, LCP model was operating in 9000 public schools across Mexico. In the 1980s, Escuela Nueva had been adopted as national policy and reached 20,000 rural schools in Colombia. These initiatives are similar in their genesis and development to social movements. Upon witnessing powerful learning themselves and observing clear improvements in the learning and engagement of their students, teachers and local educational authorities have mobilized in coordination with project leaders to activate social networks, spread interest and gain support. Leaders of both projects have been able to gain access to institutional power and political influence to disseminate the new pedagogies on a large scale. Dalila López and Gabriel Cámara, historical leaders of the LCP were invited in 2009 to the Department of Innovation at the Mexican Ministry of Education and once there brought in several project leaders to her team. Vicky Colbert, co-founder of Escuela Nueva, was Deputy Minister of Education in Colombia when Escuela Nueva was adopted as a national policy, she also brought to her team teacher leaders with strong experience on instructional innovation. Once in power, these two guiding coalitions developed a progressive partnership between policy and practice, rather than the conventional top-down separation that has characterized education policy in Latin America and abroad . Across the system, project participants, regardless of their formal role in the institution, were expected to practice and model the new pedagogies on a regular basis. 

Conditions and challenges for innovation and spread

Here are, in a nutshell 5 key conditions that were created to enable the large-scale dissemination of the new pedagogies advanced through the LCP and Escuela Nueva.

1) developing a new pedagogy that allows teachers to experience powerful learning themselves and to witness observable improvements in the knowledge, skills and attitudes of their students as a result of changing their practice

2) creating access to multiple opportunities to observe, practice, and refine the new pedagogy (e.g., classroom-based coaching, communities of practice, and school visits/exchanges);

3) gaining the support or permission from local educational authorities to depart from conventional schooling practices

4) starting at the margins of the educational system, where the needs are greatest and the presence of institutional controls over the everyday activities of schools is weaker. This offers tremendous opportunities to radically depart from conventional practice

5) creating a guiding coalition of pedagogical change leaders with access to institutional power and political support to protect and expand the influence of the innovation

A key limitation to the large-scale instructional innovation approach just presented is that it is subject to marginalization or disappearance from the policy arena when there are drastic shifts in the political agenda. This has been the case in both the LCP and Escuela Nueva. At the pinnacle of their success, changes in administration and in system priorities resulted in the departure of the national leadership of these projects from their respective Ministries of Education.

Bureaucratization, ritualization or mechanization of the original model is also a risk when the innovation is brought to scale too quickly, a phenomenon that has been observed in the two cases in question. In both cases, the work to sustain the movements of pedagogical change spurred by these projects and to ensure quality of their core pedagogical practices has continued through NGOs formed by the former leaders of LCP and Escuela Nueva. As is often the case with social movements, their visibility may be reduced for now, but they continue to cultivate a movement that is radically redefining teaching and learning in public schools. In this new phase, the model of dissemination at scale will have to rely more on the power of effective networks to consolidate and spread the new pedagogies reliably and at scale than on formal access to institutional power.