Learning About Remote Learning
Associate Teacher Simon demonstrates radio lesson activities, like sack gardens, for his students. He hopes that by leading by example, his students feel encouraged to tackle the projects themselves.
The outbreak of coronavirus has flipped the script on schools, classrooms, and education around the world. The spread of the virus and subsequent lockdowns have resulted in the largest disruption to education systems in recent history, and learning has come to a standstill for many young people. This sweeping disruption has also led to a new way of operating for Educate!. In order to serve youth during this critical time, our skills-based model—delivered previously in schools and through national education systems—has been reimagined, and we have moved swiftly to adapt our in-school model for distance learning through radio and phone. After about seven months of implementation, we’re doubling down on our commitment to building our solutions on a foundation of data collection, turning our attention to determining how best to measure the impact of our new distance learning model.
Monitoring and Evaluating: Changing Contexts Require New Approaches
While our mission to prepare youth in Africa with skills to succeed today’s economy remains unchanged, the pandemic has drastically affected the activities we use to equip youth with critical 21st century skills. With schools closed, we leveraged our proven in-school solution to develop an innovative, flexible distance learning model, which can be adjusted to the pandemic’s uncertainties. To help ensure our model is as effective as possible, we’ve had to adapt our Monitoring and Evaluation (M&E) strategy and align our indicators and measurement tools to new skills course delivery methods.
The rapidly-evolving nature of this crisis means it is critical that we continue to work with speed and flexibility. Educate! is currently operating within two- to three-week ‘sprints.’ The M&E team is embedded into the sprint teams to help identify the necessary data to answer pressing questions and process learning. The sprint teams collect and analyze this data, and then leverage key learnings to make immediate model improvements. Resulting design updates have already led to positive outcomes. For example, in Uganda we tested a few strategies to increase student participation by using reminders and nudges. We found, through examining student engagement data, that we can increase student participation by distributing weekly reminders through graphics on social media and by sending automated reminder text messages and phone calls to learners and their caregivers. We’ve also leveraged Educate!’s countrywide school networks, so youth are encouraged to participate by their teachers, school administrators, families, and peers.
In addition to adapting our M&E Team’s usual way of operating, we’ve also had to seek out new tools and mechanisms for data collection and the measurement of our distance learning model. New methods of data collection which allow us to learn and make rapid improvements include:
Assessing Learner Engagement & Outcomes via SMS/ USSD in Uganda
To ensure learning prompts are being delivered effectively, our sprint teams are tracking student engagement and understanding. Our design teams script assessment questions based on lessons, and the M&E team programs them to reach learners through SMS (simple text messages) and USSD platforms.Mobile-based Assessments in Partnership with the Government in Rwanda
Educate! is partnering with the government in Rwanda to deliver educational assignments and activities to teachers and youth through mobile phones. Currently, our team is gathering assessment and user data through WhatsApp as well as through standard text and USSD platforms.
While tracking youth participation and performance at a high level is important (i.e. measuring how many youth are tuning into lessons and what they are doing with this knowledge), we still need to know why youth choose to engage with the lessons and what their participation means for their long term skill development. In order to capture richer data, we are conducting surveys with a sample of participating teachers and youth to learn about their experience and gather feedback to improve our remote learning model.
Emerging Best Practices and Insights for Effective Distance Learning
Although remote learning is new territory for our organization, through rigorous monitoring and evaluation of our distance learning model we have already begun to identify a set of emerging best practices, key insights, and early results.
During the lockdown, Educate! Mentor Kyarisiima has tapped into her entrepreneurial energy and started a new agricultural enterprise farming crops using all-organic pesticides
Despite the tremendous challenges youth are facing in this pandemic, we’ve found that young people can continue their learning remotely if they receive adequate support and encouragement from family members and school administrators and have access to engaging skill developing programs. Through data collection and outreach, our M&E Team has determined that clear communication to youth, caregivers, and school administrators is critical for increasing engagement within our distance learning model.
However, we recognize that despite this support, some learners still struggle to participate, or to participate consistently. Knowing that girls and boys face unique challenges in and out of the classroom, we have spent time analyzing our data to make improvements to our model from a gender equity perspective. By disaggregating our monitoring data by gender, we have learned more about how participation differs between girls and boys. One of our earliest learnings was that boys were more likely to own their own phones than girls—making it more challenging for many female learners to participate in lessons. Another challenge was that girls may now be expected to take on additional responsibilities at home, so they don’t have the time to prioritize lessons. In order to actively engage female learners, our team began experimenting with targeted outreach to female learners, including facilitating all-girl conference calls as a way to create safe spaces for girls to learn at home. We also heard from girls that a rigid daytime lesson schedule was difficult to work with, so we added a second set of lessons at another time of day to make the model more flexible. Additionally, we are now connecting youth with mentors over the phone so that participants have someone they can contact directly if they miss a lesson and need support to catch up.
As participation in our distance learning solutions has continued to increase, we’ve been inspired to see just how youth are putting into practice the skills and knowledge they’ve learned through remote learning activities. Most recently, two Rapid Impact Assessment surveys have provided information about the percentage of students who have started their own businesses due to Educate! virtual lessons, or are considering doing so. 70% of youth who participated in the surveys said they were feeling “excited and in charge” about starting new businesses. We are particularly encouraged by these findings, as they show that even in times of incredible hardship, youth are committed to continuing their education and pursuing economic activity.
70% of youth surveyed about starting new businesses are feeling “excited and in charge”
Iteration and Experimentation: Our Commitment to Learning
As we continue to navigate this new reality, we have been reminded of the critical role data collection and analysis plays in designing and delivering an effective, youth-first model. Moving forward, we aim to refine the metrics of success for Educate!’s distance learning model and further develop digital systems and processes to rapidly assess youth skill development, which we believe are crucial to understanding and strengthening our impact in this complex, evolving context.
While this unprecedented environment has made it particularly challenging to monitor our delivery and to assess our impact in the same way we did before, we are dedicated to ensuring that Educate! continues to have a meaningful and measurable effect on youth life outcomes during this time. By continuously learning about what works within this new context, we believe we can further strengthen the impact and sustainability of our proven model, and ultimately reach more more youth than ever before.