EduSupply Nagaland

Kicking off with an HCD Visioning Workshop

The project began with a week-long field research sprint followed by a co-design workshop, which brought visibility to the challenges facing the textbook supply chain. Engaging diverse stakeholders from school system leadership, curriculum planners, procurement leaders, the core textbook team, principals, and teachers, this effort facilitated early issue identification, planning, and visioning. It also helped us sequence and visualise responsibilities, actions, and events for the supply chain for the first time.

Over the next year and a half, we worked across four distinct work streams:

• Research: Conducting Foundational, Evaluative, and Experimental Research
• Opportunity Identification: Identifying Process, Policy, and Product-Service-System Recommendations
• Technology: Developing Tech-Based Tools for the Supply Chain
• Implementation: Piloting Tech Tools at scale via Randomised Control Trials (RCTs)

A Mixed Methods Research Deep Dive During the Peak COVID Waves

Limited by the COVID-19 pandemic, our team pivoted to a remote research approach. We conducted over 80 in-depth interviews and consultations with stakeholders across the supply chain, including external actors such as publishers and transporters. Our remote conversations were aided by research probes like the visual supply chain map developed for the kick-off workshop. A small local team coordinated data gathering and in-person conversations when necessary. We identified several critical issues - frequent changes to content, delayed handover of final content, translation complications, use of informal data systems, inconsistent data formats, lack of real-time data, frequent personnel changes, geographic and connectivity issues, and lack of funds.

Opportunities and Idea Generation with Stakeholders

To prepare for ideation and design, we mapped the existing system, visualising various entities, their roles, responsibilities, and the nature of relationships and flows pertaining to supply of text books(information, stock, money, and oversight). ‘Delay’ and ‘Shortage’ emerged as the two broad challenge areas. These were further broken down into clear opportunity areas using tools like ‘mind-mapping’, ‘5 Whys’, and ‘Problem Tree’. We socialised our findings and conducted multiple ideation workshops with different stakeholders, identifying close to three dozen ideas for potential interventions - both tech and non-tech. The lack of a real-time and credible student database and a system to track the supply chain in real-time were prioritised for prototyping and testing.

Designing Technology for the Public System User

As we progressed with technology design, several key principles guided us:

Co-creation

• Early into the design process, we realised the gap in including input from last-mile end users of technological solutions in public systems. Therefore, we were mindful about engaging with them as part of our design efforts.

UX Principles

• Inspired by Existing Formats: We built interfaces and actions based on familiar data formats to ease the transition from physical to digital.
• Minimising Errors and Ad Hoc Decisions: We designed the system for users with basic digital skills, aiming to minimise errors and ad hoc decision-making.
• Modular Design: We focused on building modular and replicable UX libraries to accommodate various stakeholders, be it publishers, textbook teams, block and district-level officials, school officials etc.

Technical Principles

• Scalable Architecture: Research revealed multiple technology platforms were being used at the school level, and we found that users found this disorienting. We built a system scalable beyond textbooks to include all school-facing technology use cases.
• Offline Capability: Given Nagaland’s connectivity challenges, offline capability was crucial.

Our Outputs

Pocket School - Student Registry and Database System

We developed Pocket School as a school-facing app allowing administrators to digitise student records typically maintained in physical registers. Pocket School (a name suggested by a teacher) was designed to provide state-level leaders with aggregate enrollment data, detailed student records for planners, easily shareable records for schools, verification tools for observers, and a mechanism to map issues like unrecorded student transfers and duplicate enrollments. Planned as a student registration system that would eventually include unique IDs for students (as envisioned under the NDEAR framework), for the pilot, we used the PCR Book Number (Pupil Cumulative Record - a collection of report cards through the student’s life), as an ID surrogate.

Track & Trace - An End-to-End Supply Chain System for Textbooks

Our end-to-end Track & Trace system prototype linked publishers, the Depot (Textbook Team), block-level education officers, and schools, with school department leadership having oversight via real-time dashboards. The system was designed to generate requirements/orders by book, place orders, record shipments, address shortfalls and damage, place additional orders, and record excess stock. However, we were unable to pilot this system in a real-world setting due to COVID-19 delays, focusing our implementation efforts on the Pocket School RCT.

Implementing a Technology Solution (Pocket School) via an RCT

The Pocket School platform was selected for an RCT across 400 schools (~25% of Nagaland’s public schools) to understand the app's adoption and identify potential levers for scale. The principal purpose of the Pocket School application was to digitise the student database and the RCT was designed to test incentives that would nidge school administrators to complete the task within a given time frame. The RCT included one control group and two treatment groups (positive reinforcement; recognition and reward in one, and negative reinforcement; inquiry and action in another) over six months, with schools randomly assigned from 12 blocks in Nagaland.

• Pre-RCT Pilot We ran a short pilot of the app and RCT experiment with six schools over two weeks, channelling our learnings into refining our RCT strategy and application before the larger pilot.
• RCT Over three months, we operationalised the RCT, onboarding schools via WhatsApp group chats, a WhatsApp business number, and Google Meet calls. We provided predefined messages as nudges and reminders to school representatives. Despite technical hiccups, the Quicksand team offered grievance redressal as well. The RCT assessed the completeness, quality, and verifiability of schools' digitisation efforts, identifying top-performing and worst-performing schools. Schools submitted photos of their registers and PCR booklets for verification. Quicksand collaborated with a local NGO in Kohima to audit the data and calculate school-level scores.
• Post RCT Schools in the treatment groups received recognition and rewards or stern letters based on performance. We also collected feedback regarding their experience with the technology and pilot.

Outcomes from the RCT resulted in over 24,000 students (>15% of all students in the state) being enrolled during the pilot. These were the first-ever digitised student records available to the state. We identified duplicate student records, duplicate PCR booklets, interesting age trends, multiple mother tongues in single schools, and discrepancies in enrollment records.

Learnings from the RCT

• Facilitation and Service: Human facilitation and assistance in driving technology adoption were far more crucial than the digital interface design.
• Iterative Evolution: Technology solutions must evolve with iteration, testing, and continuous improvement.
• User-Centric Solutions: Schools need solutions that simplify their tasks. Top-down solutions are ineffective without enforcement, while voluntary use emerges only when users are delighted.