First, we needed to understand what teachers were looking for. We looked at data from education research reports, user interviews and observations, and conversations with team members who had expertise in the field of education.
The coach report tools support different stages of the formative assessment cycle: a low-stakes, informal way that teachers seek to understand what students are retaining throughout the learning process. Think of pop quizzes, quick surveys or polls, or when a teacher simply asks the class what their takeaways are from the lesson they just learned.
This helped us be mindful of how the tools must adapt when teachers are at different stages of this cycle. They should be able to quickly find the information they're looking for when they need it.
Credit: Teachers Know Best: Making Data WorkFor Teachers and Students
While Kolibri plays a role at all stages of this cycle, we realized the reports play the most important role at the "Plan & Teach" and "Analyze" stages - helping teachers measure their students' performance and presenting data that can help inform their next decision.
Our interviews, observations, and feedback showed us how Kolibri fits into their process for planning, teaching, and evaluating their students. Creating these journeys helped keep us grounded in the realities of our users when discussions about effective teaching and learning became more abstract and theoretical.
Broader systemic issues also create compounded effects on the problems that the average teacher already faces. For example, large class sizes due to limited resources and infrastructure make it more difficult to provide individualized instruction and teacher-student engagement. Lack of resources and quality learning materials also made it difficult for teachers to access professional development and consistent support.
Learning Equality creates tools that address barriers to equitable educational experiences. While the tech platform itself is not a "silver bullet" that can address all of these problems, it was important to keep these issues at the forefront as we considered potential design solutions.
Our research allowed us to create simple personas with two major parameters: level of teaching experience and type of learning environment.
Novice teacher
Out-of-school catch up program
Unfamiliar with digital tools in general
- Has trouble knowing her next steps after delivering a lesson
- Slightly overwhelmed and intimidated by the idea of using digital tools for teaching
- Usually doesn’t have enough time for lesson planning
- Struggles to recognize what her students are understanding during lessons and adapting on the spot
Experienced teacher
Secondary school in a refugee camp
New to using digital tools in the classroom
- Hesitant to adopt new and unfamiliar practices
- Familiar with differentiated teaching methods that meet individual learning needs
- Leans on intuition to decide next steps for teaching
- Excited to learn to use digital tools but fears breaking it
- Classes are too large to give everyone individualized support during class
- Support every student in reaching their learning goal
- Provide individualized support that meets students where they are at
- Help students take ownership of their learning
Based on our understanding of the constraints faced by teachers who use Kolibri, we prioritized the needs of teachers who fit the persona of Louise. The updated design should encourage good teaching practices while meeting teachers where they're at, offering support no matter their experience level.
In summary, Kolibri's coach reports should enable the following:
- Save teachers time. Navigation and data displays should support fast-pace teacher workflows by showing what's happening "here-and-now."
- Snapshots of strengths and weaknesses. The data should show which students are progressing well or are having trouble with specific concepts.
- Show patterns across learners. By showing data that groups learners by their performance, teachers can start to plan what they will teach next to who.
- Support differentiated instruction. Based on this new information, teachers should be able to quickly assign materials and adapt the lesson plan in ways that are appropriate for each students' level of understanding.
We started moving from insight to design by writing user stories that corresponded with the personas and the different stages of the formative assessment cycle.
As a [user type],
I want to [task],
So that I can [underlying motivation]
We visualized themes with a user story map to make it easier to discuss our next steps.
We couldn't do it all and realized we needed to narrow our scope of what we had the capacity to support in this redesign. Based on what we knew, we evaluated which stories we were already somewhat addressing in our platform, which ones were not yet met but seemed possible to address in the immediate term, and which stories we knew would be better to address in the longer term.
After this, we had a solid set of stories that aligned with our design goals, research, and were feasible in the short term. Through this process, we were also able to capture key information that should go in the reports and how it could be organized.
There are edtech platforms out there who addressed similar or analogous goals. We looked at other edtech tools who overlapped for a few reasons: to find any common patterns for experiences similar to what we were trying to achieve, as well as any pitfalls and flaws to avoid.
The personas and stories we wrote guided us to put ourselves in the shoes of teachers as we explored these platforms.
There were some problems where our design team of 2 needed more perspective. Crazy 8's are a fast way to explore many potential solutions at once for a challenging design problem where the design direction isn't straightforward. We did this activity with a few of our team members who had past experience as educators.
We reduced the top-level navigation from six tabs to three tabs that targeted teacher intent at different stages of the formative assessment cycle: Class home, Reports, and Plan.
We introduced a dashboard that showed summaries of completion status for different learning materials. To assist real-time monitoring, we included an activity feed that updated at frequent intervals.
The new "needs help" status highlighted what concepts students were getting incorrect answers for in their most recent attempts on questions, drawing more attention to what teachers should focus their next lessons on.
The "needs help" indicator could also be attributed to individual learners so the teacher can get a sense on who they need to follow up with.
Teachers could drill down further to see which specific questions and concepts that students struggled with the most, and identify patterns across difficult concepts and learners.
This feature naturally created groups of learners who had common misunderstandings to give teachers the information they need to decide interventions at the group and individual level.
User testing with a group of 12 teachers who were already using Kolibri showed that overall, the redesign improved how teachers could make sense of their learners' data from the previous version of the coach reports.
- The new terminology and language was reported as feeling more intuitive and clear
- All were able to grasp the new information architecture quickly and felt that "things were all in one place."
- The "Difficult questions" feature automated a pattern that teachers spent a lot of time trying to synthesize looking at individual student data. Some reported this as "what the teachers wanted to know, specifically."
- Teachers felt they could use the reports to know what to check in on in person with their students during class.
We were also able to get constructive feedback to improve things in later rounds of development.
- They wanted to see more detailed and visual patterns of data like most common responses and numbers of attempts
- Explanations on how "most difficult" and "needs help" was calculated
- Exposing the "needs help" metric on the student-facing side to help with self-study.
- How the platform could stay up to date when students resolve their misunderstandings through a different modality than Kolibri.
This project set a strong foundation for how the Product team at Learning Equality understands and talks about teacher needs from a design perspective. The process also helped us figure out how we can work together to create shared foundational understandings of our users that can motivate product direction.
Scoping this project meant dealing with high complexity; in addition to managing UX/technical considerations and team capacity, there were many different social problem areas intersecting and compounding on one another, from educator skill gaps to infrastructure barriers to the upheaval and changes that come while learning as a refugee. Keeping complexity from becoming complicated was challenging. The process highlighted two things: how to distill the most important information about users that directly affects the product design, and the importance of including the perspectives of cross-domain colleagues and stakeholders at every stage of the design process.
Looking back, if I think about the question of "one thing I could have done differently," it would be to find a better balance between over-documentation and "doing design." Good documentation helps a team understand why decisions were made in the long run and have a north star to point to. It also helps a careful approach when designing within problem spaces where there are risks of technology exacerbating already-existing social inequalities. However, tools like user stories and IA maps can only take you so far – the meat of design is in exploration through action, creation, and thoughtful iteration.
