Navigo learning Analytics DAshboard

Details

Role: Research Fellow, designer

Research project: iRead — https://iread-project.eu

Funders: European Union: Horizon 2020

Audience: Primary school teachers using the Navigo literacy game in their classrooms

Description: The Navigo game is designed to support primary school children in developing their reading skills through a diversity of mini games that target different aspects of reading, such as morphology, phonemics, fluency, and more. This prototype learning analytics dashboard (LAD) for teachers displays data from the gameplay in realtime, to improve their literacy teaching effectiveness by displaying:

1. Language features exercised in a given timeframe and children’s competency average competency level on those features;

2. The top features that require improvement and, specifically, how each child is performing on that feature;

3. Time spent and games played over time, who was playing when, and how well they were doing overall.

The teacher can also filter for data over different timeframes, for a specific student, by language feature (by clicking on the tree map visualisation). Based on these displays and functionality, we hope that teachers will be better able to identify problem areas and struggling students, to better orchestrate future teaching plans. The section below describes the design of the LAD in more detail.

Medium: Tableau

NOTE: While the data presented below represents genuine interaction data, the names of children are fictional for anonymity purposes.

Detailed description of LAD design

Written and designed in collaboration with Prof Manolis Mavrikis, Dr Laura Benton, and Prof Mina Vasalou.

The Navigo game

Navigo is an adaptive, tablet-based game designed to reinforce literacy learning in primary schools. It covers extensive content across a range of language categories, e.g., basic phonics, vowel digraphs and trigraphs, verb conjugation, adjectives and adverbs, and more. Within each category are a subset of more specific language features, e.g., in vowel digraphs and trigraphs, you can encounter ‘ar’ as in ‘artist’ or ‘a_e’ as in ‘cake’. With 16 mini-game mechanics, there are over 900 possible game activities. Teachers can assign learning activities to specific students or to the whole class using a web-based “Teacher Tool”. Playing the game activities generates learning analytics (LA) related to what children played, when they played it, and how well they performed.

Navigo Learning Analytics Dashboard for teachers

Building on co-design workshops with teachers, we designed a LA dashboard (LAD) with the following aims:

1. To plan learning activities addressing the most common gaps in class

2. To decide when to move to the next learning objective at a class level

3. To identify lack of student engagement and enhance their motivation

4. To assess students’ strengths and weakness and plan individualized support

5. To self-assess teaching practices

The LAD has four main components (Figure 1a). First, in the top right, are the legends and filters that apply to the entire LAD. The “competency level” legend codes the other graphical displays by color, depending on children’s mastery of target concepts and skills, where 1 (red) is very low competence and 10 (green) is complete mastery; this color-coding is intended to aid teachers’ interpretation of the analytics. The timeframe filter allows teachers to select what data they want to look at. In Figure 1a, it is currently set to the “last 10 months”, which covers the entire time the class had been playing Navigo. Teachers can also filter by student, which we will look at below. Below we review how the LAD was designed to support its five aims.

Aim 1: To plan learning activities addressing the most common gaps in class

The second component in the top left is a tree map that plots the number of games children completed in specific language categories (e.g., vowel digraphs & trigraphs, consonants, suffixes, confusing letters, etc.), wherein larger blocks represent more games played. Hovering over different blocks will provide more information about the frequency and mastery of specific language sub-features that were practiced, as shown in Figure 1b. Teachers can use this to identify common knowledge gaps in the broad language categories. A third component of the LAD is a bar chart, in the lower-right, to help teachers identify more granular learning objectives requiring the most support. This chart displays the “Top language features in progress that need work” and displays children’s recent change in competence on the X-axis. A language feature is considered “in progress” if more than one child has completed at least three Navigo mini games on that feature. An important feature of this LAD is that all charts can be filtered by selecting different components. For instance, selecting a block in the tree map will filter for the top language features that need work in any one language category. Hovering over this bar chart will show a list of all students and their performance in the hovered language feature. This is an easy way for teachers to see specific language features with the lowest level of competency, to help them decide if more games should be assigned to that feature, or if that feature requires further instructional intervention in the classroom. In other words, the LAD was designed to support teachers to translate their interpretation of the LA into pedagogical action.

Aim 2: To Decide when to move to the next learning objective at a class level

The fourth component of the LAD, in the bottom-left, is a line chart that plots statistics over time. The red/green gradient line plots children’s overall competence, to support teachers in identifying general trends over time. The thickness of the lines represents the number of games played, so very thin lines would indicate to teachers that the competence color should be interpreted with caution. Using this line, teachers can identify when children reach a high level of competence and are ready to move onto the next learning objective. For example, in Figure 2a, we have selected the vowel digraphs & trigraphs language category in the tree map, which then filters the other views, including the line chart, to display only data related to that language category. This shows that the class’s competence in vowel digraphs & trigraphs increases relatively steadily over time, before dropping off again in early February and stabilizing.

Aim 3: To identify lack of engagement and enhance students’ motivation

The orange line in the line chart plots the number of students playing each day. This is designed to help teachers take note of dips in participation, which was particularly important during the COVID-19 lockdown when children were playing at home. By hovering over each data point in the line chart, teachers can see who played on each day (Figure 2b), giving the teacher the opportunity to reach out to students with low engagement.

Aim 4: To assess students’ strengths and weakness and plan individualized support

There are several ways in which the LAD can help teachers identify children who need additional support. Above, we described that hovering over the line chart and the bar chart provides a list of students and displays the number of games played and their performance on those games (Figure 2b). If we look more closely at the output (Figure 2c), we can see that, on December 13th, Hailey Cleminson was performing well in her vowel digraphs/trigraphs exercises, while Harry Dunwell was really struggling. This might merit further investigation to determine if Harry needs extra support. In the “Select a student” filter, a teacher can select Harry’s name, which filters the LAD to only show Harry’s data (Figure 3a). Then, Harry’s teacher can select vowel digraphs & trigraphs in the tree map (Figure 3b). The line chart shows that, while Harry did struggle earlier in the year with this language category, his competence has increased steadily over time. The bar chart can help the teacher identify the specific sub-features in vowel digraphs & trigraphs that Harry still struggles with, like ‘ar’ as in ‘artist’ and ‘a_e’ as in ‘cake’. Harry’s teacher can now decide if Harry needs an instructional intervention on these sub-features or more practice through Navigo. Harry’s teacher can also look at what Harry is good at (e.g., consonants, in bright green on the tree map) and praise him for his good work, to sustain his learning motivation.

Aim 5: To self-assess teaching practices

All of the functionality described above could help a teacher self-assess their teaching practices, by comparing students’ performance in particular language categories/features to their practices in the classroom. The line chart may be particularly useful for this aim. For example, a teacher might notice children struggling in a particular language category over a period of several days. They might then choose to intervene in the classroom with an extra lesson on the topic. They could then determine if the intervention was successful by watching the line’s trend following the intervention.

In summary, our LAD was designed to facilitate teachers’ data-informed decision-making by raising their awareness of the available information collected through Navigo, aiding their interpretation of the analytics through different charts, color-coding, and filtering, and encouraging teachers to translate their interpretation into pedagogical action, by highlighting particularly troublesome areas and showing trends over time.