A few years ago I drew a flow chart. It showed the students flowing from one course choice to another in pretty colours and highlighted the popular pathways. Then I drew another flow chart which was the same, except the colour of the flow indicated the proportion who eventually got through to the qualification they were aiming for – and hence highlighted the more successful paths. I’ve been looking for a piece of software to build what I saw in my mind for a long time and about six months ago I found it.
The charts I was drawing were curved, flowing Sankey type diagrams. I had developed the idea from Ormond Simpson’s rivergrams, used in various publications including his recently updated book ‘Supporting Students for Success in Online and Distance Education’. The width of the stream indicates the number of salmon i.e. students and the rivergrams showed where students were being lost… and in some cases brought back. I’d been shown the rivergrams by another colleague, Dr James Warren (Senior Lecturer, OU Faculty of Mathematics, Computing and Technology), who wondered if we could make something similar and we worked together on options and ideas. The best I could do with the software and data I had at the time was to create what became known internally as “shards”. The shards are a basic area plot of three data points – the number of students who start, pass and progress – but the simplicity of the visualisation helped people quickly compare how students were doing on the different courses.
This is it: my woven graph of student success. It’s a type of Sankey diagram, showing the numbers of students who started a course and how they progressed through each of the part-way assessments to the end. Flowing from left to right, the students go through six numbered gateways and either pass (P, in blue), fail (F, in red) or do not submit (X, in grey) each assessment element. The final gateway is the result of the course, either to pass (PP, blue) or fail or withdraw (FW, red). On this course we can see that the majority of the students pass each assessment and in the end pass the course. Even those who skip or fail one or two – or even three – of the assessments often pass the course, leading to an eventual pass rate of about 70%.
With a graph like this we can quickly spot some types of patterns. For example, of the students on this course who passed assessment 1 and then didn’t submit assessment 2, only one managed to get back onto a positive (blue) path and pass the course. We can also see that some students might be gaming – they’re skipping or low scoring on an assessment, yet then immediately get back onto a positive path and do pass the course. Having built these graphs for several courses already I’ve been surprised at the extent of the differences in patterns of behaviour and success.
I haven’t quite decided what to call this graph. James has pointed out that the ideal layout would be a square with blue stripes – all students succeeding from start to finish – so perhaps the way to think of it is like fraying threads that we want to weave back into the fabric?
It is essentially a visual representation of conditional probabilities which would quickly become complex to read, but we can see how the choices and rates of success of students at each stage feed into their progress through the rest of the course. By combining the theories that this graph inspires with other data like student feedback, we can start to build an understanding of why some assessments put some students on a path which leads straight to success and others seem to be more of a challenge, leading to more diverse combinations of further results – the threads coming apart. We can use it to help us consider giving students extra support at key points or tweaking the structure of the assessment to weave them back in.
I’m working now on different types of these graphs and also on combining them with other types of data into a dashboard – scribbles in my notepad and gorgeous colours and shapes in my mind that need to be turned into functional code and script. A similar flow of students from course to course would enable us to see where students find the most successful routes through study, so that we could advise others on the best pathways.
I welcome any feedback on my visualisations above. I haven’t seen or heard of any examples of others using the Sankey plugin for D3 to visualise student data like this – if you have, please do get in touch.