Module evaluation analytics: how can we deliver the right reports to the right people?

As we move towards a data-driven environment within higher education, the need for access to the right information at the right time is ever increasing. This is no different for module evaluation projects and making improvement decisions, whether it is for students, teaching staff, quality management, or senior management.

Too often, universities are limited by the analytical tools within their module evaluation system, either by using the ‘standard’ basic report template or downloading results into spreadsheets. Without the right tools, the issue for module evaluation administrators is they find themselves sitting on a mountain of student experience feedback data, but not enough time to gain actionable insights at all levels across the institution.

So, what does it take to deliver the right module evaluation analysis reports to the right people at the right time? This was our area of exploration in our latest ‘Lunch and Learn’ webinar that we held on 8^th April. Topics we covered included:

• Creating and maintaining an institutional reporting hierarchy
• Developing report content for different roles and closing the feedback loop
• Adding context with demographic data
• Analysing free-text comments automatically
• Engaging staff with dynamic dashboards.

Reflecting on the webinar, I think there are five key areas of discussion that highlight how this challenge can be made easier for university colleagues:

1. Opportunity

   There is an opportunity – and this is not commonplace across the Higher Education sector – for module evaluation reports to be produced and shared with lecturers and wider stakeholders up to and including senior management. Whilst Blue has its own reporting engine, users can easily decide what these say and who these go to. This is easily integrated into existing IT infrastructures within institutions and not standalone.

2. Flexibility

   Universities have the flexibility to analyse quantitative and free-text comments through enhanced reporting. Tabular reports show how modules in each department are doing – top-level scores – and compare within departments and institution-wide. They can look at cross tab ratings with demographic data, e.g., GPA range, cross tab open-ended feedback and quantitative feedback with demographic data, and contrast averages across different time intervals, so longitudinal analysis which can be extremely valuable.

3. Personal

   Where Blue is strong is that it can provide insight on class module averages, teacher averages, department averages, and institution average – and present these concisely and powerfully to generate meaning. Blue text analytics – free-text analysis presented through a word cloud – gives true insight on how students are feeling and where attention is needed. Sentiments are then categorised into terms such as helpful, supportive, difficult, interesting, organised, clear, engaging, enjoyable, enthusiastic, dedicated, important, and relevant.

4. Ease

   In terms of where the reports go to, through Blue, universities can build department/faculty reports and distribute these to the right individuals. Reports are issued to teachers, heads of department, and senior leaders via email or through the VLE. Reports can also be downloaded and shared or accessed via interactive dashboards. And – because the way these are set up – if a member of staff leaves and they are replaced, changes can be made within one line of data giving new individuals access to everything past, present, and future.

5. Confidentiality

   A question was posted during the webinar around the confidential nature of responses and reports. With Blue, there is a number of different levels to make surveys confidential or anonymous. We can, however, ensure that students are non-identifiable in module evaluation reports but still look at demographics for overall trends. All the data that is produced in Blue can be exported or pushed into other reporting systems.