Blue Plays a Central Role in Education Transformation at Monash University

Monash University is Australia’s largest and most international university. The prestigious Group of Eight member hosts four Australian campuses, as well as two locations in Malaysia, and a campus in South Africa. As a research-intensive institution, Monash must ensure the curricula are highly relevant and well designed, placing the student at the centre of the learning process.

With approximately 75,000 students, more than 6,000 educators, and multiple campus locations, student feedback data governance is no small task at Monash.

Outdated data resource

While Monash has ongoing data gathering efforts, the information that they were collecting was not being leveraged to the fullest. As with any large institution, how the data was being used differed across faculties. Although the university has an expert group that runs their Student Evaluation of Teaching Units (SETU), at the time there were significant data governance issues.

“Monash had a lot of very valuable information from the student insights into their units but we were not fully utilising that data to inform the transformation of our pedagogy and curricula,” commented Professor Zlatko Skrbis, Senior Pro Vice-Chancellor (Academic) at Monash University.

It became clear that a new system and new processes were needed for student feedback. So the Information Systems and Surveys team, led by Kristina Kaulenas, and the Teaching and Learning Quality team within the Office of the Senior Pro Vice-Chancellor (Academic), led by Professor Zlatko Skrbis, worked closely together to investigate requirements as well as the institutional survey systems available on the market.

“This was a major initiative across the University and we had a very important high-profile mandate to ensure teaching across all faculties and all courses was meeting the strategic goal of education excellence,” said Professor Skrbis.

A partner from the start

The joint project team went through a long review process and early on decided on a series of criteria for the new system. Their main goals were to reduce manual cleansing of data, improve response rates (which were already good for Australia), and give faculty and schools the ability to customise questions that they can choose themselves.

In addition, the team brought in a data architect who worked with the project manager to understand the business and complexity of the Monash enterprise. Using an analytical approach, they were able to look at the interrelationships, impacts, and dependencies within the institution and the role that SETU played in the overall picture.

A detailed proof of concept revealed a full breakdown of what was required in a platform that met the business needs, the technical specifications, and IT standards of Monash. From that the Blue platform – along with several other providers – was reviewed.

“It was rigorous, it was long and I can do nothing but praise the professionalism of Explorance in terms of working with us,” said Ms Kaulenis. “They were running 2-3 hour sessions for each stage of the proof of concept where we were asking very detailed questions and we always got a clear, honest answer. I appreciated the way they explained everything, to both myself from a research and pedagogical perspective and our technical teams, who wanted to know the nuts and bolts.”

In addition to the usual product demonstrations, Explorance provided a test sandbox environment for the Monash technical team. There they were able to build their full implementation as well as perform white board exercises which mapped out how various potential scenarios would be handled in the new system.

“I’m the type of person who wants to understand a product in-depth before I commit to it. I undertook, independently, a significant investigation of the available offerings, and by the end, I was thoroughly convinced that Blue was the best product for us,” said Kristina

A roll out without a hitch

The implementation of the Blue system took place at a lightning pace with a small pilot prior to Semester Two 2016, and a university-wide rollout in Semester Two. As a result, the joint project team was able to make the deadline that they had promised the Academic Board.

Blue has achieved several of the team’s goals, including more automation of the survey process and more sophistication in the instrument delivery.

“Once the project is set up in Blue, it’s pretty much automated,” Kristina said. “It can almost run itself; you just have to keep an eye on it.”

To provide more flexibility to faculty, Blue also includes a customisable portion where instructors can select from a bank of questions to add to the survey form.

“This is quite a sophisticated feature,” Kristina continued. “Blue was very powerful in enabling this type of flexible questionnaire design to meet the disparate requirements of our existing 43 faculty variations. I can also now see the potential for how this will all work if we choose to go down the cascaded questionnaire design path and enable question customisation at varying levels within the university.”

In terms of their first Blue delivery, 2703 unit offerings were surveyed and more than 60,000 students were invited to provide feedback – that’s 194,000 individual surveys. In addition, 3,100 staff received feedback via the customised educator questions.

“When we went live with Blue it went exceptionally well and without a technical hitch,” noted Professor Skrbis.

Kristina concluded that, “Usually tech projects have a lot of hiccups. We’ve subsequently ran our Summer Semester and our Semester One 2017 main round as well and it’s all business as usual.

“Once the project is set up in Blue, it’s pretty much automated. It can almost run itself; you just have to keep an eye on it.”

Kristina Kaulenas,
Manager Information Strategy and Survey

Blue – a robust reporting engine

Monash sought a robust solution that could meet both the survey administration and reporting needs of the University. They had complex reporting needs and identified goal to expand the utilisation of the student feedback data beyond teaching teams and into central areas to provide more insightful reporting to relevant governing bodies.

“It was the only platform capable of meeting the complex business and reporting requirements of Monash, as well as offering significant capabilities to further expand Monash’s ability to enhance its continuous improvement objectives”, said Kristina.

Getting better response rates

The University has always had healthy response rates, averaging 35-40 per cent, with good processes and support from faculty to get the word out to students. The Blue system is making those processes even better.

“Our coordinators in every unit can now view response rates themselves in real-time,” said Kristina. “We can communicate with them easily at various stages of the survey and point to various resources if needed.”

“We’ve actually just had our highest response rates ever — 44 per cent — this past semester.”

Sustainability via the Blue training camps

In order to sustain and continue to improve their understanding of the Blue system, Kristina as well as two members of her staff have completed the Blue Expert Certification. Through a series of in-depth instructor-led presentations, interactive workshops, and reproducible exercises, participants in the Expert Certification training plan, execute, and maintain various types of evaluations, surveys, and assessments using Blue.

A third member of Kristina’s team has also done the Blue Administrator training, the first level of certification for all users who will be administrating any feedback process with Blue.

“So this is sustainable and means that we can keep building our understanding of Blue and how it applies to our business,” Kristina said.

Monash University is the first customer in the Asia-Pacific region to undertake the Blue Expert Certification training.

Getting more out of Blue – DIG and BTA

Monash is now looking ahead to what else they can do with Blue including implementing the Data Integrity Gateway (DIG). An integrated solution, DIG offers the university an effective way to clean and maintain high quality data.

“DIG will ensure that the owners of the data are able to provide timely and accurate updates and validation,” said Professor Skrbis. “Once it is set up and we’ve managed the business change it will relieve University Planning and Statistics of a lot of the manual labour.”

“We’ve also started to consider Blue Text Analytics because the Academic Board wants to start looking at a thematic analysis of the qualitative data,” he continued.

The University is also excited about the new possibilities for the assessment of the learning experience across the institution. In the past, student data was only sent to the educators and some levels above that. The school was not using data to inform where there were issues particularly around space utilisation and planning, scheduling, or the quality of their Learning Management System.

Now they are looking at ways to use Blue to create reports that will provide valuable insights to Service groups that have responsibility for aspects of the student learning experience.

Looking ahead together

Since transitioning to Blue in late 2016, Monash has made significant efficiency savings in their survey fieldwork operations. The University now has a platform that can cater to the customisation requirements of a large and complex institution, while providing a streamlined and consolidated platform for the sharing and distribution of student insights.

“We wanted to partner with someone who had the same philosophy as us: continuous improvement, knowledge transfer, and greater efficiency for organisations,” concluded Kristina.

“At every step of the way, the Explorance team was able to provide a comprehensive and exemplary level of service and expertise – demonstrating their commitment to their platform and an in depth knowledge of higher education business and system environment – with a clear roadmap and vision for their company. We see Explorance as a true partner in our development.”

“It was the only platform capable of meeting the complex business and reporting requirements of Monash, as well as offering significant capabilities to further expand Monash’s ability to enhance its continuous improvement objectives.”

Kristina Kaulenas,
Manager Information Strategy and Survey