Institute of Technology Masters Complex Survey Cycles with Blue

Manukau Institute of Technology (MIT) (Māori: Te Whare Takiura o Manukau) is a Crown-owned tertiary education institute based in Auckland, New Zealand. As one of the largest institutes of technical and vocational education in the country, they provide a wide range of academic qualifications from certificate to degree level. The institute recently broke ground for a new facility that will house New Zealand’s only air conditioning and refrigeration school, its largest industry training school for plumbing, the biggest polytechnic-based electrical trades school, and a new site for building and civil construction, engineering and automotive trades training.

Complex Survey Requirements: a Technical Challenge

As a publicly funded technical institute educating over 14,000 students a year, MIT has complex needs for gathering student feedback.

New Zealand is also going through a ‘once in a generation’ change in the way it organizes and delivers vocational education. As of April 2020, the Institute joined together with the 15 other institutes of technology and polytechnics in the country to form a new institution. As a member of this new organization, MIT has several student survey reporting requirements they must comply with.

Some of these requirements include capturing data from priority learner groups, such as Māori, Pasifika, under 25s, and people with disabilities. The Institute has specific reporting needs for these groups as part of their mission to provide an equitable learning experience. Surveys must also reach students without online access, such as students in prison or other areas where internet access is limited.

Second, MIT must run student surveys for a highly complex academic calendar, where courses run anywhere from eight weeks to a full year. “We offer more than a dozen different delivery time frames, with rolling starts and finishes throughout the year,” says Chris Park, General Manager of Academic Services at MIT.

All of this must happen within a lean staffing structure. As a 100% state-funded institution, MIT are accountable to the public for their expenditures.

Manual Processes Slowing Them Down

The institute used to conduct course surveys using very manual processes on Survey Monkey. Without integrated systems that gathered data systematically in a central repository, analysis was near impossible without a huge amount of labor from staff. “We recognized that we weren’t able to collect learner feedback in a systematic, all-encompassing way that would allow us to draw conclusions and improve our learning and teaching performance,” says Park. “We had limited ability to capture learner impressions, and conduct comparisons across schools and across the institute as a whole.”

MIT chose Blue Course Evaluations from Explorance. To meet all the Institute’s unique requirements, they implemented a hybrid course evaluation solution that allowed them to run paper surveys for the students who needed it while rolling out the online solution to the majority of the student population. This was no small feat while managing a complex web of course timelines. Further, the solution had to meet the stringent federal privacy and accessibility standards, including Web Content Accessibility Guidelines 2.1 for their visually impaired students.

“Even on the day we went live, we had surveys going. And they continued to go out all the way through the process of close out,”

Chris Park, General Manager of Academic Services at MIT.

Best-In-Breed Course Evaluations For The Institute

The new system went live in April 2021. Explorance provided support to integrate Blue with their Canvas Learning Management System (LMS), so that MIT could automate survey creation as much as possible and eliminate the need to transfer student and course information data files to and from Blue. The integration also allowed for pop-up reminders from Blue to appear in Canvas, for any outstanding survey forms a student had yet to complete. Staff members can now login to Blue to monitor response rates for each course’s evaluations.

Five months later, the institute has successfully sent over 17,000 surveys to nearly 6,000 students, providing scanning of paper forms where necessary and adding that data to the central repository.

“We had a very smooth rollout,” says Harry Hood, Project Manager at MIT. “It’s been a pleasure to deal with the team at Explorance.”

The implementation took place completely remotely, with Explorance professional consultants available in the nearby Australian time zone for any required changes or issues that arose.

“Explorance is obviously quite accustomed to running these kinds of engagements using solely online meetings,” says Park. “For us, this was a relatively new way to function – we’re more accustomed to in-person implementations. But Explorance made this very easy, and they were very flexible in accommodating our particular situation.”

“Honestly, I have not seen this kind of accommodation and comfort from any other software teams that I’ve worked with in a software implementation,”

Chris Park, General Manager of Academic Services at MIT.

Improved Processes Accompany The New Technology

The new technology rollout provided an excellent opportunity for the institute to update their course evaluation questions while they introduced the new system school-wide. “Changing the course survey questions took a significant piece of collaboration across the institute,” says Park. “The end-result is much improved over the previous survey. We’re getting a lot of positive feedback on them.

“We think the combination of the new system with the new questions is the reason for that feedback,” she adds. “The admins are of course ecstatic that they no longer have to administer paper surveys or the online manual surveys.”

Another improvement they’ve seen is around the institutional use of their Canvas LMS. It was when the Academic Centre team started implementing the integrated, centralized survey system that they discovered that some of the teaching staff were using work-arounds in Canvas, which were inadvertently affecting the student experience of Canvas as a whole.

“Between the push for more online content because of COVID restrictions, and the Blue implementation, we’ve been able to have really meaningful conversations around how staff are using Canvas, and introduced some disciplines around the LMS to prevent work-arounds that reduce the learner experience or engagement,” says Park. “We now have a much, much higher student uptake of Canvas, which is a great outcome.”

Using Blue For Other Surveys

The institute is already capitalizing on their investment by using the solution to meet some of their other requirements, in addition to their own course evaluation surveys. Every year, MIT is required to run surveys that are part of a national set of consistency evaluations run by the New Zealand Qualifications Authority (NZQA). “One of the challenges we’ve had is getting learner feedback for the NZQA survey, as it looks back over a five-year period” says Park. “We are very happy to be using Blue to manage this survey now, and focusing more of our efforts on reaching out personally to students, especially those ones that are harder to track down.”

The Academic Centre isn’t resting on its laurels, and is already looking ahead to what they can achieve with the new system. “Thinking about the reporting we’ll be able to do with the new data is quite exciting for us,” says Park. “We think of Blue as like a set of stackable, evaluative and reflective tools, and our task now that we know the system better and its many capabilities is to think hard about how we can best use them.”

Whatever they decide, Explorance remains an important partner for the school as they continue to build for their future. “Working with Explorance was unlike any other software implementation I’ve done,” Park says. “It was positive, it was engaging, and ultimately delivered everything we were looking for.”

“Working with Explorance was unlike any other software implementation I’ve done,”
“It was positive, it was engaging, and ultimately delivered everything we were looking for.”

Chris Park, General Manager of Academic Services at MIT.