Evaluating and Improving the Learning Systems Toolset: A Look Back at Academic Year 2024
Innovative technology is the backbone of the Ohio State University learning experience. The Learning Systems team within the Office of Technology and Digital Innovation (OTDI) provides learning tools such as the Carmen learning management system (LMS) and many more. The Learning Systems team is comprised of developers, trainers, administrators, analysts and support staff. They are responsible for managing and supporting the existing toolset, as well as exploring options for continual improvement.
A subset of Learning Systems, the Learning Systems Innovation (LSI) team routinely evaluates Ohio State’s current toolset, monitors its usage, and conducts satisfaction surveys. Data collected this way helps LSI know when it’s time to find a new tool that better suits the needs of students and instructors or to create a service improvement plan for an existing tool.
The LSI team recently compiled data for academic year 2024 (Summer 2023 – Spring 2024) from survey responses, service tickets, and usage data—and the results highlight instructor and student satisfaction with the Learning Systems toolset as well as the impressive impact these tools have on the teaching and learning experience at Ohio State.
Take CarmenCanvas for example. In academic year 2024, 71% of instructors agreed or strongly agreed that Carmen is easy to use—up from 49% in 2019! Instructors and students also said the tool is useful and does what they need it to do.
Within Carmen, Turnitin has been used to help deter academic misconduct. In academic year 2024, more than half a million assignments were analyzed (505,955). Instructors and students alike rated Turnitin as easy to use, so it’s no surprise service desk tickets were low for this tool.
Zoom is another invaluable tool for students, faculty, staff and even external audiences. In academic year 2024, Ohio State users hosted 624,168 Zoom meetings for a total of 2.6 million meeting hours.
As for recorded content, Mediasite usage is up compared to pre-pandemic numbers. In the 2024 academic year, 460,186 hours of content were viewed with Mediasite — more than 52 years! Instructors and students both rated the tool as “good,” but OTDI is working on continual improvements to the tool in hopes to achieve an “excellent” rating in future academic years.
Ohio State’s portfolio tool, PebblePad, is primarily used in the GE bookend courses, so usage tends to be higher in autumn and spring semesters. In summer 2023, there were 245 published assets (anything created or uploaded in PebblePad), which increased to 15,851 in autumn 2023 and 16,672 in spring 2024, for a total of 32,768 published assets for academic year 2024.
Another tool that reaches the majority of learners at Ohio State is Top Hat, a polling tool that can be used during class for quick response questions and attendance. In academic year 2024, 35,092 students were active in Top Hat, which is 53.65% of Ohio State’s enrolled students.
It’s clear the Learning Systems toolset is vital to Ohio State’s academic mission, and continual evaluation and improvement helps ensure students, faculty and staff have the most efficient and effective tools at their fingertips. You can learn more about the analyzed data from academic year 2024 and find out what’s next for the Learning Systems Innovation team.