Photogrammetry Helps to Create Immersive Learning Experiences

Instructional designers and researchers across the university are exploring the possibilities of immersive media for higher education. Immersive media includes virtual reality, augmented reality and mixed reality, and oftentimes includes 3D objects or full virtual environments. If you’ve played a video game with a virtual reality headset or experienced apps like Pokémon Go, you’ve interacted with immersive media. But there are so many possibilities for this technology beyond fun and games.

Dean Hensley, an instructional design specialist in the digital learning team in the Office of Academic Affairs, has collaborated with others at the university who are working on creating 3D objects. Hensley is currently working with Senior Researcher Jeremy Patterson from the Advanced Computing Center for Art and Design (ACCAD). Patterson is building a photogrammetry app called Mesh Fountain that, when paired with Apple’s RealityKit, makes it possible to quickly create 3D objects from photos. After taking hundreds of photos with an iPhone, Mesh Fountain can render a 3D object within 20-30 minutes, and this object can be used in any 3D program or even exported into a format that can be embedded in Carmen.

Mesh Fountain has the potential to significantly decrease immersive media’s cost to entry, which is an especially relevant concern in higher education. Patterson’s new app and the photogrammetry process also produce higher quality models than other inexpensive options on the market.

Hensley and Patterson have already been utilizing this technology to help the Bureau of Criminal Investigation turn photos of skulls into 3D images. In addition to helping solve crime, there are many useful applications for photogrammetry in higher education, particularly in online courses. Hensley said, “This technology has the ability to give students an experience they wouldn’t normally be able to have unless it was in-person.” For example, students in an online course can manipulate a 3D object by turning it, examining it, watching it move, highlighting portions of the object to learn terminology, and so much more.

Creating immersive environments can also benefit in-person classes by providing a safe learning environment in place of high-stakes or dangerous situations. For example, imagine building a radiation lab or a flight simulation so students can learn safely before diving into a real-life situation. Hensley said, “Students have a higher success rate when they learn in a safe, stress-free virtual environment before going into a dangerous or risky situation.” He added, “not to mention, there are major cost savings when using a virtual immersive environment to learn and practice.”

While it is clear that photogrammetry and immersive media can enhance the teaching and learning experience, it is also imperative that these innovative techniques are accessible for as many learners as possible. Hensley has been ensuring that any 3D objects embedded in Carmen have accessibility features built in—this includes audio prompts, the ability to tab through sections, the ability to use a keyboard in place of a mouse and even options to adjust highlight colors and increase contrast. An accessible PDF accompanies these 3D objects as well to ensure all information can be obtained via text.

Are you interested in exploring the possibilities of immersive media in your course? Are you working on similar solutions in your college or department? Email Dean Hensley for a consultation or to collaborate on future projects.