Fieldwork among the pixels: Virtual and augmented reality diversify geoscience education

A photosphere view of a glacial erratic featured in a virtual reality geology field trip to Columns of the Giants in the Sierra Nevada of California. Credit: Ryan Hollister. A photosphere view of a glacial erratic featured in a virtual reality geology field trip to Columns of the Giants in the Sierra Nevada of California. Credit: Ryan Hollister.

Weeks after taking his students on a field trip to the Sierra Nevada, Ryan Hollister, a high school earth science teacher in Turlock, Calif., led the school hiking club on an afterschool trek to a stretch of nearby river — the same river they’d seen on the earlier field trip, in fact, just farther downstream.

On the hike, one of the students walked alongside Hollister. She and her family are refugees who, after arriving in the country, were relocated to Turlock. Curious and always eager to perfect her English, she started making observations during the trip and peppering Hollister with questions.

“Look, Mr. Hollister, look at this rock: it’s rounded,” she noted. “What does that tell you?” he asked. “It’s been in the river; it’s tumbled. Is this a volcanic rock?” “Why do you think that?” “Well, look at all the bubbles in it.” “You got it!” “So, there was a volcano nearby? Where?”

Hollister smiles with pride while recounting the moment. “She put it all together,” he says. “I really wanted to get them observing and using their critical thinking skills — talking with each other and talking with me.” Seeing his student use the skills and knowledge she had learned on the earlier field trip was a great moment for Hollister — he knew he had made an impact, he says.

Group field trips and fieldwork have long fostered community learning environments in geology, which can help students expand their individual understanding through information sharing. Standing near a rock formation, making observations and proposing different hypotheses — perhaps with a few arm-waving ideas thrown in — is a time-honored tradition in geology. Discussing and arguing about how to test those hypotheses can even reveal new geologic insights and discoveries.

But there was a twist to Hollister’s class field trip: He didn’t guide his students while standing beside actual outcrops. Instead, the trip unfolded within the walls of his classroom. He took his students on a geologic journey using computers and virtual reality instead of multi-passenger vans.

In this composite photograph (upper left), Ryan Hollister snaps photos of a glacial erratic (see opening image) at Columns of the Giants from multiple angles. The many overlapping images of all sides of the rock were then stitched together using specialized software (lower left) to create the three-dimensional photogrammatic image of the boulder, which students could examine on their virtual field trip. Credit: both: Ryan Hollister. In this composite photograph (top), Ryan Hollister snaps photos of a glacial erratic (see opening image) at Columns of the Giants from multiple angles. The many overlapping images of all sides of the rock were then stitched together using specialized software (bottom) to create the three-dimensional photogrammatic image of the boulder, which students could examine on their virtual field trip. Credit: both: Ryan Hollister.

Advances in imaging and computing technologies have increased accessibility in the geosciences in the last few years, with virtual and augmented reality field trips and digital imagery spurring innovative teaching and creating more inclusive access to the wonders of the planet. It is no longer necessary to wrangle dozens — or hundreds — of students into vans and lead them through field sites. Today, whole outcrops can be brought into the classroom.

Advocates of applying such technologies, particularly virtual and augmented reality, to bring the field into the classroom say the practice can open doors to underrepresented students, including those faced with socioeconomic pressures, logistical complications or disabilities that might otherwise preclude their participation. These tools also offer new ways of introducing fieldwork, allowing students to take their time with the process while also strengthening a diverse science learning community.

Technology in Turlock

Nestled within California’s Central Valley, Turlock is a farming community with an international twist. The city of 70,000 is diverse and has been through much of its history: In 1930, for example, 20 percent of the population was Assyrian, hailing primarily from northwestern Iran.

Following the trend of the last century, Turlock continues to welcome newcomers. Since 2011, almost 2,000 refugees from around the world have made it their new home. Although many know at least some English, about 40 percent speak a non-English native language. Children enroll in public schools where many students are designated English Language Learners (ELL), polishing their second language while they learn multiplication tables and U.S. history. These students eventually make their way to earth science classes at one of two high schools in Turlock, where they are likely to have a teacher named Hollister.

Ryan Hollister and his wife, Laura, are both earth science teachers, though they work at rival high schools in Turlock. A few years ago, Ryan started thinking about how to expand his teaching methods, inspired by educators he followed on Twitter. They were sharing examples of photogrammetry and photospheres, in their classrooms.

Photogrammetry is a photography technique that captures multiple, high-resolution images of an object — on scales ranging from a roadside outcrop to a hand sample — to create a three-dimensional image so detailed that it can be used to make fine measurements and observations. Photospheres are 360-degree panoramic images that approximate standing in one location and looking around in every direction. Photospheres are widely used in Google Street View, for example, where they allow users to view a particular location and look around to see what is next door or across the street.

The imagery intrigued Ryan. “I put it in the back of my head for a rainy day, thinking, ‘Oh, that’s pretty cool,’” he says.

In 2016, when the “Science Friday” radio show put out a call to educators to create a STEM-themed (Science, Technology, Engineering and Math), multimedia-driven lesson that could be used in classrooms around the country, Ryan wanted to use the photosphere and photogrammetry technologies he had been eyeing to create a virtual geologic field trip for his students. He was one of a handful of educators selected to be a part of the Science Friday Educator Collaborative.

Hollister created multiple stops (blue markers) in his virtual field trip, just as there would be in a traditional geology field trip. In the simulation, each stop featured open-ended questions that students had to answer after exploring the location. Hollister calls the approach “facilitated learning,” and says it helps his students think critically about what they’re observing. Credit: Ryan Hollister. Hollister created multiple stops (blue markers) in his virtual field trip, just as there would be in a traditional geology field trip. In the simulation, each stop featured open-ended questions that students had to answer after exploring the location. Hollister calls the approach “facilitated learning,” and says it helps his students think critically about what they’re observing. Credit: Ryan Hollister.

Laura joined the project and they selected a field site, Columns of the Giants near Sonora Pass in the Sierras — a geologic smorgasbord filled with columnar basalts and glacially polished surfaces. It’s just a couple of hours upstream from Turlock on the Stanislaus River, which the school hiking club had visited. The location was chosen for a purpose: familiarity and access. “Some of the students have been up to Sonora Pass and they know that place,” Laura says. “When they see it’s close to home, they realize … it’s not some distant place in another part of the country that they might never see.”

In Hollister’s virtual field trip, students were encouraged to sketch and take notes about what they saw at each stop in the virtual field tour, similar to what they would do in the field. Credit: Ryan Hollister. In Hollister’s virtual field trip, students were encouraged to sketch and take notes about what they saw at each stop in the virtual field tour, similar to what they would do in the field. Credit: Ryan Hollister.

Ryan spent the summer of 2016 photographing the field site. Even after the time-consuming field photography was done, learning the software programs to create the imagery took many nights and weekends during the school year. Ryan edited and stitched together the photos to create imagery for six separate “field stops” within the virtual tour of Columns of the Giants. The technology and methods take some expertise to utilize, Ryan says. “It’s not as plug and play as it could be,” though he adds that, once mastered, creating high-resolution images for virtual reality applications is relatively quick.

He created both photogrammetric and photospheric images of the field site: photogrammetry for stationary objects like glacial erratics or outcrops, and photospheres at each “stop” to allow students to see their surroundings in 360 degrees.

Along with imagery at each field trip stop and outcrop, Hollister produced an accompanying field guide to help teach students about what they’re viewing. “It’s facilitated learning,” he says, adding that the field guide and associated questions help students research details they need to know to uncover the geologic history of the area. The virtual field trip to Columns of the Giants is free and available through Science Friday.

Gaming Geology

Taking a cue from computer gaming, Jacqueline Houghton of the University of Leeds in England and her colleagues created a virtual field trip of local geologic sites in game form, complete with animations of landmarks and wildlife. Houghton included a virtual field notebook where students can learn relevant details about the rocks at specific outcrops. Credit: all: Jacqueline Houghton/Virtual Landscapes - Geoscience Education in Digital Environments. Taking a cue from computer gaming, Jacqueline Houghton of the University of Leeds in England and her colleagues created a virtual field trip of local geologic sites in game form, complete with animations of landmarks and wildlife. Houghton included a virtual field notebook where students can learn relevant details about the rocks at specific outcrops. Credit: all: Jacqueline Houghton/Virtual Landscapes - Geoscience Education in Digital Environments.

Creating a virtual field trip with photogrammetric and photospheric imagery is one way to take participants into the field, moving directly from stop to stop. More challenging, however, is using technology to mimic the type of free movement through a landscape that a student might experience, for example, at field camp. In that setting, geologists-in-training are often turned loose for the day to conduct independent field mapping, a task that requires them to make their own decisions about where to go and what outcrops to investigate. Jacqueline Houghton, a structural and field geologist at the University of Leeds in England, wanted to recreate that experience in the classroom, so she looked to gaming technologies for inspiration.

Houghton and her colleagues created a virtual, animated landscape of three field sites to render a three-dimensional version of the world where students could “walk” around in the landscape, like in a first-person video game. “The students get a blank map that shows topography, and they walk around and work out where they are,” noting rock types at different locations.

It’s the little details the team included in the simulation that catch students’ attention. The animated world includes hidden surprises, like a dead sheep (common in the rural U.K. countryside) and the occasional sounds of a Royal Air Force flyover. Houghton says these touches entertain and intrigue students, and provide the same sorts of “distractions” that students might encounter on traditional field trips.

Houghton notes that the virtual trips replicate the same struggles and challenges for students that a traditional trip can provide — for example, missing key observations, trying to finish quickly, or not quite seeing the big picture. “What we see is [students] make the same mistakes” in the virtual field trip as they do in a traditional trip. She adds that the virtual reality (VR) trip helps her drive home lessons about the geology of an area to students. “In the real world, I can’t [always] send them back over to an outcrop,” but in the VR world, students can revisit an area again and again to collect details they might have missed.

Often, students who can’t attend traditional field trips, perhaps because of disabilities that affect their mobility, are given alternative exercises like essays, which deprives them of a field experience, Houghton says. But on VR trips, she adds, everyone experiences the same trip in the same way.

Beyond Pokémon: Augmented Reality

In 2016, it was hard not to run into gaggles of kids — and sometimes adults — wandering around outside and huddling over their smart phones in seemingly random locations. The reason for this peculiar phenomenon was the sudden popularity of Pokémon Go — a game that layers animations of characters and other features from the cartoon Pokémon over real-life images of streets, parks and other public spaces on one’s smart phone.

The game is an example of a technology called augmented reality, or AR. Like Pokémon Go, AR applications often involve artificial animations that appear atop images of real places on a screen.

Pokémon Go uses augmented reality technology to layer animations of characters and other features over real-life images of streets, parks and other public spaces on smart phones. Credit: Matthew Corley/Shutterstock.com. Pokémon Go uses augmented reality technology to layer animations of characters and other features over real-life images of streets, parks and other public spaces on smart phones. Credit: Matthew Corley/Shutterstock.com.

Beyond entertainment, AR technology has been taking off in advertising, retail and military applications, but it can also be used for education. Art and natural history museums have integrated the technology into exhibits, for example, to encourage interaction and engagement of visitors with displays.

AR has also made it into the classroom. Kent Hups, a teacher at Northglenn High School in Northglenn, Colo., has been using AR in his classroom to bring geologic maps to life. With an app called HP Reveal, students use their phones to scan geologic maps, and the app can then display pop-up images of rock outcrops or three-dimensional hand samples on top of the map.

Hups concentrated on a few locations in Colorado with distinct geologic features and used a geologic map of each spot. “I started to add depth to the maps,” he says, explaining that in a 1:500,000 map showing the entire state of Colorado, details and subtleties of geologic features can get lost. But by adding photos of locations, students can see rocks, faults, unconformities and other geologic formations in three dimensions, which brings them to life.

He’s been experimenting with broadening his AR-enriched lessons to include cross sections and hand samples as well. Using more detailed 7.5-minute (1:24,000) quadrangle maps, a standard scale of topographic maps produced by the U.S. Geological Survey (USGS), students “can scan [part of a map] with their phone and the overlay will be a picture of that very location.” Numbers associated with different scannable sites on the map also correspond to bins in his classroom that contain hand samples of the rocks at each site. “I can actually get the rocks in their hands from that location,” Hups says. “They can see it and touch it without leaving the classroom.”

Using AR in the classroom has increased accessibility for Hups’ ELL students, he says. For example, Spanish-speaking students can scan part of the map and have the lesson delivered to them in their native language. “It’s a huge leap in teaching because a lot of times we have kids that are disengaged if they don’t understand,” he says. “But now I can take them on a tour of this geology in a language they understand.”

Giga-Imagery Zooms In

New technologies have helped make VR and AR a possibility, but great imagery is a huge part of the puzzle when it comes to recreating landscapes. Over the last decade, the advent of extremely high-resolution imagery, often called giga-imagery, has opened many new avenues for geology education and communication. Unlike the megapixel pictures we take with our digital cameras or phones that have millions of pixels, giga-imagery pictures are composed of billions of pixels. This very high resolution can capture the incline of sand beds within a 100-meter-long outcrop — all within the same image — allowing students to see both the fine details and the big picture.

GIGAmacro, a company that has made a name for itself by developing tools to produce such detailed images, is putting a new spin on the integration of technology into geology. Gene Cooper, the founder of GIGAmacro, started his foray into giga-imagery by taking 360-degree photographs in national parks. He also worked with museums around the country that expressed interest in having very high-resolution images of their collections and samples. After some experimentation with equipment and some creative computer programming, the company created a way to photograph in incredible detail and then allow users to view objects in a fast and user-friendly way.

GIGAmacro’s equipment allows users to take overlapping photos of objects — whether a hand sample of rock or an insect, for example — in both vertical and horizontal dimensions. Once stitched together into a single image, the resulting giga-image is seamless and zoomable, allowing the viewer to see minute, sometimes microscopic, details without any delayed loading. The technology has a definite “wow” factor.

But the images can be used for much more than just demonstration. Scientists, educators and curators can scan a geologic sample or outcrop and upload it to GIGAmacro’s website, or they can use images already available on the site. They can then have students annotate what they see right on the image. With a free GiGAmacro account, students can save their work, or even add to an image as a class.

In Kent Hups’ earth science course, students at Northglenn High School in Colorado use augmented reality to make geologic maps come to life. An app on their smart phones displays videos and photographs of geologic sites around the state. Credit: Kent Hups. In Kent Hups’ earth science course, students at Northglenn High School in Colorado use augmented reality to make geologic maps come to life. An app on their smart phones displays videos and photographs of geologic sites around the state. Credit: Kent Hups.

One example of an image available on GIGAmacro’s website is of a slab of fossiliferous sedimentary rock from the Stonerose Interpretive Center and Eocene Fossil Site in Republic, Wash. This particular sample is chock full of fossils ranging from parts of insects to leaves, each of which was identified with virtual push pins by Stonerose scientists, who included as much detail about the remains as possible, allowing students to study the rock and fossils in minute detail.

There are also images of rock thin sections, which allow students to view and interpret mineral grains, textures and fine-scale features. The company is always working to improve the tools available to users, Cooper says. “We would love to connect more with geologists and other scientists about what tools would make their jobs easier,” he says. “How can we help them use this in a classroom setting?”

In the last several years, Callan Bentley, a geologist at Northern Virginia Community College (and a contributing editor of EARTH) and his students have amassed more than 2,000 high-resolution giga-images, on scales ranging from whole outcrops to thin sections. Bentley formed the Mid-Atlantic Geo-Image Collection (MAGIC) on GigaPan, showcasing their work and creating a place to share resources with other educators and students.

The images were made freely available to provide access and opportunity, Bentley says. “Some of these sites that we’ve made imagery of are in places that are out of the way,” he says. “Students in Virginia can access outcrops in South Africa and then be back at their desks again in 45 seconds.” He adds that this imagery can also provide access to people who might not, for whatever reason, be able to reach mountaintops, valleys, outcrops or otherwise remote, or inaccessible, field sites.

Another Tool in the Toolbox

Giga-imagery of a basalt thin section at 10 times magnification: On the GIGAmacro website, users can compare two images of the same thin section using both waveplate and brightfield lighting, just like you would be able to do with a petrographic microscope. Credit: GIGAmacro. Giga-imagery of a basalt thin section at 10 times magnification: On the GIGAmacro website, users can compare two images of the same thin section using both waveplate and brightfield lighting, just like you would be able to do with a petrographic microscope. Credit: GIGAmacro.

Fieldwork is often thought of as integral to geologic study, so being unable to participate in fieldwork has long been a barrier to the discipline. Sometimes it is familial or work obligations that get in the way of a student’s ability to participate, and differently abled students have also traditionally been precluded from most field experiences, says Christopher Atchison, a faculty member in geoscience education at the University of Cincinnati and executive director of the International Association for Geoscience Diversity (IAGD).

About a decade ago, Atchison started exploring how to tackle issues related to ableism and fieldwork. He created a class geared toward students with physical disabilities — the plan was to use VR for a culminating field trip to Mammoth Cave National Park. But he decided that it would be good to first get the students into Mammoth Cave during the course, so he could later compare the field and VR experiences to help develop how to structure future classes.

The students in the new course were just like any other group of future rock hounds, Atchison says. During one class session, they focused on the basic geology of cave and karst formation so the students would be prepared for their visit to Mammoth Cave. “They were sponges,” he says, very eager to learn and experience a field trip in person.

Their visit to Mammoth Cave did not disappoint. “None of them were science majors, but they were just glued to this whole experience because they’d never been given this opportunity before.” He says he realized that VR wasn’t the only answer for addressing barriers to accessibility in geology.

The experience solidified Atchison’s view that fieldwork should be open to everyone. “The enjoyment and excitement that I shared with these students … completely changed my focus and direction,” Atchison says. He now advocates for accessible and inclusive field trips, and has written about his efforts in EARTH.

Imagery of this fossiliferous slab of sedimentary rock from the Stonerose Interpretive Center and Eocene Fossil Site is available on GIGAmacro’s website. Fossils in the slab were identified and annotated so interested users can hover over each pin and learn about each preserved organism. Credit: GIGAmacro. Imagery of this fossiliferous slab of sedimentary rock from the Stonerose Interpretive Center and Eocene Fossil Site is available on GIGAmacro’s website. Fossils in the slab were identified and annotated so interested users can hover over each pin and learn about each preserved organism. Credit: GIGAmacro.

Most proponents of VR and AR in earth science classrooms do not suggest that it should replace field experiences altogether. Instead, they see the technologies as educational tools that can be leveraged in many different ways. “You can’t see the subtlety of information” and teach observational skills with the same level of depth using VR, Houghton says. But using the tools can “prepare students for fieldwork” in the real world and increase the diversity of people being exposed to geology.

The technologies can also help manage large student groups and, ultimately, create more inclusive learning communities. “One of my classes is EL [English language]-specific, and right now I have seven languages [represented] in that class,” Hollister says. Those students have found ways to share their ideas with the help of Google Translate and by sharing sketches, he says. This unique collaboration is facilitated by VR technology, allowing them to communicate in the comfort of their classroom — a more conducive setting for a diverse group of students compared to the outdoors. “They were conversing and making discoveries and learning things about their new home — it’s really cool.”

Sharing information among peers, proposing arm-waving hypotheses, and engaging in inevitable arguments about whether a rock is granite or diorite are experiences at the heart of learning geology, and in the case of Hollister and other earth science instructors, VR is helping grow communities of students interested in doing just that.

Sarah Derouin

Sarah Derouin is an EARTH editorial intern.

Derouin is EARTH's acting associate editor. Before becoming a science writer, Derouin earned a Ph.D. in glacial geology from the University of Cincinnati and worked for the Bureau of Reclamation in seismic hazards and geomorphology. She is a graduate of the science communication program at the University of California-Santa Cruz. You can see more of her work at www.sarahderouin.com.

Wednesday, May 23, 2018 - 17:00

Did you know ...

EARTH only uses professional science journalists and scientists to author our content?  In this era of fake news and click-bait, EARTH offers factual and researched journalism. But EARTH is a non-profit magazine, and at least 10 times more people read EARTH than pay for it. As advertising revenues across the media decline, we need your help to ensure that we can continue bringing you the reliable and well-written coverage of earth science you know and love. Our goal is not only to inform our readers, but to inform decision makers across the economic and political spectrum about the science of our planet. So, we need your help. By becoming a subscriber or making a tax-deductible contribution to support EARTH, you can fund our writers and help make sure the world knows about our planet.

Make a contribution

Subscribe