Virtual reality helps scientists, soldiers tackle real-world problems.


The idea of a holodeck -- an all-powerful virtual reality room where physical and digital worlds meet seamlessly -- has long been the stuff that sci-fi geek dreams are made of.

But now, taking a cue from the popular television series "Star Trek: The Next Generation," researchers at two US laboratories have developed advanced, immersive, virtual reality (VR) systems to help both scientists and soldiers tackle real-world problems.

See hard science

At the Center for Image Processing and Integrated Computing (CIPIC) at UC Davis, researchers are using virtual reality environments to bring complex scientific data and esoteric concepts down to earth.

"The goal is to put the scientist into his data, and allow him to make decisions about his data in a real-time manner," CIPIC scientist Ken Joy said. "It's about making the invisible visible."

The CIPIC virtual reality lab is currently equipped with an immersive workbench, which projects 3D images onto a tilting table. Wearing goggles and special gloves connected to the computer, researchers can reach "into" the workbench, pick up virtual objects, and move them around.

According to Bernd Hamann, co-director of CIPIC, virtual reality is ideal for scientists and engineers who need to assess large amounts of data -- for example when designing automobiles or checking the stability of a bridge. Seeing a 3D representation of the data puts it all into perspective, he says.

"Virtual reality opens up completely new avenues for the interactive exploration of very large data sets, medical data sets, neuroscience data sets, and data sets generated by sensors," Hamann said.

Explore the human brain in 3D

One of the major uses of the CIPIC virtual reality workbench is in the medical field. The researchers have taken thousands of pictures detailing different "slices" of the human brain and layered them to create a 3D virtual brain.

"The doctor can come in and look at the image, rotate it around, and get a very good idea of what the patient's brain can look like," Hamann said. "If he is looking at a brain tumor, he then can step in and look at how he could access it, where he would apply surgery equipment to make the least possible impact to the patient."

Eventually, the CIPIC lab plans to build a VR "cave," a room fitted with projectors generating 3D images on the walls, floor, and ceiling, which would allow scientists to literally walk around inside their data.

"My dream is that in the next 10 years here at the center we'll be able to see the real world and the virtual world at the same time and have them interact," Joy said. "This would allow people to carry on conversations with other colleagues and see their data in their own virtual space."

Virtual combat

At the University of Southern California's Information Sciences Institute (ISI), researchers have taken the concept of the "holodeck" experience one step further.

Blending advances in artificial intelligence software with VR technology, the researchers have created a 3D environment, complete with animated characters and sound, that takes US soldiers on a virtual mission in a troubled town in Bosnia. There, they must deal with a situation threatening to spin out of control.

Developed for the US Army, the mission rehearsal exercise uses a movie-theater-sized curved screen that looms around trainees. Combined with multidirectional, realistic sound, the screen images create a convincing illusion of being present at the scene, rather than observing a show.

In one simulation scenario, a lieutenant enters the village to deal with one problem -- a weapons inspection team being threatened by an angry crowd -- and finds another one as well: an American jeep has accidentally struck and injured a local child.

The scene is populated with animated figures created by basic robotic programs that carry on a limited range of pre-scripted, routine behavior -- such as milling around -- as well as more complex characters that can react to what a soldier does, even changing their expression in response to speech.

"The work we have done in one way shows how far away the holodeck is -- but in another shows how useful it may be," ISI project leader Jeff Rickel said. "The project represents a grand challenge for both AI and virtual reality, but the potential payoff is a powerful new medium for experiential learning."