HP Labs

HP Labs research helps cement the connection between the virtual and the real

By Simon Firth, HP Labs Correspondent — October 20, 2017

HP Labs researcher Sarthak Ghosh

HP Labs researcher Sarthak Ghosh

“In the future, people are going to spend a lot of time in virtual reality environments,” suggests HP Labs researcher Sarthak Ghosh. And they won’t just be using VR for entertainment. “VR will also become a key tool for employees working in fields as diverse as engineering, healthcare, media production, and space science,” Ghosh says.

That begs a question Ghosh first tackled while interning at HP Labs in 2016 as a masters student in Human Computer Interaction at Georgia Tech: how can we ensure that people working in VR environments keep track of what’s going on in the real world, of having a sense of passing time for example?

“If you are making a VR game, you don’t mind if your users are so engrossed in it that they lose track of time,” Ghosh observes. “But if you want people to use VR to do a job, they also need to attend meetings, write up reports, talk with colleagues and more.”

One solution would be to put a real time clock in the VR display that users see. But that takes up valuable visual real estate and taxes a human sense – vision – that is already being worked hard in such a visually immersive environment.

Instead, Ghosh decided to explore using haptic feedback – creating physical sensations with small motors – to offer clues about what’s going on outside the VR experience. Traditionally, haptic feedback has been deployed to make VR feel even more immersive. But could different types of haptic feedback also strengthen our feelings of connection to the outside world?

To find out, Ghosh built a series of five ‘haptic backpacks’ to be worn along with a VR headset. Inspired by HP’s own Omen VR Backpack, which makes it possible to create “untethered” VR experiences, each of these backpacks was augmented to deliver a different kind of physical nudge to users immersed in a virtual reality task. One backpack created the sensation of a shoulder tap at regular intervals to mark the passage of real world time, another buzzed at the shoulder, while a third buzzed the entire back. The fourth backpack created a “hugging” sensation and the final pack used small fans to blow air across the wearer’s neck.

Trials on colleagues in HP’s Immersive Experiences Lab quickly revealed that the hugging and blown air solutions didn’t give clear enough external signals. But the first three showed promise. Ghosh led efforts to test these other forms of haptic feedback on a larger group of participants as they undertook two different VR tasks.

“Perhaps our main finding was that people did notice the alerts they were getting and for the most part they were able to connect that with the real world, so it does seem possible to use your body’s surface area to create notifications about the real world,” says Ghosh.

The study also revealed a discrepancy between the intellectual calculations people make as they count buzzes or taps to measure time and their instinctual sense of how much time has passed. Many felt more inclined to believe their less reliable instincts over their more accurate counts, offering a useful window on the dominance of our instinctual sense of time in VR environments.

Alex Thayer, Chief Experience Architect for the Immersive Experiences Lab

Alex Thayer, Chief Experience Architect for the Immersive Experiences Lab

In addition, participants reported a strong inclination to believe that the physical sensations they were experiencing had a significance in the virtual world.

“If we can get a better handle on all of these things, it could help make for a better VR experience itself as well as letting us send clearer signals from the outside,” notes Alex Thayer, Chief Experience Architect for the Immersive Experiences Lab. 

On the issue of external notifications, the study suggested multiple areas for further analysis, such as the best patterns to use for signaling and the degree to which priming participants with information about what to expect can impact outcomes.

After completing his initial research, Ghosh returned to Georgia Tech to finish his degree. The work on his thesis with adviser Gregory Abowd was inspired by the HP Labs study. On graduation, Ghosh was hired into HP Labs as a full time researcher in the Immersive Experiences Lab so he could continue his explorations.

“One of our next steps is to ask how we can apply what we’re learning in these studies to future iterations of VR interaction and design,” Ghosh says.

That will help HP’s Immersive Experiences Lab further its goal of helping people achieve “supernatural productivity” – productivity far beyond what’s currently possible.

“We see VR as one of the technologies most likely to both disrupt and enhance how professionals do their work in the next five or ten years,” adds Thayer. “Research like this helps us anticipate that moment by enriching our understanding of what it will take to have VR be a major part of our work lives.”

Addendum - The haptic backpack project was a collaborative effort with other members of the Immersive Experiences Lab, including Hiro Horii, Kevin Smathers, and Mithra Vankipuram.