In this project, we explored how to add haptics to walls and other heavy objects in virtual reality. Our main idea is to prevent the user’s hands from penetrating virtual objects by means of electrical muscle stimulation (EMS). The figure below shows an example. As the shown user lifts a virtual cube, our system lets the user feel the weight and resistance of the cube. The heavier the cube and the harder the user presses the cube, the stronger a counterforce the system generates. The figure below illustrates how our system implements the physicality of the cube, i.e., by actuating the user’s opposing muscles with EMS.

In fact, when the user grabs the virtual cube, the user expects the cube’s weight to create tension in the user’s biceps and the cube’s stiffness to create a tension in the user’s pectoralis. In order to create this sensation, the system actuates the respective opposition muscles. In order to put a load onto the user’s biceps, it actuates the triceps and in order to put a load onto the user’s pectoralis, it actuates the user’s shoulder muscle. This creates the desired tension in biceps and pectoralis, thereby creating the desired experience.

As illustrated in the figure below, our system stimulates up to four different muscle groups. Through combinations of these muscle groups, our system simulates a range of effects. When pushing a button mounted to a vertical surface, for example, the system actuates biceps and wrist. In the Example Widgets section of our publication at CHI'17, we detail how this allows our system to simulate a wide range of objects, including walls, shelves, buttons, projectiles, etc.

Our system can be worn in a small backpack, as shown in the figure below. The backpack contains a medical grade 8‑channel muscle stimulator, which we control via USB from within our VR simulators. We use our system in the context of a typical VR system consisting of a head-worn display (using a Samsung/Oculus GearVR) and a motion capture system

The figure above illustrates the naïve approach to rendering objects using EMS: (a) From the moment the user’s fingertips reach the virtual wall, we actuate the user’s hand just strongly enough to prevent it from passing through. We achieve this with a current essentially proportional to the user’s force (further details in Implementation).

When we built this version, the results looked great. The design prevents the user’s hand from passing through the object and thus bystanders observing the scene would typically conclude that the illusion was “working”. However, during piloting, it became clear that this design did not work. Since the EMS actuation was as long and as strong as the user kept pushing, the EMS signal (a tingling in the respective muscles) could become arbitrarily strong. This would draw the user’s attention to the EMS-actuated muscles. These, however, were pointed in the wrong direction, i.e., they were pulling, when the sensation was supposed to be about pushing. One participant in our pilot said this design felt “like a magnet pulling the hand backwards”. While this design was reasonably impermeable, consistent, and definitely familiar, the strong EMS signal made this design fail with respect to our primary objective: it was not believable.

We, therefore, created two alternative designs with the objective of increasing believability:

The soft object design (Figure above left): We created our first alternative design by applying a cut-off to EMS intensity. We picked a reasonably low cut-off, allowing users to penetrate objects by about 10 cm. This resulted in a design that produced the impression of soft objects. Based on this general concept, we explored various visuals, including the soft surface material shown in the cube above, which is designed to suggest an increasingly solid inside under a soft, permeable surface. This general design became the basis for most of our object designs. For example, the figure above shows the same concept wrapped in visuals suggesting a magnetic field, suggesting a magnetic force that carefully pushes the user’s hand backwards. In some versions of this design, we attached a block of metal to the back of users’ hands to suggest that the magnetic field would apply there in order to affect the hand.

The repulsion object design (Figure above right): We created our second alternative design by reducing the duration of our EMS signal. This resulted in what we call repulsion objects. This design uses a brief EMS pulse (of 200-300 ms, using the user’s calibrated maximum intensity) where the EMS propels the user’s hand backwards, removing it from the virtual object it is trying to touch. We achieve this with an EMS pulse of still reasonably low intensity, which, like all other EMS signals in our system, is pain-free at all times (for EMS pulses of similar intensity see our project Impacto).