Manipulation Schemes for Dioramas
Virtual Reality (VR) technology offers an exciting new potential for people to experience everything from digitally reconstructed environments to entirely new worlds. While much of this potential is already on offer, most experiences provide little in the way of intuitive, comfortable, and compelling control over motion. While there’s a whole world of possibilities, it’s generally difficult to engineer controls in a way that meets these three criteria together. For example, flinging your arms to "swing" or "fly" yourself can move you a reasonable distance, but is both tiring and embarrassing in practice; translating real-life footsteps in virtual footsteps is also intuitive, but it’s difficult to take more than a few virtual footsteps before you hit your bedroom wall (that might change with the Quest though); and simulating objects to behave like real-life again makes sense, but feels like an underwhelming deployment of the capabilities of VR.
In a previous post, I described how we take a world-in-miniature approach to open up the potential for an intuitive, comfortable, and compelling control scheme. In that post, I talked about some of the problems that I came up against when implementing the scheme for the first time the Unreal Engine. Since then I’ve done more engineering work and our implementation is almost ready for production, with just a few more bug fixes and tweaks to go. Our next step is to present our control scheme in different configurations to a real-world audience. I’ll be running user tests and hoping to verify that our control schemes offer a next-level experience that is intuitive, comfortable, and compelling all at the same time.
I’d like to take a chance to reflect on a few of our configurations. In the rest of this post I’ll present each configuration, summarize its affordances and limitations, and give general comments. With the different configurations laid out here, I’ll come back in a future post to discuss what worked well and what didn’t and answer the ultimate question as to how control over motion and the miniature worlds of Minimum Mass can be linked to offer an entirely new story-telling medium.
The Grip and Trigger Buttons
A picture of the Oculus Touch controllers. Image retrieved from https://www.oculus.com/blog/get-ready-for-rift-s/, 28/06/2019.
In the descriptions below, we talk about grip and trigger buttons. In our current implementation, we use Oculus Touch controllers (shown above). For the grip button, we use the button on the palm grip (activated by the middle finger). For the trigger button, we use the button at the tip of the controller, sitting just under the main face of the controller (activated by the index finger).
Base Model (without constraints)
This scheme implements our "base" model, which combines Ware's Scene-in-Hand approach with Mapes & Moshell's Two-handed Manipulation approach. Scene-in-Hand manipulation enables the diorama to be both translated and rotated when the grip button is activated. When rotating, the diorama rotates around the hand gripping the button. Two-Handed Manipulation enables the diorama to be translated, rotated, and even resized when both hands activate the grip button. When rotated and scaled, the diorama pivots around the point that lies halfway between the hands. While the combination of Scene-in-Hand and Two-Handed Manipulation offers the greatest flexibility – the diorama can be transformed into any configuration – the controls take time to learn (particularly the two-handed rotation and scaling) and it also easy to become disoriented.
Base Model (with constraints)
In this configuration, we apply the same combination of Scene-in-Hand and Two-Handed Manipulation as above, but add constraints that limit freedom to save users from becoming disoriented. Through informal testing, we found that removing the ability to change the pitch and roll of the diorama helps to reduce disorientation (even when the diorama is at a larger scale). We also found that limiting the diorama to never increase beyond a 1:1 scale was important to avoid unexpectedly slow translation. While this base scheme with our constraints applied reduces disorientation significantly in comparison to the unconstrained version, we still find that users had difficulty learning the difficult controls.
Simon
To address the time it takes to learn the controls, we've created configurations that simplify how the controls work. In this scheme – named "Simon" – we make use of both the grip and trigger buttons. The grip button is used to enable translation (via the Scene-in-Hand approach), while the trigger button enables rotation. When rotating, the diorama pivots around its own centre point. As above, we lock the pitch and roll axis; however, different from above, we also disable the ability to change the diorama scale. We find that this scheme not only reduces disorientation but also reduces the learning time; users are able to learn the controls quickly. Despite these advantages, this configuration offers only limited control over viewpoint.
Areito
We also developed a one-button only scheme, using only the grip button. When the grip button is active, the user can translate and rotate the diorama (Scene-in-Hand style). New to this scheme – the user can reach out and push against the diorama to rotate it (imagine turn at tap or knocking against a box). As before we lock pitch, roll, and scaling. Similar to "Simon" scheme, we also find that this configuration reduces disorientation and learning time, but limits flexibility. While some users find the knocking preferable, others seem to prefer using the pointing button (since it doesn't require reaching out to touch the diorama).
Designer
To address the lack of flexibility, we have experimented with another configuration that offers more control. In this scheme, we choose to lock pitch and roll, but allow resizing. Different from the previous two configurations, this scheme triggers translation when the grip button is pressed and rotation while the trigger button is pressed. The diorama can be moved by pressing the grip button on one hand, and then rotated by pressing the trigger button on the other hand. When rotating, the diorama pivots around the gripping hand (allowing finer scale control over rotation). While this scheme is certainly more flexible, it suffers from taking more time to learn and also can lead to disorientation.
Next Steps
We haven't yet figured out which configuration of these configurations is best final production. Based on our own informal tests, we believe that the "Simon" and "Areito" configurations are most effective for providing enough to participate in the story of Minimum Mass while being easy to learn and comfortable to use. We also think that the wider control offered by the "Designer" scheme is best suited for use by artists developing scene composition and other effects. In any case, having these configurations in place means that we can now begin formal user-testing. During testing, we plan to examine the experience of using each configuration impacts satisfaction, usability, sickness. I'll be back in a future post to report what this formal testing concludes and propose a final configuration for production.