VR Therapy: Multisensory Experience
Video T1. Tilt brush control panels. From left to right: color palette, tool panel, brush panel, and options panel
Basic drawing
Figure T1. HTC VIVE Controller. From darkest circled to lightest: side button, trigger, touchpad
Source: http://www.tomshardware.com/reviews/vive-rift-playstation-vr-comparison,4513-6.html
The side button has only one input–it can simply be pushed, and although there is one on each side of the controller, both behave identically in Tilt Brush. The trigger also has only one input– simply pulling it. The touchpad however has multiple inputs.
-
You can push it on either the left side, center, or the right side
-
You can swipe your finger across it vertically and horizontally
-
you can move your finger around its edge in a circular motion.
All of these are registered as different inputs in Tilt Brush.
Controller functions
Tilt Brush requires the use of both hand controllers. Your dominant hand will control interactions such as moving objects, teleporting, and drawing. Your non dominant hand (or off hand) will be a control panel that includes a color palette, tool panel, brush panel, and options panel. Video below shows each of these panels. You navigate between them by swiping horizontally on the touchpad on your off hand controller, and select the options by pointing and pulling the trigger on your dominant hand controller.
HTC VIVE controllers
In order to understand how to use Tilt Brush, it is first important to understand the HTC VIVE controllers and establish some terms that will be used throughout to refer to their various buttons and controls. Figure T1 belows shows an HTC VIVE controller from 3 important viewpoints. These controllers are tracked by the system and allow you to interact with the VR environment. Tilt Brush primarily uses three buttons on the controllers. We will call them the side button, the trigger, and the touchpad for simplicity. These are circled below from darkest to lightest.
It is important to note that you can delete drawings but you cannot edit or move them in any way. We used the above techniques to create the majority of our project. The tree, dance floor and river are a good examples of drawn objects in our project.
Tilt Brush Instruction Guide
Overview and purpose
​We created our final project deliverable in Google’s Tilt Brush application in the HTC VIVE. Tilt Brush is a drawing application that allows users to make 3D sketches in VR, and is available through SteamVR, a platform for VR games run on your personal computer. This guide will outline the features of Tilt Brush we used to create our final deliverable, and will show where we implemented these features and techniques into our design. It’s purpose is to acquaint the reader sufficiently with Tilt Brush, and equip them with the necessary tools so that they would be able to approximately recreate our final prototype.
Video T2. Selecting brush and color, drawing, adjusting brush size, redoing and undoing
Tilt Brush is most basically a drawing application, and so to best understand it, it makes sense to start with learning how to draw.
Using the your dominant hand controller and the control panel of your off hand controller, select the brush type and color. Then you can draw freely in space by pulling the trigger on your dominant hand controller.
You can change the brush size by swiping horizontally on the touchpad on your dominant hand controller, and you can undo and redo by clicking on the left and right sight of the touchpad, respectively, on your off hand controller. Video T2 shows all of these features.
Color, brush width, redoing and undoing functions
You can also select the straight edge tool to be able to draw straight lines in space. While drawing with this tool, if you press the touchpad, it will lock your line direction. Additionally, you can select the eraser tool to delete any of the lines you have drawn. Video T3 shows these features.
Video T3. Straight edge and eraser demonstration
Straight edge and eraser
Finally, the brush picker allows you to copy the brush type and size as well as color of an existing drawing, and then use those settings to continue drawing. Video T4 shows this process.
Video T4. Brush selector
Brush selector
Navigation
Once you are in an environment in Tilt Brush, there are three things to consider in terms of navigating that environment:
-
The physical limitations of your space (represented by a blue box in the VR)
-
The ‘Teleport’ feature in Tilt Brush
-
The ‘Resize’ feature in Tilt Brush
Figure T2. Physical Space in Garage for VR
In Tilt Brush, you are able to walk around in your environment. However, your movement is limited by size of the physical space outside of the VR. Figure T2 on the left shows the physical space we worked in. You had to stay on the green mats for the VR experience to work, in order for the trackers to pick up the VR headset and controllers.
To make the environment smaller (make yourself bigger in it), push the buttons on both of the controllers and move your hands together, in a shrinking motion. Shrinking the environment is very useful for navigation as it allows you navigate quickly between locations, as shown in Video T7.
Video T7. Shrinking environment
The ‘Resize’ feature changes the scale of the VR environment. To enlarge the environment (make yourself smaller in it), push the side buttons on both of the controllers and move your hands apart, like you are stretching something. Video T6 shows this process.
Video T6. Enlarging VR environment
VR space
This physical space is represented in Tilt Brush as a blue gridded cube that appeared only when you approach one of its sides (one of the boundaries of the physical space). This boundary cube is shown in Figure T3. You can’t walk past this cube, so your movement is restricted within this cube. It is important to note that the size of this cube is only related to the size of the physical tracking space setup –therefore, the size of the blue cube is fixed because the size of the physical tracking space is fixed. This presents a problem for navigating an environment in Tilt Brush larger than the boundary cube. The ‘Teleport’ feature and the ‘Resize’ feature help resolve this problem.
Figure T3. The walls of the blue boundary cube appear when you approach them
source: http://www.etr.fr/articles_images/236485-xhtc-vive-chaperone.jpg.pagespeed.ic.AAPw7SPeId.jpg
Blue grid
The 'Teleport' feature allows you to move within the VR environment without walking or moving in your physical environment. It is important to note that your location with the blue boundary cube remains the same when teleporting because your location in the physical space remains the same. The Teleport tool can be found on the tool panel and is represented by two shoes.
​
When this tool is selected, you are able to pick with your dominant hand controller where you want to move in the VR environment, and able fairly quickly to move throughout it. Video T5 shows this process.
Video T5. Teleporting through VR Environment
Teleport
Resize - zoom in
Resize - zoom out
When the side buttons are pushed on each controller, you can shift or translate the VR environment. Finally, to reset the environments size and your location, press both of the side buttons, and while holding those, press both of the touchpads. Video T8 shows both of these functions.
Video T8. Shifting VR environment and resetting size and location
Reset size
Guides
Tilt Brush offers a feature called ‘Guides’ that allows you to draw things on a 3D Shape. There are 3 guides: a sphere/ellipse, a cube/box, and a pill capsule shaped guide. These 3 shapes are shown below in Figure T4.
Figure T4. All three guide shapes
These guides can be resized and moved using the side button, much like the VR environment can be. However, there are two additional features when resizing objects such as guides. They can be pinned or locked in place using the trigger and pressing the touchpad on the dominant hand controller forces the object to snap into a grid, not allowing them to be moved or rotated freely.
Guides provide a flat surface to draw on in an open 3D space. When your brush gets close to a guide, it will snap to the surface and only allow you to paint there. Video T10 shows this process.
​
This feature makes it easier to write and text, as you can put it all on a flat 2D plane, and this is exactly what we did to write most of the text in our final deliverable.
Video T10. Painting on surface of guide
Finally, grabbing these guides and throwing them deletes them. Video T9 shows all these features: Moving, resizing, pining and unpining, and locking into grid, and then throwing the guides away.
Video T9. Move and resize guides
Move and resize guides
Additionally, guides offer a shape to trace to make it easier to draw things such as spheres or cubes. We used this feature in our final deliverable to draw the outline of cubes around some of our rooms, as shown in Figure T5.
Figure T5. Picture of outlined cube from our environment
Drawing on guides
Video T12. Placing 3D object and resizing it
3D Models are used extensively throughout or project. Objects such as the apples on the tree, the giant chair in the drawing room, and the huge dinosaur are all examples of 3D models.
​
Conclusion
We designed our VR environment using the techniques and features of Tilt Brush outlined in this appendix. Combining the drawing and resizing features of Tilt Brush along with the ability to insert images 3D models gave us a simply yet powerful set of tools needed to create our immersive environment.
Models and images
Tilt Brush also allows 2D images and 3D models to be loaded into tilt brush. External images must be loaded into the directory:
C:\Users\”username”\Documents\Tiltbrush/MediaLibrary\Images
From here they can be accessed via the media library menu within Tilt Brush, where they can be moved around as 2D objects. They can be pinned, moved and resized just like guides using the side button. Video T11 below shows the process of adding and then manipulating an image in Tilt Brush. We used images for the space in which you design your own chair.
Video T11. Adding images
​
3D models
​
3D Models must be loaded into the directory
C:\Users\”username”\Documents\Tiltbrush/MediaLibrary\Models
These 3D models must be in .obj format, and if they have a texture, it must be a .jpeg file. Once they are in this directory, they can be accessed in Tilt Brush via the media library menu and they manipulated as 3D objects. They behave exactly like guides except that they cannot be drawn on like guides can. They can be pinned, resized, and moved just like guides however, as shown in Video T12 below.
2D images
Side Button
Trigger
Touchpad
