I’m going to show you how you can start playing PC games, in this case Skyrim, through immersive Virtual Reality in your room using some of the devices out on the market.
With this setup you will be able to move through the game and look around naturally, similar to like in the real world. You’ll be using the head mounted display to make you feel like you’re inside the game, the head tracking will let you look around the world, the motion tracking allows you to move naturally in real life to correspond to actions in the game, and some basic speech recognition to do more abstract tasks such as navigating through a menu or initiating a chat with NPCs.
You’ll need the following, or an equivalent in some cases, to get this to work:
- Skyrim – PC game.
- Sony HMZ-T1 – A high definition head mounted display that also supports Stereoscopic 3D.
- Microsoft Kinect – Gesture and motion tracking
- TrackIR5 – Head tracking.
- TrackClip Pro – Used with the TrackIR5 for more accurate head tracking.
- Shoot – Command based speech recognition software.
- FAAST 0.9 – Software used to map Kinect gestures to the keyboard.
- Microphone – For speech recognition.
- Windows 7
- Nvidia Graphics Card with Nvidia 3D Vision Support – For 3D stereoscopic.
I won’t explain how to setup the different devices as there are articles you should be able to find easily in search engines, but I will provide the scripts I used to let you map gestures and speech recognition commands as well as any other important software and settings I’ve used.
Setting up Sony HMZ-T1 Head Mounted Display
This one is pretty basic, just follow the normal steps to setup the HMZ-T1 from the manual. If you have a TrackClip PRO, attach it to the left hand side of the HMZ-T1 in a similar position as in the image below.
If you want to enable Stereoscopic 3D as well for Skyrim, the Sony HMZ-T1 supports this and you can enable it in the NVIDIA Control Panel if you have an Nvidia graphics card. Just go to the Sterescopic 3D > Set up stereoscopic 3D link on the left side of the control panel and tick Enable stereoscopic 3D then press Apply. Note that only newer graphic cards support Stereoscopic 3D so check first that your graphics card supports Nvidia 3D Vision.
Setup TrackIR5 for Head Tracking
The TrackIR5 will track you head movement around in real life and at the same time move your view in the actual game through head tracking.You can also possibly use older versions of TrackIR too, such as TrackIR3 or TrackIR4. In addition to setting this up through the normal way, you will need to somehow place the TrackIR5 at head level in front of you, which may be tricky if you are going to stand up so that the Microsoft Kinect can track your motions at the same time.
What you can do to have head tracking work while standing up is to get a wire (like a coat hanger) and use blu-tac or sticky tape to make it hang from the roof, and then use blu-tac to stick the TrackIR5 to the wire. See the image below on one way you could hang the wire from the roof.
Skyrim doesn’t support head tracking natively, so you will need to open up another software that comes with the TrackIR software which emulates the head tracking via emulating the mouse. You will need to open the mouse emulator for the TrackIR5 called TrackIR Mouse Emulation, which is located in the same folder as where TrackIR5 is installed (C:\Program Files\NaturalPoint\TrackIR5\TIRMouse.exe in Windows 7). Now your head movements will correspond with your Mouse’s movement on the screen as well as your character’s orientation in the game.
I set the Mouse Emulator to allow X Axis Auto Panning and set it to 75, which means that when you look left or right at 75% of the TrackIR’s range then it will start panning your Skyrim character’s view automatically to the left or the right so you don’t have to turn too much in real life.
You will need to first connect the Kinect to your PC using PrimeSense’s OpenNI drivers (see these instructions). Once you set that up, download FAAST (I used version 0.9) and load the following script for Skyrim:
To run FAAST, just click on Connect and then Start Emulator.
The included script will map the following real life movements to the game’s movements:
- Walking on the spot – Walk forward in the game
- Lean left – Straff left in the game
- Lean right – Straff right in the game
- Lean backwards – Walk backwards in the game
- Right arm forward – Use sword
- Jump – Jump in the game
- Crouch – Crouch in game
You should stand about 1.5 metres/5 feet in front of the Kinect so it can pick up your whole body’s movements. You can easily map more movements from the Kinect for Skyrim using the instructions from FAAST’s website.
If you want to have some speech recognition as well in the game, just grab a microphone and set-up Shoot (instructions for Win 7) with the following Skyrim Profile below:
These are most of the voice commands you can use with the script, and you can easily add more too:
- Open Menu – Open main menu in the game
- Close/Exit Menu – Close the menu
- Go up/down/left/right – Navigate around the menu
- Enter/Confirm/Accept – Accept
- No – Deny
- Talk – Talk to NPC
- Pick up item – Pick an item or any action that maps to the E key.
Hopefully these instructions will help you set up your own VR simulation! If you are stuck on any of these or have any feedback at all just comment below and I’ll try to reply back as soon as I can.
This should give you a feel of how Virtual Reality is like before it’s more ready to enter our homes, perhaps not too far from now.
Chris Zaharia is the Co-founder & CTO of Zookal, a start-up providing a range of educational services for University Students in Australia. You can follow him on Twitter @chrisjz.
Update – This article has been re-posted from Zookal’s Blog, which was originally published on March 13, 2012.