How to make your own mixed reality VR setup
Just how do you show people what virtual reality is like? Valve, along with the help of some games developers and skilled video producers came up with an answer earlier this year: using a green screen and some video know-how, it transposed players from the real world into the virtual world, displaying the results onto a separate monitor. With mixed reality, it’s possible for onlookers to see what it’s like to blast drones inside Space Pirate Trainer, or pet a robotic dog inside The Lab—the results are hugely impressive.
And yet, despite looking rather difficult in the Valve video, doing this sort of production work yourself is not only possible, it’s surprisingly easy. At E3 2016, HTC showed off its own mixed reality solution, one that’s been developed specifically for showcasing VR at events. That means it’s portable, easy to set up, and relatively cheap, showing everyone the wonder of immersive VR without necessarily having to put a headset on.
The way it works is simple: you take a camera that has a video output of some kind (HTC used a Sony A7S with a 17mm lens at E3), point it at a green screen, and strap a Vive controller to the top of it. The controller tells the game where the camera viewpoint is in relation to the player. The game then renders a second viewpoint that matches that of the camera in the real world. Then, all that’s left to do is overlay the green screen footage from the camera onto the game footage, and hey presto, mixed reality.
OK, so there are a few things to bear in mind. For one, the sensor size of the camera and the focal length of the lens affect how the game renders the secondary image. For example, a 17mm lens attached to a full-frame sensor has a wildly different field of view to that same lens attached to a smaller micro-four-thirds sensor, or even an APS-C sensor. Ideally, you want to match the field of view of the camera with that of the game. Usually that requires a special build of the game that lets you adjust the field of view, as well as support a third Vive controller and output a secondary video stream.
HTC had a special build of Space Pirate Trainer on hand at E3 with the functionality built in, although there’s talk of Valve possibly opening it up to all games in the future. You also need a video capture card of some kind (something like a Blackmagic Intensity Pro 4K will do the trick) and a way to overlay the green screen footage onto the game footage in real time. Luckily, Open Broadcaster Software will do that just fine and doesn’t cost a single penny, or you can opt for a hardware-based solution like the NewTek Tricaster. Oh, and you’ll need a suitably beefy system in order to render that secondary viewpoint and mix everything together.
Fact is though, creating this sort of mixed reality footage is accessible, and for developers struggling to showcase how their games work in VR, it’s a brilliant solution.
By Mark Walton | ars technica