Adventurer VR

Mixed reality merges real + virtual through bespoke real-time tracking.

Detroit, USA
6k
viewers
1
USA Drum Award

Adventurer VR is a mixed reality experience created to showcase a yet-to-be-released vehicle at the North American International Auto Show in 2019. The large-scale experiential marketing activation saw audiences explore famous American landscapes in virtual reality while they sat in a real car, being driven around a real-world obstacle course mapped to their virtual experience. Brought to life through a bespoke system of Optitrack cameras, Unreal Engine and Microsoft Mixed Reality Headsets, Adventurer VR was a responsive, immersive experience like no other.

Virtual reality with a twist

The Idea

When automotive companies debut their new vehicles at events like the North American International Auto Show in Detroit, they often face a problem: the products they’re unveiling may not exist yet. For the 2019 Auto Show, we partnered with Imagination to create a new and innovative solution to this problem.

The idea was to merge physical and digital in a virtual reality experience situated inside a real car moving around a physical track. High quality real-time content would give audiences a first-hand experience of the vehicle and its features, while the tactile feedback of movement around the track would provide an unparalleled feeling of immersion.

Adventurer VR mixed reality experience opens at the North American International Auto Show in Detroit.

Photo-real cars, stylised landscapes

Art Style

Our art style for Adventurer VR aimed to balance photo realistic depictions of the vehicle with highly stylised versions of iconic American landscapes. On one hand, the realism of the car’s interior would showcase the vehicle’s features and ground the audience throughout the virtual reality experience. On the other hand, the vivid exaggeration and stylisation of surrounding landscapes would capture the audience’s imagination, echoing the nostalgic distortion of reliving treasured memories.

Real life haptic feedback

Physical Track

The physical track itself was a spectacle. With large ramps, canvar turns and a motion platform, the experiential marketing activation would have been exciting even without the virtual reality component. However, linking the virtual and physical journeys was critical to delivering realistic haptic feedback – and a more satisfying experience overall. Every bump, dip, stop and push that audiences felt throughout the experience was from the actual car itself.

Sketch of the Adventurer VR track at Detroit Auto Show.

Custom testing solutions

Collaboration

One of the most exciting and challenging parts of this project was our ambition to map the physical car location directly to the virtual reality experience. The idea was to use an Optitrack camera rig – similar to what’s used in big Hollywood productions for motion capture – to capture data about the car’s speed, angle and position on the track. But, given we’d be tracking entire cars (three, in fact) across a 60m track, a typical setup wasn’t going to cut it.

We teamed up with the University of Sydney’s Design Research and Robotic Engineering team to troubleshoot a more customised tracking solution. Their lab has a huge Optitrack rig where we were able to test our ideas at ¼ scale inside an actual moving vehicle. While we’d come up with a few interesting testing solutions in the studio – a tissue box car, for example – being able to physically move around a track would deliver vastly better feedback.

In this case, we weren’t tracking an SUV but a shopping trolley. One person would sit atop the trolley in virtual reality, with a laptop strapped between their legs, while others pushed them around the track. Not quite the same as a real car, but it did allow us to feel what it was going to be like inside the mixed reality experience and adjust what we were making accordingly.

A bespoke tracking solution

Tracking Systems

The final Adventurer VR tracking system consisted of 44 individual Optitrack cameras placed on the roof of the autoshow warehouse and carefully calibrated to provide coverage of the entire track. Each car then had 16 ‘active’ markers, pre-programmed with unique identifiers that would allow us to differentiate between cars and track them even when as few as two markers were visible to the cameras.

From there we could calculate the telemetry of the car and send that – through a small antenna mounted to the car and wireless access points throughout the warehouse – to the Alienware laptops running the virtual reality experience inside the cars. These laptops were then each powered by two deep-cycle batteries connected (through a 2000W inverter) to the car’s alternator.

Leveraging social virtual reality

Social VR

A virtual reality experience can be like going on a vacation by yourself – you have an amazing time, but there’s no one to share it with. For this experience, we didn’t want that to be the case. We wanted audiences to be able to interact throughout the mixed reality experience – to both see and hear each other as they progressed on their journey.

To start, we tapped into the car’s inbuilt audio system (a stunning 14-speaker Bangen Olsen setup) instead of providing individual headphones. This exposed audio meant that visitors in virtual reality could talk to each other without compromising the music and sounds of the experience.

We also used Unreal Engine’s networking system to represent each person in the experience as floating virtual headsets. While relatively simple, these digitally rendered headsets reflected exactly the positioning and actions of the wearer. Now every head tilt, turn or nod could be seen in virtual reality. This allowed us to bring back body language as a communication tool, increasing empathy and engagement between visitors.

Collaboration across continents

Deployment

When it came to actually putting the experience together in Detroit, we sent over a fairly small team to link up with Imagination, who had been working on the physical track. This marked the beginning of our first experiment in a 24 hour development cycle. The Detroit team would hack things together on site, before handing off to the Sydney team who would then spend their day making all the necessary adjustments.

It was humbling to think that for a while our project never slept - which was needed, because we only had about a week to get everything pushed together.

Jack Condon
Lead Developer

One of the most interesting parts of this deployment was working closely with the team of professional drivers who would be actually running the mixed reality experience.

While we were tempted to try, the drivers weren’t going to be in virtual reality during the experience. That meant we needed a way for them to keep track of where the visitors were in throughout the experience.

Professional driver tests an app to guide them through the Adventurer VR experience is tested on a tablet inside the SUV during deployment.

On a whim, we asked the drivers if there was anything in particular they’d like in the app we were building to support them. Lucky we did, because they came up with a whole host of features that made the experiential marketing activation that much more immersive. For example, with their input we were able to build in the ability to switch on the air conditioning during the mountains of Colorado, and the heater for the deserts of Monument Valley.

Team from digital agency S1T2 stands with the professional drivers who controlled the cars during the Adventurer VR experience at the North American International Auto Show in Detroit, USA.

The first of its kind

Results

Launching at the 2019 North American International Automotive Show in Detroit, USA, Adventurer VR represented the first time a virtual reality experience was presented in a moving vehicle.

With an estimated 5,600 visitors through the experiential marketing activation, the experience generated considerable interest during the 7 days it was live. It has also won a Drum Marketing Award from the inaugural Drum Awards USA, and travelled to New York, USA and Frankfurt, Germany as a modified tabletop experience.

Most of [these VR experiences] have been self-contained where you sit on some sort of platform. This is the first free rolling (track) that will take you in a vehicle and take you through the experience.

Dan Rose,
Ford Motors

And you get to go a little bit offroad, up into the mountains, down the highway and learn about the technologies in the vehicle and really experience the vehicle without ever leaving the building.

Garett Carr,
Ford Motors