Leading filmmakers use 360-degree video, 3D-game-like environments, immersive sound, head mounted displays, motion chairs and even scent emitters to pull movie fans into the action.
In one of the summer’s most anticipated movies, Dunkirk opens to hundreds of thousands of British and Allied troops surrounded by enemy forces. Trapped on the beach with their backs to the sea, they face an impossible situation as the enemy closes in.
But this intense theatrical experience doesn’t end when the final credits roll.
Movie goers can experience the harrowing miracle evacuation of troops by land, sea and air – all from the safety of their living room – thanks to the magic of virtual reality (VR).
And it’s just the beginning of the latest trend to hit Hollywood and content creators around the world: virtual reality experiences (VREs) that offer 360-degree, immersive experiences to pull fans into the action.
Capitalizing on successful films like Oscar-contender Dunkirk and Spider-Man: Homecoming, VREs bring stories to life for movie fans long after they’ve left the theater. VR is even poised to become the newest film medium with the release of Le Musk, the first VR multi-sensory feature film.
Industry experts say the total VR market is expected to reach $80 billion by 2020. A growing piece will belong to cinematic endeavors.
“Technology is going to change future of filmmaking, as virtual reality has changed the equation,” said Ravi Velhal, a media technology strategist at Intel and VR Society board member who collaborated on all three film projects.
“It has expanded boundaries, defined new expressions for storytellers, and brought new immersive multi-sensory experiences for audiences, making them a part of the story to explore and expand new dimensions,” said Velhal.
A Virtual Dunkirk Miracle
Save Every Breath: The Dunkirk VR Experience propels viewers into the action on land, sea and air. The intense 360-degree short film immerses the viewer in the world of filmmaker Christopher Nolan’s epic action thriller, Dunkirk.
In the film, 400,000 Allied soldiers are trapped on the beach of Dunkirk, France. When the shore proved too shallow for massive naval vessels to get near enough, hundreds of civilian small boats answered the British government’s call to help pick up the increasingly desperate men. An estimated 338,000 British and French soldiers were saved by the combined effort of the small boats and the Royal Navy.
Through three tightly woven sequences, the film gives a focused view of what an infantryman, a Royal Air Force pilot and a small boat captain went through during the 1940 Operation Dynamo evacuation, which was dubbed the “miracle at Dunkirk.”
“In just a few taut moments, the Dunkirk VR Experience provides a thrilling glimpse of the three main settings in our film in a uniquely subjective way,” said Nolan, who wrote, directed and produced the film.
VR viewers experience the horrors of war, from waiting on the beach with thousands of trapped Allied soldiers to swimming underwater in the French Channel to flying in the skies in a RAF Spitfire.
“The world Christopher Nolan created for Dunkirk played right into the strengths of VR,” said Matthew Lewis, president of Practical Magic, and director and producer of Dunkirk VR Experience. “It’s relentless storytelling with the enemy always a breath away. We knew exactly how we wanted to bring that into VR — to put you right there in that moment.”
While 360-degree VR videos are optimized for HD and 4K media, the production company had to stitch images 40 times that resolution for this VR experience. Using Dell blade servers and workstations that were powered by Intel Xeon processors, Practical Magic had real-time access to high-resolution media during the post-production process.
“The Dunkirk VR Experience will likely transform all of our expectations about what immersive entertainment means in the years to come,” said Intel’s Velhal, who was also the executive co-producer for Dunkirk VRE.
Becoming a Superhero
A collaboration between Sony Pictures, CreateVR and Intel, the Spider-Man: Homecoming VR Experience is a 360-degree, immersive adventure that offers movie goers the opportunity to feel what it’s actually like to be Spidey by simply donning a VR headset.
For many, including Velhal, becoming Spider-Man fulfills a childhood fantasy.
The VR experience starts on the rooftop of Peter Parker’s apartment building, where players are given a virtual Spidey suit. Players then test their skills as a superhero by slinging webs, zip lining to the top of a crane and swinging off through the canyons of NYC in pursuit of the web-slinger’s nemesis, the villainous Vulture.
The experience feels so real that some people experience vertigo. To reduce such side effects, filmmakers employ a stylized pinhole view to reduce dizziness, said Velhal.
To create such cinematic realism – including players seeing a mirror version of themselves as Spider-Man – is complicated, to say the least.
“When we put the suit on, we see ourselves in a mirror. We have to actually have a reflection in real time that mimics the actions that we’re making,” said Sony Picture’s Jake Zim, a senior vice president of VR who oversaw the project. “That’s tough. Lighting, textures, all kinds of things that require really, really smart people and great technology.”
Zim said that was why Sony Pictures partnered with CreateVR and Intel.
The award-winning developers at CreateVR used Intel hardware and software tools, including dual Intel Xeon processor-based Dell workstations to create and render the 360-degree, ultra-high resolution graphics of New York.
“We want people to walk away from this really knowing what it feels like to be Spider-Man,” said Jake Black, head of the CreateVR team.
Le Musk Brings VR to Theaters
India’s A.R. Rahman, the two time Oscar and Grammy-winning composer turned filmmaker, was one of the first innovators in the VR cinema space. He’s behind the soon-to-be-released Le Musk, considered to be the world’s first VR multi-sensory feature film.
Shot in Rome, the VR movie follows the story of Juliet, an orphaned heiress and part-time musician.
In addition to sight and sound, Le Musk was produced for movie goers to experience seated inside a motion controlled egg-shaped pod by Positron.
The seat rotates 360 degrees and pitches forward, backward and from side-to-side to coincide with narrated movement in the storyline. It delivers smells via a spritzing device attached to the pod. The latter is especially appropriate given that the character of Juliet is obsessed with scents and perfumes.
The production team used a pair of Jaunt ONE cameras, each fitted with 24 camera sensors, that capture up to 8K (or 4K per eye) quality video. They also used a drone-mounted VR kit and other imaging gear.
Every frame was stitched into a 360-degree digital world. Computer generated (CG) effects were used to create almost every frame. It required powerful computer processing, fast memory, storage and retrieval, and rendering of the hundreds of terabytes of raw footage captured by the crew.
Creating a VR experience that pushes boundaries and incorporates multiple senses was a guiding force for Rahman, who wrote, directed and composed music for the film.
“I think a good smell brings a positive vibration,” Rahman said in an Indian TV news interview, “and that’s one sensory aspect which has never taken over in storytelling.”
The film’s launch at the National Association of Broadcasters (NAB) Show this past April solidified a vision for the future of VR cinema. Watching viewers seated in a row of Positron VR chairs, one could easily envision a theater filled with rows and rows of the pods.
Intel’s Velhal, who collaborated with Rahman on the project as a VR technology producer, said that the VR multisensory experience was a huge hit, marked by long lines and rave reviews. He said people were willing to stand in line more than an hour to participate in the VR experience.
“It was a promising step toward the realization of a VR theater ecosystem down the line,” said Velhal.
Immersive Tech and Endless Possibilities
It’s only recently that tech has evolved to the point to make such immersive experiences possible.
“The sheer file size and quality of the media moving through the VR workflow pushes most available technologies to the limit,” said Velhal. “While the vast majority of existing pre- and post-production processes have been optimized for standard HD and 4K media, the 360-degree immersive VR format has to work with media that is exponentially larger in size, higher in resolution and performs complex operation on that media.”
With its tech expertise and data capabilities, Velhal said Intel is poised to play a significant role in the growing field of VR cinema.
As the technology continues to evolve, content creators will have the opportunity to create virtual realities on par with a Star Trek holodeck.
“The future of cinematic VR will be totally immersive,” said Velhal. “Multiple sensory experiences will keep on improving in quality, while the line between the real and virtual worlds will continue to blur.”
Editor’s note: This article has been updated since it was first published. Dunkirk images courtesy of Warner Bros. Spider-Man: Homecoming images courtesy of Sony Pictures. Image of VR chair courtesy of Positron.