High-quality camera-equipped sports arenas and speedy processors help Replay Technologies give fans mind-boggling perspectives on every play.
Basketball fans who regularly berate the television over blown calls and blind refs might not need to raise their voices thanks to new high-definition sports replay technology. Used by many professional sports leagues, freeD video captures 360-degree views, giving fans access to the same real-world angles as the pros on the sidelines.
For the 2016 All-Star game, a partnership between the NBA and Turner Broadcasting will give fans access to Replay Technologies’ freeD video format. The San Antonio Spurs, Golden State Warriors and Cleveland Cavaliers stadiums are equipped with this technology. Twenty-eight high-definition cameras positioned around the arenas capture video that can seamlessly be compiled into one shot.
Even someone sitting in the nosebleed section can see 360-degree views of important plays. FreeD replays will be available to fans watching live on TNT or following on NBA.com, the NBA App and social media.
At CES 2016, Intel showed exactly how these riveting replays can change the game. The tech company is acquiring Replay Technologies in order to better capitalize on the opportunity to bring immersive sports experiences to the masses.
— The Verge (@verge) January 6, 2016
“What you are seeing here is the power to redefine what it means to watch and experience sports,” said Brian Krzanich, Intel CEO in his CES keynote. “We look forward to seeing this technology come to the biggest playing fields in the world. It is going to change what it means to celebrate the thrill of victory.”
freeD replays not only allow viewers to see victory points and plays from every angle, it enables them to re-watch key moments and post a custom clip on their social networks. Although the technology is primarily used for sports replays, the idea first materialized from somewhat unexpected places.
“freeD was born when Industry experts from the defense, computer generated images and visual effects worlds came up with the ‘crazy’ notion that perhaps there is a way to view reality in a way that is unconstrained from where a physical camera is,” said Preston Philips, vice president of marketing and communications for Replay Technologies.
Replay Technologies was named one of the top 10 most innovative companies in sports by Fast Company in 2014. Today, freeD tech wizardry is currently used in dozens of sports arenas across Major League Baseball and the National Football League, and it was used to cover the Olympics in London and the US Open Tennis Finals in New York.
This growing phenomenon will lead to what Intel’s Krzanich calls “the digitation of sports.”
“Digitization is a big word that’s thrown around a lot today, but when it’s used with sports, it’s truly transformational,” he wrote in a blog post. “It means that everything we do in a sport can now be captured as a piece of data. Data like we’ve never experienced it before. The tiniest details of play — from an athlete’s elbow angle on a free throw to the vertical on a slam dunk — can be measured and broadcast in real time.”
From the fan perspective, these 360-degree replays offer new insights into highlight-reel plays and potentially botched calls. If LeBron James makes an epic 3-pointer only to have a ref say he stepped out of bounds prior to shooting, fans can look at his foot from every angle instead of from whatever angle the cameraman captured.
Teledyne Dalsa Falcon 2 cameras make the 360-degree replays possible by creating a grid of 5k sensors, Philips explained. An Intel high-performance computing (HPC) system then converts the 2D data into 3D volumetric pixels or “voxels.”
“It captures the sports action you love and turns it into a true 3D experience that you can re-watch as if you are right in the game,” Krzanich said. “With this technology, you can control and change the view to any perspective you want. You become the director.”
What’s most impressive about the technology is the speedy turnaround from 2D video footage to 360-degree replays. Philips said this has as much to with the engineers who used general purpose graphics processing unit (GPGPU) techniques to develop the system’s algorithms as it does with the hardware.
“This data, called ‘freeD video,’ allows us to not only create ‘impossible views’ replays for broadcast, but also gives the home user complete interactivity on their 6th Generation Intel Core processor, ushering a new type of video format and visual language,” Philips said.
“We get speed gains from using an Intel-based HPC [high performance computer], fully optimized for efficient processing,” he said. “Additionally, the ‘freeD renderer’ allows for real-time rendering of novel views anywhere within the coverage of the sensor grid.”
With the technology already used by the Yankees, Dodgers, White Sox, Mavericks, Cavaliers, Cowboys, Ravens, 49ers—to name a few—the company plans to implement freeD in all venues across the various sports leagues. Look for the capabilities to extend to fans of UFC, rugby and other sports.
Philips imagines that once “free dimensional video” is unleashed, viewers won’t want to go back to the regular, flattened video of yesteryear.
“Eventually, perhaps sooner than we think, we will live in a world where video is a complete 3D representation of reality, merging existing concepts of entertainment,” he said. “Cinema will fuse with theater, interactive games will fuse with sports broadcasting and, perhaps most importantly, people will be able to digitally share a physical space with other people thousands of miles away using holographic or VR types of enabling technology.”
Philips gives the example of a yoga lesson where all the students are thousands of miles apart. As the instructor guides each person through a series of poses, the students would wear hardware that projects their avatar in the same room. The teacher could then note their form and make adjustments as necessary.
CCS Insight technology analyst Ben Wood sees 360-degree videos and replays impacting how people interact and control what they’re watching. When this technology is tied to TV remote controls that function with gesture and voice commands, Wood imagines it could be mind-blowing for fans.
“If you’re watching a football game on your TV, you can actually say, ‘pan left’ or ‘pan right’ like you would adjust the volume up and down,” he explained. “I don’t have to wait for the director to cut to it. I can just kind of swipe across on the screen and look at what’s around.”
While Wood’s example involves 360-degree-capable televisions, Philips foresees fans being able to jump into the game from any device—and get a truly close-up view of the action.
“We believe that in a not-so-far point in the future, people would be able to turn on their viewing devices, be it a 2D tablet or a 3D VR/AR device, and leap right inside the game,” he said. “They could run beside their favorite players, watch several feet away from the ball or simply hang around the sidelines.”
Armchair refs can get so close to the action they won’t need to yell at their screens. Unfortunately, the virtual players are likely to get an earful.
Feature image credit: Replay Technologies Facebook Page