Facial Expression Recognition: Comedy Plugs into Technology

Kristin Houser Writer & Editor, LA Music Blog

Entertainers are turning to facial recognition technologies and polling via smartphones to measure audience reactions, redirect performances in real time and bring new comedy experiences.

Programming computers to be funny has been called “one of the most challenging tasks in computer science,” but what if instead of trying to create funny computers, we used computer technology to help us better enjoy comedy?

At George Mason University, the glow of a smartphone or tablet from the audience during performances of Tony Award-winning musical comedy “The Mystery of Edwin Drood” doesn’t signal a rude patron but a participatory one.

Drood Hashtag

Written by Rupert Holmes and based on an unfinished novel by Charles Dickens, the musical was the first on Broadway to have multiple endings.

During traditional productions, the ending is determined by audience vote (taken at a break during the show), but GMU’s late-2014 production incorporated technology instead.

It’s a trend happening throughout the world of performance using technology to encourage audience participation.

“Theater, in general, will include more and more technology such as automated fly systems, LEDs, programmable lighting instruments, fully digital design, projected scenery,” said Ken Elston, director of the School of Theater at GMU.

“Audiences will expect more ‘gamified’ experiences like ours, and ultimately, theaters will utilize this technology to build audience loyalty.”

During the show, audience members vote on various elements of the production (the detective’s identity, the sound effects, the murderer, etc.) using the conferencing app Poll Everywhere, which they access by scanning a QR code on the musical’s programs.

Elston said the audience loved being part of the show, and ultimately felt like it had a hand in the production.


Creative ways to use tech in productions are happening around the world.

In Barcelona, when audience attendance dropped 30 percent after the Spanish government issued a theater production tax increase, indie comedy theatre TeatreNeu incorporated a tech element into their venue to help build its audience.

The company fitted the seatbacks in its theater with tablets loaded FaceTracker, a facial recognition program tracks each audience member’s reaction. If you laugh more, you pay more. Each laugh costs around $.35 (with a price cap of about $27).

The “Pay Per Laugh” project allows attendees to enjoy a night out without worrying that they won’t get their money’s worth.

Not only does this sliding–scale system provide audience members with a value-based rate, it also provides comedians with data-driven, somewhat objective feedback on their performance as they can see which jokes killed and which jokes just died.

Like the facial recognition system powering the “Pay Per Laugh” project, Intel RealSense technology also has the potential to make humor high-tech.

The RealSense 3D camera incorporates gesture, facial and voice recognition to streamline users’ computer interactions, and the potential applications are endless.

“One of the main intentions of RealSense is to allow computers to interact more naturally with people,” said Eric Mantion, evangelist for RealSense technology at Intel. He said the simplest intent is understand a user via a head nod or hand gesture.

“But,” said Mantion, “the broader intent is that the computer will get to know you on a personal basis and to understand your needs, to optimize what you get out of your computer.”

For example, one day soon you may be able to install software that interacts with your device’s camera to determine which types of YouTube clips inspire you to laugh the most. The software could then create a custom video playlist of clips that tickle your funny bone.

Another bit of RealSense tech, Snapshot, incorporates three different cameras to capture pictures with much more embedded information than standard cameras can capture. It also provides the shooter with the ability to creatively manipulate pictures after taking them, so the images to rise to the top of “r/funny” on Reddit could look pretty different (and uniquely hilarious) in the not-too-distant future.

We may have to wait a while before we’re guffawing at the gags of an artificially intelligent stand-up comedian, but today’s technology is impacting comedy in interesting ways, like never before.


Kristin Houser is co-founder and lead writer/editor for LA Music Blog, a Los Angeles-based music news and review website.


Share This Article

Related Topics


Read This Next

Read Full Story