Better Living Through Big Data

Mobile Technology for the Deaf and Hard of Hearing

Julian Smith Writer

From hearing aids to apps that support ASL, new technologies are changing the way the deaf and hearing impaired experience the world.

When people call technology like mobile devices and apps “life-changing,” the term usually carries a healthy dose of hyperbole.

But for the 360 million people worldwide who are deaf or hard of hearing, the smartphone revolution has been just that, allowing them to do things that were impossible five or ten years ago.

“Email, texts and instant messaging all have made my interactions with hearing people much, much easier,” said Don Grushkin a professor of Deaf Studies at the California State University, Sacramento.

But certain technology just doesn’t work or isn’t accessible to deaf people, such as keyless or voice-activated systems like Siri.

“In many ways, mobile technology has proved to be quite empowering to the deaf and hard of hearing community,” he said. “On the other hand, mobile technology is still more limited than fixed technology such as computers.”

technology for the deaf

The first technology was the WyndTell pager. Released in 1998, it eventually morphed in the BlackBerry and was joined in 2002 by T-Mobile’s Sidekick phone.

By this time, email had already rendered TTY teletypewriters, invented in the 1960s for text communications over telephone lines, all but obsolete.

Today, smartphones, tablet computers and various apps let the deaf and hard of hearing do almost everything the hearing community does, from ordering pizzas to calling Über drivers (some of whom are also deaf or hard of hearing).

But they also make it possible to do things like find movie theaters playing captioned films or alert users to common sounds like doorbells and fire alarms.

Popular video chat apps such as Facetime and Skype are useful for communicating via American Sign Language (ASL) and lip reading, said Cynthia Compton-Conley of the Hearing Loss Association of America.

“It’s really important to be able to do ‘speech reading’ [such as] expressions and body language, she explained. “Cell phones have really opened that up.”

Creating short videos with apps such as Instagram, Tout and Glide is another handy way to exchange short ASL messages, she continued, either by sending them directly or posting them online.

The increased availability of closed captions or subtitles are also a major advantage of mobile technology, says Zainab Alkebsi of the National Association of the Deaf.

“Before, captions were only available on traditional platforms, but now they are passed through to mobiles as well, such as Netflix, Hulu, or YouTube…provided such content is even captioned in the first place.”

They still have room for improvement, however, as auto-caption programs often produce garbled or nonsensical text.

Effective real-time transcription systems are starting to appear, however.

Transcense, an app currently in development, will transcribe conversations with multiple participants in real time. Group conversations are especially difficult for deaf and hard of hearing people, who have a much easier time one-on-one using ASL or lip reading.

The Transcence app, when installed on several mobile devices, essentially turns them into a distributed microphone system. It transcribes multiple voices simultaneously and assigns each speaker a different color bubble in a single scrolling readout.

At an estimated cost of $360 per year, Transcence is significantly cheaper than hiring ASL interpreters at $50-$120 per hour.

MotionSavvy, available this fall, will use a specialized camera to track a user’s finger motions and transcribe ASL in real time, speaking the translation out loud.

The system is available as software or on a Leap Motion-equipped tablet and will supposedly recognize at least 2,000 signs when it is released.

Users will be able to add and upload their own signs to share with others. Installing the system on smartphones will have to wait until their processing power can support Leap Motion technology.

The cost will be $99 for the software or $799 for the tablet, with an additional subscription fee of $20 per month.

“Mobile devices have had a huge impact on the deaf community,” said Michele Westfall, a writer who has been deaf since birth. “It has changed the way we communicate, added expectations and rules, and made the playing field a lot more level.”

technology for the deaf

However, as much as they promise, Westfall still has doubts about software like MotionSavvy or Transcense.

“I don’t think this type of technology is as ‘real-world-ready’ as their developers think they are,” she said. “While such apps might recognize fingerspelling and signs, they still don’t take in account facial expressions and body movements, both of which are very important to the grammatical structure of ASL and other signed languages.”

Without these nuances, she says, translations become the equivalent of monotone speech, or are simply incorrect. According to MotioSavvy’s website, its developers eventually hope to correct this by making the program recognize facial and body expressions.

For the hearing impaired, a new generation of hearing aids can now pair directly with smartphones, TVs, MP3 players and computers via Bluetooth or the iPhone’s proprietary 2.4 GHz wireless network

Normal hearing aids suffer from interference, reverb and background noise, and are only designed to function close to a sound source, said Cynthia Compton-Conley of the Hearing Loss Association of America.

Being able to hear a conversation, phone call or TV show directly through a pair of hearing aids is a game-changer, she said. “Two ears are better than one.”

For example, ReSound brand hearing aids can stream directly to smartphones, using accessories like a transmitter that attaches to the back of a TV. A wireless mini microphone accessory, which transmits spoken words clearly even under noisy conditions, could be considered a mixed blessing — especially on road trips with carloads of children.

Overall, technology for the deaf and hearing impaired has improved the user experience, but there’s still a lot of work to be done.

According to Westfall, what the deaf community would really like to see is see more deaf involvement in research and development and for hearing developers to remember than there are millions who need viable alternatives to default systems.

“Developers and inventors need to stop thinking in sound-based terms and start thinking visually,” she said.

Share This Article

Related Topics


Read This Next

Read Full Story