View: Sign of the times

Researchers from Germany, Sweden, Spain and the United Kingdom are working together on a project to incorporate sign language recognition technology into 3G cellular phones, according to a CNN.com article. The project is called Wireless Information Services for Deaf People on the Move (Wisdom).

With this technology, hearing-impaired subscribers will be able to use sign language to call up news, weather and sports information from a video server via 3G phones, give commands to their phones and access a real-time interpretation service to help them communicate with hearing people.

In lab trials, the technology can understand sentences having as many as nine signs with a recognition rate of 90%.

The European Union is contributing more than $5 million in funding to the project over the next three years, with sights set on helping 0.2% of the European public that communicates primarily in sign language.

At first glance, I was impressed with this forward-thinking effort. However, after further reflection, I now wonder why short-message service, which is wildly successful in Europe, mobile instant messaging, e-mail or other existing capabilities wouldn’t suffice.

For example, last year at Deaf Expo 2000, Wynd Communications, a division of GoAmerica, introduced wireless Web access to the hearing-impaired through its WyndTell 2-way wireless messaging service. Subscribers can use this paging service to communicate using 2-way wireless Internet e-mail, TTY, chat, fax as well as voice through text-to-speech and speech-to-text.

Doug Timewell, Wynd VP of sales and marketing, points out that the hearing-impaired community struggles daily with distance learning. “Any technology that helps our community communicate is a good thing,” he said.

I agree wholeheartedly, but that still doesn’t explain to me why signing to a handset would be so much better than current technologies. For example, for movie listings, presumably you could sign “movie listings” and call up the selections and times. Or you could just pull it up from your favorites off your home deck of your wireless device.

Granted, sign language is as unique in structure as Spanish or French is to an English-speaking individual. It likely would be as nice for hearing-impaired individuals to communicate using their “native language” as it would be for me to speak English as opposed to, say, Greek.

However, I know hearing-impaired individuals, like those of us in the hearing community, learn to read and write. Further, TTY on the landline side requires the ability to read the message and respond by keyboard. So all of the current capabilities on 2.5G and 3G would provide effective communications.

I’m not opposed to niche-specific developments especially where the hearing-impaired community is involved. I just wonder about the “wisdom” of this funding. Are there other developments that the funding could be spent on? Or better yet, why not contribute it to hearing-impaired schools directly?

Insider Feedback

This letter from the field arrived shortly after the posting of this story.

I'm a professor at Hofstra University on Long Island and a governmental affairs consultant for the National Association of the Deaf (NAD).

When this story appeared in May, it excited some interest. There are, however, several things you should be aware of.

First, even though a project in Germany has worked on software that "understands" some sign language, this is a long way from practical reality. Research is ongoing on this side of the Atlantic, too, and it's slow slogging. Formidable problems remain ... notably the fact that everyone signs differently from everyone else. This isn't going to be practical for years and years to come.

Second, British Sign Language (which this project is using) is very different from American Sign Language. Thus, even if the software could understand as many as nine discrete British signs, none of these are used in the United States. So the software cannot cross the pond.

Third, 3G is progressing on this side of the pond much more slowly, and much more erratically, than in Europe.

For all of these reasons, we at the NAD are taking a skeptical "We'll believe it when we see it" attitude toward this.

That being said, yes, the concept is very exciting and appealing ... It will be great, if it ever happens, and if anyone alive today will live to see it.

Frank G. Bowe, Ph.D.

Governmental Affairs Consultant

National Association of the Deaf

Silver Spring, MD

More Feedback

I received your editorial, titled "Sign of the Times," from a colleague, who asked if our Imagetalk project competes with this initiative. My answer was no. When cleaning up my inbox I took a closer look at your arguments, and I find them good.

The Imagetalk project has passed the proof-of-concept phase, and we are now trying to find solutions to ensure the funding of finishing the product and launching the service. All our applications, both server-side and the user applications for the actual user and for the assisting user (a configuration tool running on a personal computer), are Java-based.

Imagetalk is essentially a network service built on existing Internet and mobile network standards. Its objective is to offer means of mobile communication to persons with severe speech and language disorders, that is, persons who cannot use text-based or speech-based devices.

What remains? The already well-proven symbol and pictorial "languages" for creating simple messages. We can also imagine icons based on sign language as one possible library from where to create the Imagetalk communication environment, which necessarily is unique for each user.

Contrary to the project which you described in your editorial, Imagetalk is deployable almost right now. The only missing link technology-wise is a mobile device with large-enough touch screen, preferably color. As the rollout of Symbian-based devices seems to take time (Nokia Communicator doesn´t have a touch screen), the first such device seems to be Compaq iPaq.

We are still proceeding with quite a low profile, but you might want to take a look at this animated demo describing the main use cases of Imagetalk. With Imagetalk a communication-impaired person can create messages with an icon-based user interface, while the communication partner can use ordinary mobile phone or e-mail. Imagetalk also incorporates an icon-based calendar for day-planning tasks. demo

Juhani Selänniemi