Mitsubishi Electric Develops World’s First Voice-activated Drawing Function That Displays Spoken Words Where Finger Is Traced on Screen

0
49

Mitsubishi Electric Corporation (TOKYO:6503) announced today that it has developed the world’s first* technology combining voice-recognition and drawing functions to enable users to display their spoken words on a tablet or smartphone by simply dragging their finger across the screen. The technology is expected to be combined with other functions, such as, picture drawing and multilingual translation, to help people transcend disabilities and foreign languages for smoother global communication.
* As of the date of release: February 9, 2016

Key Features

World’s first user interface for voice-activated drawing

By pressing down on the screen and then speaking, spoken words can be displayed wherever the finger is dragged across the screen.
Words stream behind the user’s fingertip for an intuitive and enjoyable experience.
No need to write out words, so easier than conversing by writing.
Other functions can be added for innovative and diverse applications.
Works with other functions for wider applicability

The technology can be combined with a picture-drawing function to create illustrations, maps and other graphics augmented with text, enabling richer expressiveness and enhanced understanding.
The addition of multilingual translation would enable displayed words to be translated for face-to-face communication with speakers of other languages.
Possible Applications

With picture drawing

People with hearing disabilities cannot watch the speakers’ mouth and hand movements simultaneously, making it difficult to understand a conversation involving spoken words and hand movements. Mitsubishi Electric’s new technology allows a listener with a hearing disability to continue looking at the screen while the speaker talks and gestures with their hands.

With multilingual translation

Words displayed on the screen can be translated into other languages and then read out automatically.

With background images

Stored images displayed on a screen can be overlaid with text of the speaker’s words to explain the content of the images more clearly.