Kinect To Read Lips, Sign Language


Posted August 6, 2010 - By Stephen Johnson

Video Kinect's Body Tracking & Video Shared Viewing

According to a patent application recently uncovered, Microsoft's Kinect will recognize American Sign Language well enough that those who "speak" sign language will be able to navigate menus, enter text, and even participate in chats in real time.

Even more amazing, the device will apparently be finely tuned enough to recognize lip motions and recognize what is being "said" by those who can't speak. Here's the example from the patent:

"For example, when the user kills another user's character, that victorious, though speechless, user would be able to tell the other user that he had been 'PWNED'. In another embodiment, a user may be able to speak or make the facial motions corresponding to speaking words. The system may then parse those facial motions to determine the user's intended words and process them according to the context under which they were inputted to the system."

Actual lip-reading has been over-promised by movies and TV for many years, so I doubt that the Kinect will "really" be able to tell what a person is saying just by "looking" at him and her -- your lips don't move distincitvely enough to really be readable by anyone, without the aid of the sounds you're making as well. But American Sign Language should be easier for Kinect to understand.. .in fact, I'll bet a computer understanding ASL is "easier" than a computer understanding and/or translating speech.

Either way, anything that makes gaming more inclusive to more people is good. I look forward to trashing talking and murdering my deaf brothers and sisters online just like I do everyone else.

A side note: Kinect will be able to count your freakin' toes. From the patent application:

"[Within the skeletal mapping system] a variety of joints and bones are identified: each hand, each forearm, each elbow, each bicep, each shoulder, each hip, each thigh, each knee, each foreleg, each foot, the head, the torso, the top and bottom of the spine, and the waist. Where more points are tracked, additional features may be identified, such as the bones and joints of the fingers or toes, or individual features of the face, such as the nose and eyes."

This may not prove to be a boon to hardcore gamers, but as technology, Kinect is very impressive. The thing will apparently know where you're looking.

Source: Edge Online


Tags: Kinect, Xbox 360
Kinect To Read Lips, Sign Language


Blog Tags

  • International Sexy Ladies Show: Messy Cat Fight

    Posted: January 27, 2010

    644,696 Views | 00:49

  • Sara Underwood's Naked Bike Ride

    Posted: June 22, 2011

    1,316,284 Views | 05:20

  • Laser Snake Robots, More Mars Rovers and BigDog Is Back

    Posted: September 25, 2012

    1,290 Views | 03:00

  • NBA 2K13 Launch Trailer

    Posted: October 4, 2012

    5,601 Views | 01:53

  • Casual Vomiting - Web Soup Investigates

    Posted: March 30, 2011

    7,570 Views | 02:52