Skip to main content

Serious Games As 3D Multi-Touch Applications

Via: NewScientist - Multi-Touch Display Can 'See' Objects Too

London, Oct 19: Scientists at Microsoft Corporation’s lab in the UK have developed a prototype interactive touch-screen that can see and recognize one’s hand movements and anything near its surface.

The screen acts as a two-handed touch interface and a crude infrared camera. Users can operate the display with both hands, in a similar manner to the display in the film Minority Report, say its developers.

However, the screen can also recognize particular hand gestures as well as objects placed within a centimeter of its surface, they say.

“It can sense much more than fingers, and is essentially a low resolution scanner and camera,” said lead researcher Shahram Izadi, adding that the screen can even communicate wirelessly with other devices nearby using the same infrared technology it uses to see.

He said they developed the technology, dubbed ThinSight, by adding an extra layer of electronics behind a normal laptop screen.

Sensors in position behind the display

This added a couple of centimeters to the overall thickness, but completely transformed its abilities, Izadi said.He said the screen “saw” by using a grid of paired infrared sensors and transmitters that sit just behind the backlight of the laptop's LCD panel.

The sensors formed crude images when infrared light bounced off an object, which allowed the screen to identify hand gestures or to see objects, and let them interact with onscreen images, he added.

Detect and Display

"This is early days for this approach – most prototypes are still small and slow," says team member Steve Hodges. "In a decade there will be a lot more screens in every environment," he adds. "Why not let them detect, as well as display, information?"

Other multi-touch and imaging interfaces have been developed previously, but these often use cameras placed behind the display screen or behind the user, which is cumbersome. Apple's iPhone, can detect multiple touches through changes in electric fields, but cannot take images of an object.

ThinSight can also work as a wireless infrared transmitter and receiver, responding to an ordinary TV remote or other infrared-enabled gadgets. Data could be sent to and from the screen using a PDA or cellphone too, Izadi says. This could let a user operate the screen remotely or send images for it to display

Multi-touch screens and surfaces support the work of people who increasingly rely on visualization or interactive multimedia to do their jobs. In addition, 3D multi-touch applications created for laptops and tablet PC's might be good for education, training simulations, and serious games.