This new technology enables users to touch objects in any video

Image Courtesy: designboom

MIT’s CSAIL lab has developed a new technique where viewers can touch and manipulate objects in videos. The effects are similar to person touching a real world object.

For example, if you are watching a video of a man playing a guitar, you will be able to use the mouse to strum the guitar yourself. The strings will vibrate similar to you strumming an actual guitar. The researchers behind the technology are calling it Interactive Dynamic Video.

The technique uses algorithms that can analyze vibrations given out by every object when captured by a video camera. Based on the analysis, it provides realistic prediction models that anticipate how the objects would react to movement.

Interactive virtual objects have been used before in video games and other interactive media. Microsoft’s hololens is great example of this – the augmented reality glass places digital objects in real world and enables users to make changes to them with gestures.

There is also the Roger Rabbit school of filmmaking, where virtual characters interact real environment.

However the ability to apply changes to videos captured using a traditional cameras is something extraordinary and only imagination can predict the number of application such a capability can have. Think about it, you are watching a video shot on top of a hill and you will probably able to use your mouse to throw the stone all the way to the bottom.

“One of the most important ways that we experience our environment is by manipulating it: we push, pull, poke, and prod to test hypotheses about our surroundings. By observing how objects respond to forces that we control, we learn about their dynamics,” the IDV website reads.  

The team specifically used the game Pokemon Go to give us a idea of how the technology can be used in augmented reality. Watch the video below: