The runaway hit of the summer Pokemon Go makes use of augmented reality to amaze players, and now MIT scientists have come up with some technology that could make the game even better.
Wandering around town and seeing augmented reality Pokemon in your normal environment is one reason behind the massive success of Pokemon Go. Now MIT’s Computer Science and Artificial Intelligence Lab has developed technology that could make these augmented reality apps more interactive.
PhD student develops interactive dynamic video (IDV)
The technique is based on making what happens in the virtual world affect the real world around us. The creator of the technique, PhD student Abe Davis, has dubbed it interactive dynamic video (IDV), and will publish his dissertation on the subject this month.
“This technique lets us capture the physical behaviour of objects, which gives us a way to play with them in virtual space,” said Abe Davis, a PhD student at Massachusetts Institute of Technology’s Computer Science and Artificial Intelligence Laboratory (CSAIL).
“By making videos interactive, we can predict how objects will respond to unknown forces and explore new ways to engage with videos,” Davis said.
Pokemon Go provides perfect demonstration of technology
Davis chose to demonstrate the possibilities of IDV using the hugely popular Pokemon Go app, and has shown how Pokemon can appear to interact with the real world around them. If you want to make a “real” object move in most forms of augmented reality, you have to make a 3D model. However making a 3D model from 2D video is simply beyond the processing power of our smartphones.
However Davis took a different approach by looking for vibrations in videos in order to work out at which frequencies objects move in real life. This includes swinging cables, the wobble of a bobblehead and the bouncing of a branch in the breeze. The technique does not demand too much computational power in order to make these objects move in video.
As a result, Pokemon could appear to move realistically through the real world, moving objects in their path on your smartphone screen. Aside from the potential use in Pokemon Go, there are other ways that IDV could come in handy.
Other uses of IDV also possible
For example in other augmented reality apps, the technique could be used to allow users manipulate “real” objects in what feels like a natural way. Another potential use is in filmmaking, for example in films that feature interaction between CGI and real-world characters. Here IDV could be employed as a way of getting better results even if the film is on a low budget.
Davis even believes that the technology could be used to save lives, by putting the algorithm to work in real-world applications. He thinks that by recording videos of old buildings and inserting them into simulations of hurricanes, it will be possible to see if the building needs repair.
“The ability to put real-world objects into virtual models is valuable for not just the obvious entertainment applications, but also for being able to test the stress in a safe virtual environment, in a way that doesn’t harm the real-world counterpart,” Davis says in the press release.
For fans of Pokemon Go, IDV could mean better gameplay. For others it could end up saving lives.
