Skoltech Robotics Lab’s Postgraduate Artem Pavlov and graduate student Marko Simic published an article about their experience with the Google Tango developer kit, and their development of a remote control application for robots, made for this device. How could this technology be useful and how it will change the Pokémon Go experience? We talked to Pavlov and Simic, who are about to publish an article on this matter in September, on the popular Russian IT and computer science portal Habrahabr.

Google’s Tango Project has long been known, but just in case you didn’t know about it, it is a computer vision and localization platform for use in mobile devices. Using data from two cameras – with wide-angle lenses and a conventional depth sensor, which is a miniature version of the Kinect, accelerometer, gyroscope and barometer, Google Tango is able to perceive the surrounding three-dimensional space, and keep track of your position in it. This is a revolutionary technology, primarily because ATAP (Advanced Technology and Projects) engineers managed to fit all this equipment into a mobile device, and to develop a friendly SDK (software development kit) to high-level developer. It takes care about a basic hard work in data processing with sensors and carrying out the necessary reforms, allowing the developer to work with a convenient abstraction.

In the article, which will be published in Habrahabr, Artem and Mark describe their experience with the Google Tango SDK. We asked them to tell about their experience and impressions of this new technology.

Artem: We had a target for a week, because we didn’t have experience essentially. We decided to make something simple, but never the less good looking. Our idea was to create a remote-control app, so we can control robots, using virtual reality through Google Tango.

Q: What’s new at this task?

Artem: There are many different approaches to controlling robots, including augmented reality. For example, students from MIT in 2013, created a small project that used augmented reality and in this project we added virtual interfaces to the physical objects. So you can say that we follow the same principal of introducing into the augmented reality world, physical objects, while it’s usually the reverse way. We introduce into the physical world some augmented objects.

Marco: As Artem said, we only had the device for some days, which is a short period and we don’t have much experience with Tango. The whole thing with Tango is a novel concept, it has a lot of sensors in a small packaging.

Q: What defines or differs Tango from other systems?

Marco: Well, that was primarily a tablet device, like a big phone, but the special thing about it are the sensors it has inside of it. It has a camera that perceives depth, that can calculate distance from your device to an object, and it has two lenses – one regular, and one is wide-angled – which allows tracking the location and position of the device in space. Those are the main components of the Tango device. When you combine these systems, you get some interesting effects, like tracking the position of the device space and making 3D models of objects. That’s the first development kit that Google had, and they will have another one in a year.

Q: What did Google get from your experience?

Artem: They will get some PR from our work. They will be able to show what the device is applicable for, not only for games, but for real applications too. In today’s world, we have more and more robots in daily situations, like carrier robots, and we will be able to create intuitive interfaces, using augmented reality for controlling them. For example, it can tell you about a robot’s planned path, so it will not be a surprise for humans when the robot suddenly moves from the corner and collides with them.

Q: How long does it take to come out as a commercial device?

Marco: this summer will be the first commercial device, so it will be widespread probably.

Q: Will Pokémons be on a new level?

Artem: this is definitely true. In Pokémon Go’s case, you don’t see real augmented reality. It’s not real augmented reality because, for example, if you come close to the wall, and see a Pokémon 3 meters from you.

Marco: because right now, as Pokémon Go works, it doesn’t have a perception of real environment, it just overlays the image of the Pokémon on the camera image. It doesn’t know what you see from the camera. If you use a Tango device, for example, you can accurately position that Pokémon on a table, in front of the screen or on the floor.

Q: Will it look like a hologram?

Marco: No, it’s not going to happen that you, for example, position the Pokémon in mid-air. In reality, nothing hangs in the air, and right now, when you play Pokémon you can see that – it hangs all around. Sometimes it’s on the floor, but sometimes it just hangs in the air.

Artem: If you create an illusion of a hologram through precise localization, if you will move your tablet, the Pokémon module will change its perspective accordingly. It will be like a real 3D object in front of you. In addition, I think it’s necessary to mention the using of Tango capabilities. You will not only be able to position models in real space, but also you will be able to make virtual objects to interact with the real space. For example, if we continue to talk about Pokémons, Pokémons can run on the table, or jump from the table to the floor etc. It will be a whole new level of interactivity, and in this sense the Google Tango device is a unique one, in my opinion. It will re-define what we think of mobile devices.

Q: except games and remote control for robots, what are other applications of this system?

Artem: It can be doing 3D scanning.

Q: You mean, almost everybody will be able to have some kind of 3D scanner in their phone?

Artem: It’s not the most precise scanner, but it’s viable, for example, for determining if a new couch will suit to your room. It will be viable for measuring.

Q: human body as well?

Marco: With advancement, why not?

Artem: In several years from development, maybe, but not now. It’s more suitable for static environments.

Q: Are you pleased with your article?

Artem: It hasn’t been published publically yet, so it’s not available for everyone. I guess we did a good job, considering the time we had. We had several ideas which could be improved, like a robot’s planned path.