Skoltech’s Intelligent Space Robotics and VR/AR Laboratory (Skoltech Space Center) led by Professor Dzmitry Tsetserukou presented two projects at the high-profile online ACM SIGGRAPH Asia 2020 Conference and Exhibition on Computer Graphics and Interactive Techniques which was held on December 4-13.
The Emerging Technology section featured 20 most interesting projects from around the world, including two Skoltech applications.
Skoltech’s first project, ZoomTouch is a robot-human interaction system that allows an operator or a group of people to control the robotic arm from anywhere in the world using a video calls application and DNN-based hand tracking. No glove or hand tracking device is required. All that is needed is a laptop with a camera and Zoom app.
“Video conferencing has become ingrained in our everyday life. But can it help to communicate with robots? We have shown that robots can be controlled via Zoom using a standard camera and a sophisticated DNN-based algorithm for real-time gesture recognition. In our video, the robotic arm picks out samples in a lab, which is a valuable feature when it comes to studying viruses or other hazardous substances. A user can control the robot from their desktop computer without leaving the office. A salient feature of ZoomTouch is that several users can jointly make a design, for example, build an architectural, life-size clay or 3D printed model of a car using a swarm of robots. Literally, humans can jump from a digital world to a tangible one!” Professor Tsetserukou says.
The application was developed by a team of Skoltech Master’s and PhD students, Ilya Zakharkin, Arman Tsaturyan, Miguel Altamirano Cabrera, and Jonathan Tirado.
“The second project, MaskBot, is the first robot with real-time projection mapping capability. MaskBot detects the user’s face with the aid of a computer vision (CV) algorithm and projects images of different features, such as makeup, glasses, jewelry, masks, and the like, onto the user’s face. This technology allows getting an adversarial image dataset in automated mode for the deep neural network (DNN) to perform face detection. When viewed from the side, the process looks like snake charming: the robot precisely follows the user’s head motions without taking its eyes off their face – a mesmerizing sight to behold! In the future, MaskBot can help build art installations with a 3D art object illuminated by a swarm of robots fitted with cameras and projectors. Also, MaskBot can imitate object textures and colors for training CV algorithms oriented towards Industry 4.0,” Professor Tsetserukou says.
The MaskBot developers team includes Skoltech Master’s and PhD students, Miguel Altamirano Cabrera, Igor Usachev, Juan Heredia, Jonathan Tirado, and Aleksey Fedoseev.
Online demos of the two projects garnered high praise from the ACM Siggraph Asia 2020 audience. The killer demo of ZoomTouch showed a Skoltech lab with the robot alternately controlled by gestures in real time via Zoom by Ilya and Arman, each at his own remote location.
Links:
ZoomTouch https://arxiv.org/abs/2011.03845
MaskBot https://arxiv.org/abs/2011.03844
Contact information:
Skoltech Communications
+7 (495) 280 14 81
