Volkswagen Group AI Research organises the 2017 Deep Learning and Robotics Challenge. The DLRC is set up in a collaboration between Volkswagen Group and NVIDIA. In this challenge, groups of students compete to solve a robotics and deep learning challenge. Winners will be announced at the GTC in Munich on Oct. 10. Follow their progress in this blog.
Please note that these contents were not edited. They do not necessarily express the views of AI Research or Volkswagen Group.

It was a blast

An overview of our project


Last monday was judgment day for our dear ROB, he needed to cluster bricks by color and put them into the correct box while evading objects. Furthermore people tried to confuse it by using print outs of lego bricks on carpet (did not fool ROB), leaves (did fool ROB) and …


We apply some image processing using OpenCV on each segmented frame: lego bricks are bounded by blue rectangles, obstacles by red rectangles and edges by yellow contours. Since the obstacles and edges are segmented as the same class, we identify an edge as an obstacle which …


The video above shows the results of training a Variational Auto-Encoder on LEGO blocks. The resulting clustering is not perfect, but is much better than other results we got previously. In previous attempts the main problem was that clusters focused more on shape than color.

The Variational Auto-Encoder has 3 …


Throughout this project we developed three different deep neural networks that were used to solve this challenge. Below we detail how we used each one.

  1. Semantic Segmentation Network
  2. Variational Auto Encoder
  3. Triplet loss clustering Network

Network 1 – Semantic Segmentation Network

This neural network takes an image from the camera and …