Get out of my way!

Achievements 3 & 4


Hey Y’all! So, a lot has been changed since my last post. I’ve got a makeover which includes my vision sensors and most importantly I’m carrying my computational power with me anywhere I go - the Jetson TX-2! In the meanwhile I’ve also learned to identify a brick when I see one, I know many different kinds of obstacles which are available in the Data:Lab (such as tires, bottles, candy bars, etc.) and I know the surface of my workspace together with its borders. Pretty cool ah?! For more info about my learning process, check out the guys’ post here.

Achievement 3 - Locate known brick in work space

Current status: I can’t distinguish between different lego bricks on my own. They all look gray to me as you can see below.

Hence, someone had to do the job for me, and by someone I mean the guys. And you can’t imagine how much they don’t like to do so. But, in order to complete this achievement they have made an exception. My goal was to locate the red brick in the workspace. So for each brick which I had found, they calculated the mean HSV value of its pixels. In case this value was within the range of the red color, the brick was marked with a red circle, otherwise the circle was colored with blue. I must admit, I was a pain for the guys in this achievement. Even while wearing the cool sunnies I couldn’t handle the bloody light! It took them a while to find the correct range of the red color, which would be stable enough in different lighting conditions. I think I would have done way better, if they had told me to locate the purple brick ;)

Achievement 4 - Grasp known brick at unknown location

Let me describe the logic I was following: A frame was divided into three sections: left, middle and right (demonstrated in the image below).

The possible frames in the workspace were divided into four scenarios:

  1. Only background. In this case, explore the workspace. This is done by moving forward 20 cm and then performing 360 degrees turn. If nothing is found - repeat.
  2. The closest brick is red. Closest is defined as the lowest brick in the frame. The approach here is greedy: If the brick is on the left - turn left, if it’s on the right - turn right, if it’s in the middle - move forward. Once it is at the bottom edge of the middle section - stop and pick it up.
  3. The closest brick is not red. The objective here is to avoid the brick. If it’s on the left - turn right, if it’s on the right - turn left, if it’s in the middle - check. If it’s at the bottom edge of the middle section, you are too close and therefore you might want to move backwards. The problem with moving backwards is that you don’t know what’s behind you, unless you turn around and check. To avoid doing so simply stop, pick it up and put it out of your way (turn 90 degrees to your left, drop it and turn back). If it’s not at the bottom edge of the middle section you may still want to avoid it, but it depends if you spot the red brick in the same frame, and if so, you don’t want to miss it. Hence, if the red brick is in the frame - move forward, else you may choose randomly whether to turn left or right. Choosing randomly an action helps to avoid getting stuck in an infinite loop of turning left and right.
  4. An edge is in the frame. Unless you spot the red brick in the frame (and then you follow case 2 or 3), you have no reason to get closer. Hence, choose randomly the direction you to turn 90 degrees (left or right).

Check out how I nailed it last Friday.