This project is the first step in creating a scalable system for training a humanoid robot, Nao. The robot will learn new words and associations using knowledge it gained in the past. In this case, the robot has previously learned to identify colors that it sees. The robot sees the world through pictures taken by its cameras.
The color detection process is learned via an artificial neural network. Thus, Nao can look at an image and know where in the picture each color is located. For this example, I split the image into 3 vertical columns. The picture is passed through an image processing program that reviews all pixels in the image. The color for each vertical segment is determined along with their pixel positions. The robot can report back if it found a color and at what pixel locations the color resides. Since the pixel locations are known, then the robot could use that knowledge to learn words for those pixel positions. For example it could associate word like ‘on the left’ for the first column and ‘on the right’ for the third. Thus a new association is made such that the robot could now find a new color in a specific location. We could take this a step further and have the robot use colors and locations to understand shapes, and so on. The goal is to build a complex interconnected learning system. This project is the baseline for that learning system.