Machines That Sense, Think, And Act

Sep 27, 2018 By Frank Dellaert
Anonymous's picture

If you have built a simple robot from your lego kit or participated in First Lego League competitions at your school, you are familiar with the basics of robotics.

Robotics involves designing machines that can “sense, think and act” by themselves. It is a continuous cycle where you sense the world, think about it, and act – and your action will change the world, so you sense again, think and act.

Robots are machines that can do all three (sense-think-act) without human intervention. And if anything violates this principle, then it is not a robot. 


Sensing can be done in many ways. The most common and cheapest sensor is the camera. There are other sensors such as laser range finders (or LIDAR) that use laser pulses to understand their surroundings. The most popular application of LIDARs is the self-driving car, which often has a spinning laser sensor mounted on top. These sensors collect large amounts of digital data which need to be processed.

Computer vision is the field of interpreting this digital data to form a model of reality, such that a machine can act on it. So really computer vision is a part of the “thinking” aspect of the robot, and it allows us to plan actions. Finally, “acting” those actions are exactly is the mechanics of getting robots to walk, fly, crawl or drive.

Applications of Computer Vision

When we think of an application for computer vision, we usually imagine humanoid robots, drones or self-driving cars. The Roomba vacuum cleaner is an example of a household robot today. But did you know that you might be seeing computer vision in action every day? Google and Facebook analyze billions of images that are uploaded to their platform. They then implement face recognition and give suggestions such as “This is your friend -- would you like to tag them?” That’s the most common application of computer vision that most people are perhaps not even aware of.

What about specialized robots that assist in surgery or manufacturing? Well, they are not “true robots” yet because they are controlled by humans and do not think for themselves. 

Looking Into The Future

In the immediate future, I imagine we will see more and more autonomous vehicles like the Google self-driving car. Perhaps someday we can replace valuable parking spaces in big cities with affordable housing! However, there are still many challenges before this can become a reality. You see, a car that drives by itself has to get it right 100% of the time. An error of even 1% can be disastrous as accident rates can be high.

I believe it will be some time though before we see robot helpers at home assisting senior citizens, or people with disabilities. There are mechanical challenges in building a robot that can sense, think and act like humans. So significant advances in technology need to be made before we can build humanoid robots.

Robotics is a very interesting and emerging field with the power to transform society. And just as with any new technology, it is our responsibility to make sure that people do not misuse it for evil purposes or to inadvertently hurt humans.