How Embedded Vision is Helping Build the Autonomous Mobile Robots of Tomorrow

This blog post was originally published at e-con Systems’ website. It is reprinted here with the permission of e-con Systems.

Cameras are a vital part of autonomous mobile robots. They are used in different environments such as warehouses, retail stores, hospitals, outdoors, and more. Get in-depth insights into the different camera-enabled functions of these robots, and learn how cameras become their ‘eyes’.

Advancements in artificial intelligence, edge-based processing, and camera technology have enabled robotics engineers to make huge leaps in building truly autonomous mobile robots (AMRs) that are capable of mimicking human actions. From autonomous navigation to spreading fertilizers and material handling, robots have been able to perform many tasks which were once possible only by humans.

Vision and cameras play a huge role in enabling this human-machine collaboration using robots in warehouses, retail stores, agricultural fields, classrooms, hospitals, factories, etc. In this article, let us learn in detail how embedded vision has made human life easier across multiple industries such as agriculture, industrial, retail, education, medical, etc., by making cameras the ‘eye’ of autonomous mobile robots.

Autonomous Mobile Robots – where they are used and how vision forms an integral part of them

As the name suggests, AMRs are robotic systems that can perform their tasks autonomously with minimal or little human supervision. And to do this, AMRs need to see and perceive their surroundings with superior quality, and in many cases take real-time decisions. This is where embedded vision technology comes into play.

Embedded vision refers to nothing but integrating camera modules into embedded devices (like a robot). One of the major (or probably the major) functions of cameras in AMRs is to help them navigate safely in their designated environment. As simple as it may sound, this involves using special types of cameras meticulously designed and integrated to deliver the right images and videos.

Some of the places and related tasks where autonomous mobile robots are used today are:

• Warehouses for navigation, material handling, inventory tracking, cleaning, etc.
• Outdoor for delivery, patrolling,
• Agriculture for plowing, weeding, harvesting, spreading fertilizers, etc.
• Retail stores for concierge services, curb-side pickup, shelf monitoring, etc.
• Offices, large buildings, hospitals, and restaurants for remote communication, indoor delivery, and various services

With the above in mind, let us have a quick look at how cameras equip autonomous mobile robots with vision capabilities in the different environments they are used.

How cameras help transform warehouse operations using robots

The shortage of labor has led to warehouse owners automating many tasks, and robots have played a key role in this by positively impacting material loading and unloading, goods transportation, picking and placing items, and more. Following are the different types of camera-enabled robots used in warehouses:

• Goods-to-person robots: These autonomous vehicles use cameras to measure depth for the purpose of seamless navigation
• Pick and place robots: They pick and place objects from and to warehouse shelves with precision. This is made possible by using cameras for locating an object’s position and in some cases identify them.
• Automated forklifts: These are camera-enabled forklifts that use vision to facilitate loading, unloading, lifting, and transportation of heavy objects.
• Inventory tracking robots and drones: These are used to monitor warehouse stock and inventory. Cameras are used in them for reading barcodes/RFIDs and object recognition.
• Robotic arms: These can be small or large arms that are typically used for industrial purposes – material handling, machine tending, heavy lifting, etc. Here, cameras help to detect, identify, and locate the objects to be handled.

Delivery robots – cameras for automating last mile delivery

Delivery robots are deployed either outdoor or indoors. Outdoor, they are used for delivering food, packages, and other items. While used indoors, they do the same in office spaces, hospitals, restaurants, etc.

In delivery robots, cameras are predominantly used for depth measurement and monitoring the items to be delivered. In the former, the cameras point outward, while in the latter cameras generally are placed inside the vehicle.

Patrol robots – smart surveillance on wheels

Patrol robots are smart surveillance devices that can guard buildings and compounds 24×7 to identify theft, intrusion, or any other unlawful activity. Since they need to have a complete view of the surroundings, a multi-camera system that can create a 360-degree view is typically used. The robots will also need cameras to enable safe navigation as they sometimes have to move around in crowded locations.

Smart farming with agricultural robots and vehicles

Examples of autonomous agricultural vehicles and robots include autonomous tractors, harvesting robots, automated weeders, etc. They need cameras to automate critical agricultural tasks like plowing, detecting bugs & weeds, spreading fertilizers, harvesting crops and plucking fruits & vegetables, measuring NDVI (Normalized Difference Vegetation Index), and more. As in the case of others, agricultural robots also use cameras for enabling autonomous navigation.

Enhancing shopping experience and retail operations using robots

In this era of phygital shopping, AMRs are used not just to offer a superior experience to shoppers, but also to improve staff experience at retail stores. Following are some of the ways in which AMRs do these:

• Concierge or companion services to shoppers: This helps them locate the items/section they are looking for.
• Curb-side pickup: Robots can deliver items to the curb-side pickup point at the retail store to fulfill online orders.
• Shelf monitoring: This is done with the purpose of monitoring shelf stock so that the retail staff can do a timely replenishment of goods and items.
• Remote communication and assistance: These are done using telepresence robots where staff can assist shoppers from remote locations. These robots can also be used for employee communication.

From the above functions, it is clear that cameras are used in retail robots for barcode reading, object recognition, navigation, and telecommunication. If we consider restaurants and hotels under the retail umbrella, robots find applications there as well in the form of delivery robots and companion (or concierge/service) robots.

How camera technology is positively impacting the medical industry through robotics

In the medical industry, robotics finds applications in telehealth devices, robotic surgery, laboratory diagnostics, drug delivery (automated medical carts) in hospitals, and more. Robots make life easier for patients as well as the staff by enabling effective communication, timely care & treatment, and carrying out medical procedures.

Cameras help these types of robots to perform various tasks such as picking and placing objects, patient-practitioner communication, reading barcodes and text, object identification, and finally navigation.

e-con Systems’ contribution in building cutting-edge cameras for AMRs

e-con Systems has been in the embedded vision space for over 18 years now, and has pioneered in designing, manufacturing, integrating, and supplying specialized camera solutions for autonomous mobile robots. Its wide portfolio of cameras for robots includes MIPI camera modules, GMSL cameras, and 3D depth cameras, just to name a few.

To learn more about e-con’s camera solutions for AMRs, please check out our AMR markets page. You could visit our Camera Selector to browse through our entire portfolio of cameras by features and applications.

If you are looking for any help in integrating cameras into you AMRs, please write to us at camerasolutions@e-consystems.com.

Gomathi Sankar
Head of the Industrial Camera Business Unit, e-con Systems

The post How Embedded Vision is Helping Build the Autonomous Mobile Robots of Tomorrow appeared first on Edge AI and Vision Alliance.