Four eyes are better than two! Robots are brilliant at handling monotonous tasks and less pleasant repetitive activities. Combined with image processing, they become reliable, "seeing" assistants to humans. They are used in quality assurance for component inspection, they help with assembly and positioning of parts, they detect errors and deviations in production processes and thus increase the efficiency of entire production lines.
An automotive manufacturer uses the opportunity to improve cycle times on its press lines. VMT Vision Machine Technic Bildverarbeitungssysteme GmbH, Mannheim, has developed the FrameSense 3D robotic measuring system for fully automated container loading and unloading. This enables molded parts to be safely and accurately inserted into or removed from containers. Four Ensenso 3D cameras from IDS Imaging Development Systems GmbH provide the basic data, forming the basis for process automation.
The actual work process, which is to be automated using FrameSense, is part of many manufacturing operations. A component leaves a machine - in this case, a press - and travels on a conveyor belt to a container. There, it is stacked. As soon as the container is full, it is transported with it to the next production step, such as assembly in a vehicle.
Until now, employees have been responsible for loading the containers. This relatively simple partial task is more complex than it first appears. In addition to the insertion process itself, the first step is to determine the appropriate clearance for the part. At the same time, any disruptive factors, such as locking devices, have to be eliminated, and the "loading crate" has to be inspected for defects.
All these tasks must now be handled by a robot equipped with a vision system - a technological challenge. In addition, the containers come from different manufacturers, are of different types and therefore vary in size.
Control of type, shape and position using four 3D cameras
For fully automatic loading and unloading, the position of several relevant container features must therefore be determined for a so-called multi-vector correction of the robot. The basis for this is control of the type, shape and position of each container. This is the only way to guarantee process safety and avoid collisions when guiding the loading robot. All this has to be integrated into the existing production process. Delays must be ruled out, and component positioning must be accurate to the millimetre.
To remedy this, VMT uses no less than four 3D cameras per system. The four sensors each record part of the entire field of view. This can be made up of two containers, each measuring around 1.5 x 2 x 1.5 metres (D x W x H).
Two cameras always look at the same container. This allows data to be obtained from two perspectives for a more detailed 3D point cloud. Point clouds from all four sensors are combined for subsequent analysis. Relevant features of the container are recorded in areas of interest (ROI, Regions of Interest) of the global point cloud. A registration is the precise determination of the position of a feature using a model in all 6 degrees of freedom.
Other ROIs are searched for disturbing contours that could lead to collisions during loading. Finally, the overall image is compared with a stored reference model. This makes it possible to check the condition and position of containers simultaneously and fully automatically. Even containers that have been delivered deformed or positioned at an angle can be processed.
All this information is also recorded for use in a quality management system, which enables the status of all containers to be traced. Calibration as well as measurement data collection and subsequent evaluation are carried out on a dedicated IPC (industrial PC) with on-screen visualization, operating elements and connection to the corresponding robot controller.
The main result of the image processing solution is multivector correction. The robot is then corrected so that it can place the component in the nearest and most appropriate storage position. Secondary results are error messages due to the presence of disruptive edges or objects in the container, which would prevent filling. Damaged containers, whose general condition is too poor, can be identified and eliminated using the data. Image processing takes place in the MSS (Multi Sensor Systems) image processing software developed by VMT. FrameSense is designed to be easy to use, and can also be modified directly on site for other components.
Robust 3D camera system
On the camera side, VMT is relying on Ensenso 3D cameras - initially on the X36 model. The current FrameSense development is equipped with the Ensenso C variant. The reasons for this change are first and foremost improved projector performance - thanks to a new projection process - and higher capture speed. What's more, the Ensenso C allows a larger measurement volume. An important criterion for FrameSense, as the robot can only approach the containers to be filled up to a certain distance. The specifications of the Ensenso C therefore correspond exactly to VMT's requirements, as Andreas Redekop, Project and Technology Manager, explains: "High projector power and resolution, combined with fast data processing, were our main technical criteria when choosing the camera. In addition, being installed in a fixed housing was also an advantage."
The Ensenso C camera meets today's challenges in the automation and robotics industry. Compared with other Ensenso models, it provides color information in both 3D and RGB. Customers benefit from even more precise image data. The robust 3D camera system housing meets the requirements of protection class IP65/67. With 5 MP resolution, it is available with baselines currently up to approx. 455 mm. This means that even large objects can be reliably detected. The camera is quick and easy to operate. It is primarily intended for high-volume applications, for example in medical technology, logistics or manufacturing automation.
Thanks to automatic container loading and unloading and integrated 3D container inspection, manual workstations can be automated using FrameSense. Against the backdrop of a shortage of skilled workers, the system can therefore make a significant contribution to process automation, not least in the automotive industry. It thus meets the industry's key challenges. The Ensenso C camera provides the decisive basis for data generation, and exceeds the requirements of many applications.
Lukas Neumann, from Product Management, sees their added value: " The high power of the projector and the high resolution of the sensors are particularly advantageous in intralogistics. Here, high-precision components need to be captured at great distances and with large measurement volumes. "
For other unstacking or bin-picking applications in traditional logistics, it's possible to imagine a similar camera with high projection power, but lower resolution and fast image capture.
So there's nothing to stop further developments and automation solutions in conjunction with "sighted" robots.