The expert’s view: artificial intelligence

Today's manufacturing automation systems are mostly characterized by rigid specifications and have no intelligence or even the capability to develop intelligence, which is why the flexibility of these systems is still severely limited. The effort to plan, program and safely commission such a system is too high when there are frequent changes in manufacturing processes. This is in stark contrast to the growing need for highly individual and personalized products. I’m leading a group called "Cognitive Production Systems" at the Institute of Industrial Manufacturing and Management (IFF) at the University of Stuttgart, which is seeking to address this issue. We’re using machine learning and artificial intelligence methods to enable future manufacturing systems to adapt autonomously to changing requirements.

At the moment, we’re focusing on performing machine learning in simulation systems. This allows us to test the ability of manufacturing systems to learn new capabilities without actually using the physical system. A core challenge here is to bridge the so-called “reality gap” because the simulation is always an abstraction of the real world to some extent. So, if we learn in simulation and transfer the learned capabilities to a real manufacturing system, we still need to ensure that it works well and as desired.

A further focus of our work is to combine data-driven approaches like machine learning with model-driven or knowledge-based approaches. This is often called “hybrid AI” or “informed machine learning”. One example of this is to use a mathematical model of a particular manufacturing process on the macroscopic level, which is then extended on the microscopic level by learning procedures, for example to compensate for modeling errors or inaccuracies. At the IFF, we’re involved in a collaborative research project called KIRK (which, in English, stands for AI-based robot calibration), in which we use this idea to improve the calibration of industrial robots. Typical calibration procedures make simplifying assumptions about the robot and thus, form the macroscopic level I mentioned previously. Deformations due to wear, thermal expansion or high payloads, as well as mechanical issues in links, joints, and actuators, can influence the robot’s accuracy, but are often not considered during calibration. These microscopic effects are incorporated by means of machine learning models like neural networks.

At Fraunhofer IPA I’m heading up two departments: the Image and Signal Processing Department and the Center for Cyber Cognitive Intelligence (CCI). The first department focuses on quality control and prediction for manufacturing systems, for which we combine classical image processing algorithms with the latest machine learning techniques. The CCI in turn supports companies and especially small and medium-sized enterprises in applying AI and machine learning to manufacturing processes. The CCI is an AI research institution funded by the State Ministry of Baden-Wuerttemberg for Economic Affairs, Labor and Housing Construction. Its special focus is on the transfer from basic research to industrial application. One of the things it can provide to businesses is a “Quick Check”, which is a feasibility analysis of a company’s use case. A Quick Check can be followed by an “Exploring Project”, in which the findings of the feasibility analysis are used to create a prototype.

We’re also doing research on machine learning topics. One core focus of our research is “Dependable AI”, the aim of which is to achieve explainable, trustable, and robust machine learning solutions. For instance, advanced machine learning approaches like deep neural networks behave like a “black box” – in other words they’re opaque to the user. This behavior, however, is not desirable in many scenarios, such as safety critical applications or strongly regulated manufacturing processes. By means of explainable AI (xAI), it’s possible to increase the transparency of the inner workings of neural networks and other machine learning models.

I’m excited about so many aspects. For instance, there’s the enormous pace of development in the field of machine learning. Problems that were considered difficult five years ago are now solved. Just look at algorithms like AlphaGo (an AI computer program that defeated the Go world champion) or GPT-3 (a language model that produces human-like text). In addition, it’s becoming increasingly easy to use even advanced AI algorithms. This allows for swift experimentation with different algorithms to solve a problem at hand. It’s nice to see that more and more manufacturing companies are thinking about employing AI in their manufacturing processes.

From an algorithmic perspective, I’m excited about reinforcement learning. When people speak about AI, they often mention the term “self-learning algorithms”. Reinforcement learning is actually the subfield of machine learning that comes closest to this term. It’s inspired by the way humans learn: trial and error. Just think about learning to walk. In the beginning, a child tries to stand up and will often fall down. When this works well, it will begin to take small steps by holding onto objects such as chairs or tables. Again, the child will fall down many times. But from all these attempts to stand and walk, it learns over time until one day it can chase a ball in the garden. This principle is used in AlphaGo. We also use reinforcement learning in our research to build AI systems for automated robot programming or customer order processing.

When the pandemic hit, we thought very hard about how we could use our AI knowledge to help. One thing we came up with is the so-called “Access Checker”. It’s a contactless device that measures a person’s vital signs. It consists of a thermal camera to measure the core temperature and a radar sensor to measure the breath and heart rate. The radar sensor in particular is different to many existing similar systems. Machine learning is used to analyze the recorded thermal images and radar signals. We built the system during the first wave to support a local hospital. The hospital staff had to get up close to many people to check their vital signs, which posed a high infection risk for the staff. The Access Checker allows vital signs to be measured within a distance of 1.5 meters, which significantly reduces the risk of infection.

From the prototype developed in the first wave, we continued to improve the system and we just recently found a company that is now making a product out of it.

There are many advances that hold great potential, but currently the big breakthroughs are clearly in image processing and computer vision. Quality control is a key process in the manufacturing industry, whether it be inline or end of line. In many cases, cameras are used for quality control. The development of a good image-based quality control algorithm is a hard task and requires significant expert knowledge. Classical image processing for instance requires a well-defined specification of the defects that need to be detected. Using convolutional neural networks instead can reduce this effort significantly. Rather than developing an elaborate defect specification, it’s sufficient to provide defect examples.