Risk Assessment for Artificial Intelligence Applications in Manufacturing

doi:10.21203/rs.3.rs-2130791/v1

Download PDF

Research Article

Risk Assessment for Artificial Intelligence Applications in Manufacturing

https://doi.org/10.21203/rs.3.rs-2130791/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

This paper proposes an approach for assessing the risk associated with the deployment of models based on artificial intelligence methods in manufacturing environments. A technique for defining deployment scenarios, identifying perturbations of the models in the training and deployment phase under particular conditions, and an estimation of the damage associated with the model output is developed. As a result, measures for minimizing the risks are assessed under consideration of the likelihoods. For validation purposes, two real manufacturing scenarios are defined, and possible errors of their existing models are weighted by their likelihood, allowing consistent risk assessment across various manufacturing domains. In particular, the robustness of two artificial-intelligence-based models to changing industrial environments and fluctuating product properties are assessed.

Risk Assessment

Artificial Intelligence

Deep Learning

Process Mining

Manufacturing

Risk assessment is becoming a pressing issue due to the deployment of artificial intelligence models in manufacturing environments. Failure mechanisms of a specific model must be identified, as well as the likelihood of failure occurrence and the degree of damage produced by a failure must be assessed [1]. In particular, artificial intelligence (AI) has increasingly affected many risk types, including models and compliance, operational, legal, reputation, and regulatory risks. Many of these risks are new and unfamiliar in industries without widespread analytic use and established model management history. Even in industries with a history of managing these risks, AI manifests the dangers in new and challenging ways [1].

Moreover, employees delivering suggestions based on artificial intelligence (AI) systems, the risk is not that one piece of individual information is biased but that, if the AI recommendations are biased, the manufacturer is systematizing bias into the decision-making process [2]. Furthermore, AI is becoming widespread and decentralized across the plant, making it difficult for many manufacturers to track. However, AI solutions are increasingly embedded in vendor-provided software, hardware, and software-enabled services deployed by individual business units, potentially introducing new, unchecked risks [3]. Unfortunately, there is little consensus on the failure of artificial intelligence models and how to evaluate them in manufacturing environments.

This paper aims to propose an approach for assessing the risks associated with the deployment of artificial intelligence models in real use cases of manufacturing and the derivation of measures to minimize the identified risks. On the basis of the risk definition, this approach allows deployers of AI models to define possible risks (perturbations) and their magnitude in terms of likelihood, so that realistic test scenarios can be set up. Therefore, we developed and implemented a method that defines deployment scenarios (use cases). The AI methods are evaluated under specific conditions, and an estimation of the damage associated with the AI-model output is assessed. The result of our method pretends to support deployers' decisions on whether an AI model is suitable for a real application.

Next to estimating the risks of AI models, the method presented can also be used to understand the failure modes of a particular model. Exemplary, two manufacturing scenarios are assessed within this paper. The manufacturing scenario in Chap. 4 focuses on an individual process step using machine learning. In contrast, in a second use case, the usage of decision tree methods for the company-wide-process analysis and optimization is evaluated (see Chap. 5). Specifically, in this paper, we make the following contributions:

Development of an FMEA-based approach for managing the risks of AI applications in manufacturing.
Standardization of the risk assessment of AI applications in industrial environments.
A likelihood of occurrence together with the potential effects of the faults and their detection, key risk indicators are defined.
The method is implemented and evaluated on two real cases to validate its application and benefits on different forms of AI, namely machine learning and decision tree methods.

2.1 Methods of AI in Manufacturing

In the context of manufacturing, there is a need for innovative solutions to increase (product) quality of manufacturing activities while costs are reduced. In this regard, artificial intelligence (AI), leveraged by Industry 4.0 and key enabling technologies, such as the Internet of Things, VR and AR, has already been proven helpful in today´s manufacturing [4–7].

In 1959, machine learning (ML), as a subfield of AI, gives “computers the ability to learn without being explicitly programmed” [8]. ML explores the study and construction of algorithms that learn from and make predictions on data [9]. ML is often compared with data mining, which focuses more on exploratory data analysis. Indeed, ML methods devise complex models to lend themselves to forecasting/prediction [10].

One approach that utilizes methods and Data Mining to optimize business processes is process mining [11]. By extracting event data from event logs of process-aware information systems (e.g., Enterprise Resource Planning Systems, Supply Chain Management Systems), it enables the automatic discovery and modelling of actual business processes, the evaluation of conformance towards existing process models, the determination of process deviations as well as the extension or improvement of the existing process model [11–13].

However, the complexity of other tasks, which has much more permutations, a different form of AI had to be invented, namely deep learning (DL). The use of this expression in Artificial Neural Networks was introduced in 2000 [14]. DL, also known as deep structured learning, studies artificial neural networks and related machine learning algorithms that contain more hidden layers [15]. It belongs to the broader family of ML-based on learning representations of data, which attempts to make better representations and create better models. Deep neural networks have demonstrated their ability to outperform other ML algorithms on tasks such as object recognition [16].

AI offers a variety of opportunities, but AI-based approaches are currently integrated into existing production processes to a minimal extent. One reason for this is the risks of AI-based applications in manufacturing. In this article, we propose a methodology for analysing risks belonging to AI applications in existing manufacturing environments. Then a more effective use of AI in that the related risks are better known and can be controlled and managed is suggested.

2.2 Methods to Risks Assessment in Manufacturing

Today, AI can offer solutions and heuristics for problems that are still unsolvable or intangible, e.g., for case-specific use for NP tasks like calculating the fastest way through production in a very short time. These are based on a "knowledge gain" of the AI and patterns in terms of a heuristic. They do not necessarily cover 100% of the problem space. This leaves a risk of wrong decisions, which must be considered and evaluated before a new AI or AI-based method can be used extensively. This does mean not only testing as it is currently done in autonomous driving but also considering the risks of wrong decisions of the AI in daily operation. The tests can only reduce errors and not exclude them. The risk assessment must consider the dangers of the area of application but also the quality of the AI training environment. Existing approaches to risk assessment and risk management as well as error correction procedures form a first basis for the risk assessment of AIs.

A standard method to analyse and understand such risks is the “Failure Mode and Effects Analysis” (FMEA) [17]. Disaster analysis methods are also checked to understand the distribution of events such as breakdowns and decision errors. The procedure we propose is based on previous risk management experiences in several domains. This paper considers AI methods within the scope and their application to gain experiences in real application cases. Prediction and management of errors in decisions are essential for AI risk management because AI provides findings based on learned patterns. Therefore, this method also included the learning phase into the approach. A rule-based AI could be understood as the check of the rules regarding completeness, correctness, and consistency (non-contradiction). An additional aspect is the self-learning behavior of some AIs, which can be seen as a specific learning environment with more uncertainties.

From the research perspective of the authors, the following research question is in focus for the scope of this paper:

How can an FMEA-based approach help identify and evaluate the risks of AI applications in manufacturing environments to develop recommendations for risk management and reduction?

The use of risk management to predict and manage critical events became a common approach in industry motivated by ISO9001-2015 [18, 19] and SOX404 [19]. The related procedures need to manage risks and disasters, empowering the use and evolution of risk and incidence management approaches. It became even more important with challenges for supply chains, environmental regulations, and temporary lockdowns [20]. In terms of AI, it supports or automates decision-making. This can initiate wrong decisions eliciting accidents and misleading behaviors.

AI is still a new technology within industrial processes, and quite often, the risks are underestimated because of the challenge to realize a well-working AI. Therefore, the related risk management is sometimes seen as an obstruction for the new development. However, more and more AI components and applications are part of manufacturing devices, e.g., to control robots, smart devices, barrier-free machine interfaces, or decide the next production step. Moreover, new concepts like pervasive computing and the internet of things create large networks with fuzzy boundaries. AI can use such an environment but will become less and less understandable. These requests to better understand the potential effects of AIs, especially in terms of their industrial usage.

In this article, we consider under the term AI not only decision support systems and autonomic decision systems in the sense of rule systems like process mining but also neuronal systems from the area of deep learning. The risks of these technologies are essential to understand in terms of avoiding the appearance of disasters and being prepared for potential misbehavior of the AI in an environment.

3.1 Procedure

The following procedures are proposed to analyze and evaluate risks and incidences appearing related to the use of AI. This chapter covers the use of AI to help humans provide better solutions and AI to improve processes with new services.

An intention to use AI is the reduction of failures/faults and the acceleration of the (process) decision. For example, spoiled food has to be sorted out of unspoiled food in the food industry. In this case, the pattern matching of a trained AI can be beneficial and reduce the risk of producing poisonous food. On the other side, the AI might still have problems identifying unexpected parts such as stones, screws, and nails that are easy to detect from humans. Expanding the range of objects found may result in too many right things being sorted out. The effects need to be clearly understood. Decision failures and final related costs have to be analyzed. This fact calls for risk analysis and its continuous management.

The procedure, illustrated in Fig. 1 and explained in detail in the following sub-chapters, provides a way to analyze and evaluate the risks in different steps. The strategy aims to understand the effects of the AI-related to its (application) environment as well as the impact of the learning phase on the AI. We defined a sequence of steps inspired by FMEA as a starting point to understand better the effects of an “intelligence” in the scope, e.g., the misinterpretation of a pattern. Then potential effects and results are identified. Most of the AIs have a machine learning concept that supports the establishment and evolution of the capability of the AIs. The related learning environment can have different forms, such as, e.g.:

Emulation of the real environment where the AI can get experiences by try and error (Reinforcement Learning)
The provision of a set of qualified data (Supervised Learning)
Classification and pattern reorganization via target functions (Unsupervised Learning).

We considered all these learning forms as a learning environment which could also be the real work environment or a simulation. To view this learning aspect, we added a separate path for the learning environment in Fig. 1 because of the strong influence of the learning environment to the trained AI.

The steps ‘Scoping’ (S1) and ‘Observation’ (S2) are applied to the considered AI system as a whole, as the subject matter is the environment the AI system will be used to as well as the consequences that may arise from the application. In contrast, the steps ‘Function and error analysis’ (T3 and L3), ‘Network and error analysis’ (T4 and L4) and ‘Measure analysis and evaluation’ (T5 and L5) focus on the specific function and network errors as well as the respective mitigating measures. Hence, these steps have to be conducted for the Trained AI and the Learning Environment, respectively. Finally, the identified risks of the Trained AI and the Learning Environment have to be managed holistically (S6). Questions at each step support their application and clarify the specific scope of a stage.

3.2 Scoping

Scoping (S1) describes the area the AI will be used in. It includes potential restrictions and borders of its usage, including examples of patterns and rules, which the AI should learn, and expected benefits from its use. The chapters about application cases (see Chap. 4 and 5) provide details about the instantiation of the scope for a specific AI. The scoping step is also in charge of delivering the configurations for the different stages because a trained AI requires a training environment and related tests. A self-learning AI needs to consider the real world as a training environment, and a rule-based AI might only need to check the rules. The processing of the step is supported by answering the following questions:

What will the AI do?
Where will the AI be used?
What should the AI learn?
What is the used AI technology?

3.3 Observation

The step of observations (S2) is taken out of the scoping to have a more prominent view of the potential risks and disasters. Two perspectives, (1) current and known risks and disasters within the scope as well as (2) the projected help of the AI together with its potential effects of wrong decisions, are considered. A specific approach will be used in the next steps on functions and network risks. A path from the analysis of disasters is the creation and analysis of previous events recorded in incident logs. Incidents should be considered if people initiated them in the same scope or if there is prior experience with AIs in this context. The following guiding questions with this step are:

In the scope, which incidences do appear by humans or AIs?
What are the expected effects?
What is the difference between Human and AI intelligence in the target situation?

3.4 Functional and error analysis

The function analysis (T3) covers the analysis of components influenced by the AI and significantly affected by decisions of the AI. This step includes identifying modules of the AI in the learning phase and in the later production phase. The relationship between a training environment and AI functions must be understood, e.g., between a rule system and its application.

The analysis needs to consider also necessary infrastructure, e.g., sensors and interfaces, which could influence the AI. A further important point is the continuous learning of an AI or the update process. Continued learning is a function that can be implemented in AIs, and it is sometimes necessary. The question is how the AI evolves, or is it better restricted to learning. It depends on the specific environment and scope of the AI and needs to be under control in terms of potential risks. This should be considered in the learning environment analysis, although in this case, the learning environment is the real world. Guiding questions for the functional analysis are the following:

Which components does the AI system consist of?
Which functions are provided by the components of the AI system?
How is the expected influence on incidences? E.g., sensors for pattern detection, analysis of sensor signals, interpretation of the signals, imitating action, etc.
How can a function fail?
Exist an evolvement function of the AI and is a consideration regarding learning environment needed?

3.5 Network and error analysis

Considering the network function (T4) focuses on analyzing potential errors or conflicts between the AI and its environment. Related effects from the surrounding world on the AI as well as the impact of the AI on its environment. Examples are temperature influences, light marks, energy fluctuations, and attacks on AI. The source might be the working environment or the learning environment of the AI.

A particular case can be the learning phase within the working environment or the ongoing learning within the working environment. Here also impacts the relevant learning results and needs to be considered potential root courses in the learning effect analysis. The result should include influences on the AI in terms of the settings of its rule sets and the patterns established within a deep learning AI.

During the discussion concerning the use cases, this procedure step appeared as a complex task. Therefore, we propose using different analysis methods and detailed sub-steps, illustrated in Fig. 2 (as the extraction of Fig. 1). The external influences are analyzed based on possible incidents and data of past incidents identified in the step ‘Observation’ (S2). Next, the consequences of the external influences for the respective AI system are derived, followed by an analysis of the results the AI system would come to under those external influences and what actions the AI system would take. Finally, the error consequences of those actions as well as the effects on the environment, are discussed.

The details of this approach are described within the use cases. To explore the implications of AI's interconnectedness with its environment, the following questions are helpful:

What is the effect of AI on the environment?
What is the interaction between AI and its environment?
Which are the patterns learned by the AI?
What are the effects of interface information like input / output issues, e.g., sensor input mismatch or no reaction to initiated action?

3.6 Measure analysis and evaluation

So far, the potential incidences, errors and related effects are identified. In the next step “Measure analysis and evaluation” (T5) the focus is on their relevance in terms of risks for an enterprise and how these risks can be reduced or avoided. This includes the detailed results about affected functions and modules of the AI and its environment from the previous steps. It targets the parameterization of the risk analysis and to find measures to reduce risks and potentially increase the benefits of the usage of AI by improving its application.

The application scenarios in this article use the following approach related to FMEA for the evaluation:

Occurrence: The probability of the event of the incidents or errors needs to be approximated.
Meaning: Calculation of the importance of the potential effects of the incidents or errors related to its criticality. For example, a car could simply stop due to a detected fault. So, a fault occurs, but probably with little risk of damage. Instead, it might decide to break hard, causing other vehicles to crash. That would be a significant financial risk.
Recognizability: Understanding how fast and predictive incidents or errors can be recognized.

The following scheme (Table 1) is used for parameterization in the application scenarios.

Table 1

Criteria for risk parameters
Value	Occurrence	Meaning	Recognizability
1–3	probability above 5x	low effect	immediate
4–6	probability within 5x	medium effect	high
7–9	probability within 2x	high effect	low
10	probability within x	critical survival	Not possible

The two application scenarios in the following chapters express the evaluation's detailed usage. The multipliers and steps were derived via common procedures from the literature and corresponding experiences of the authors in the application field. Afterward, they were harmonized for both use cases. The following form calculates the specific risk:

Occurrence * Meaning * Recognizability = Risk

This formula provides a priority related to the importance of avoiding the risk. This is addressed by specific measures to be implemented. The following questions need to be answered in this step:

Which actions reduce the risk of failure and critical incidences? E.g., Redundant sensors and corresponding input balancing
What is the proverbiality of risks appearance between maximum disruption and expected benefits? E.g., a high reduction of accidents but a low occurrence of other accidents would be an overall advantage if the accidents are similar.

3.7 Management

The measures to reduce the risks are important and need to be managed across each risk. The risk management (S6) should be repeated in a periodical way to check the effectiveness of the measures, understand potential further risks, and take into account AI updates. A question to be answered is the following:

What is the process to ensure minimizing risk over lifetime?

3.8 Learning environment and learning effect analysis

Learning environment refers to a set of learning technologies that satisfy learning patterns in data collections. A learning environment needs to be set up and predefined to learn, i.e., relevant features and disrupt irrelevant ones. Then the learned model needs to be evaluated before being deployed in a real environment.

The learning environment of AI has an essential influence on its behavior and functionality. It can take different forms, e.g., a simulation environment where specific application situations are performed, a large amount of annotated data, or even learning in the real world. Therefore, we have included additional steps for this environment, which basically follow the steps "Function analysis", "Learning effect analysis" and "Measure analysis and evaluation". Only the "Learning effect analysis" is significantly different from the previously described steps.

The learning effects need to be understood primarily for neural networks. However, it is also important for rule-based AIs to know about the rule system and the pattern targeted in the rule system. It is important to know the relation between training and the real situation and the effect of the training. In the case of learning in real situations, the effects of continuous learning and the (new) AI behavior need to prospect. Here it is also important to understand what cannot be predicted., which means the limits of control. The following questions are starting points to guide the analysis of the learning effects (L3-L5):

How can the training effect be understood?
How is the mapping between the training and the pattern created within the AI?
Which effects are not predictable, and how to deal with them?

Risk management needs to cover both the working environment and the learning environment. For the learning environment, important aspects are the classification of data and interferences with the training's activities in terms of matching “right” patterns.

Producers in the animal feed industry, depending on market availability, use different natural ingredients (seeds) to form a mixed feed with a constant and controlled quality. Despite the fluctuation of the components, the animals must receive the exact amount of nutrients, such as proteins, fats, etc. on a daily basis. The ingredients and the consistency of the feed have an essential effect on the produced feed. Therefore, natural raw materials (cereals) must be individually assessed and analysed for the specification process [6, 7]. Depending on the natural material quality, experienced staff adapts the processes by applying the “truth is not so much taught as caught” principle. After a long professional experience, many of them are able to interpret noises and feel the fine-tuning of process parameters by touching the feed pellets. Many process settings have high tolerances to ensure that the final product quality complies with the customer requirements. Further, experienced employees regulate manual process variables as a function of the existing product quality and customer requirements. However, this solution has two significant drawbacks. First, the employee´s knowledge is not recorded. Second, they only consider fulfilling the product´s requirements without understanding the root causes of quality damages. As a result of inconsistent product properties, a deep model was integrated in the production line to analyse and evaluate automatically the feed quality [6]. The possible risks caused by integrating a deep model in feed processing are analysed, and this paper's procedure is evaluated in a real use case.

4.1 Risk Assessment of the Trained AI

S1 Scoping

Whole grains are crushed into flour in a feed process with a so-called roller mill using counter-rotating rollers [6, 7]. The particle size distribution is one quality property of the produced flour after the crushing process. The ingredients and the feed's consistency have an essential effect on usability and animal health. Quality deficits in particle size may cause digest issues to animals. A particular focus lies in retaining the quality of the feed product under fluctuating cereal properties. To identify the quality of natural raw materials (cereals) and then gain clarity about the process behavior, deep learning methods are to be deployed into a feed processing plant. First, deep learning techniques on product data can detect quality poorness in the produced feed. Product quality poorness refers to heterogeneous quality. Then the identification of control variables in the process, such as the roller speeds, can be optimized for improving feed quality. These models can vary the control variables so that the product quality (particle size) is improved through a proposal of better process settings.

S2 Observation

From the product perspective, changing properties of raw materials over seasons is a continuous issue in a feed production plant [6, 7]. Grains as natural materials may have different harvesting or drying times, so elasticity is different from grain to grain. The elasticity of the grain can cause the roller mill to behave differently. Modeling the elasticity of each seed of a feed formulation, or even the formulation samples, is too expensive and complex to measure. In this sense, the grain's elasticity and/or size is not included in the modeling of a product behavior. As a result, correlations between elasticity and control variables of a process are not reflected in the model, so that the best proposal of process settings to support a better product quality cannot be suggested. As a result, one potential incident can result in a wrong particle distribution in the feed, either too fine or too rough, resulting in product rejection. Product rejection refers to unnecessary production costs and resources waste.

One further example of a potential incident is the inclusion of a new feed formulation. New variants of formulations are unknown to the (AI) assistance system, so the new product behavior can result in a wrong process setting proposal. The AI system assisting operators derives misleading settings, resulting in high costs and damaging the company image. The quality of new products (formulations) is then not retained, as the new textures or moistures are not previously known.

T3 Component function and error analysis

The analysis of the critical components and required modules for an AI-based model in a manufacturing environment presents some of them.

Data acquisition, which collects the data from sensors, machines through interfaces to a database
Data preparation, to select and/or discard representative features of data and then create a model
Process control unit, which feeds the predictions of the models into the control unit for a better setting of the process

The identification of control parameters affecting the quality needs to be explored on data collections. Hence, data are acquired. Historical data of the process settings and product qualities are collected. The more variability in the data, the higher the quality of a model to reproduce the real behavior. In this step, interfaces to the data are not always available or the quality properties of the product are difficult to measure. Further, high tolerance sensors or wrong calibrations can extremely affect the data value.

For the data preprocessing, it is needed to augment the value of the data: Measurements can be filtered out of outliers, irrelevant datasets can be excluded and/or data gaps are to be filled. Proving that all relevant features to your target area in the data is needed. However, getting to know the datasets and its features is not easy. Incorrect interpretations of values, overlooking tolerances or misleading unit values can become an issue in the analysis.

After some modeling and optimization techniques, the AI system can assist operator innervations during production. Either the process will be automated on the basis of the model results or the model proposals will be visualized to assist workforces on the manual settings.

In total, the system proposals can only be as good as the data used. Process operators cannot understand all the relationships in the process and product features and rely on the data collections, which at some points may be incomplete or imprecise. In terms of transparency, visualization of data and model results in the form of suggestions allows operators and managers to understand all the correlations in the process better.

T4 Network function and error analysis

In the presented approach, an AI system's relevant components and functions need to be analyzed as a network. Then, the AI system itself needs to be further understood.

As an example, in the network function and error analysis, the potential error “incomplete measurements as a result of a cold winter” is further considered. Incorrect input data to a deep AI model means that the model interpretations will be inadequate. The correlations within the data are not valid and the suggestions given by the model result invalid. Invalid predictions mean, for the particular case of feed processing as shown in Fig. 3, that the use of the AI-model as a reference system for the manual operations of workforces cannot be applied anymore. Because of inexperienced operators or automatic processes that override some of the process parameters provided by the AI model, two possible errors need to be overcome. On the one side, higher production costs because of rework or resources waste. On the other side, wrong settings of the process configurations can lead to low quality of the product or even a total product rejection.

T5 Measure analysis and evaluation

In this section, previously identified risks need to be minimized. The risks can be prevented by more intensive maintenance of the sensors collecting data or interfaces. For instance, system updates and/or calibrations are also relevant for continuous maintenance. Furthermore, tools and libraries treating the data and creating value from them in the form of (new) knowledge need to be continuously updated and avoid deprecation of the model. Especially for upcoming products, that is, new recipes to be integrated into the AI model. Its previous learning environment and data collections need to be further used and new data are integrated, requiring more memory and processing units. On an experienced basis, the potential risks were listed in Fig. 4 and evaluated on the foundation of event occurrence, meaning and visibility.

4.2 Risk Assessment for the Learning Environment

In the following, the risk assessment results for the learning environment are presented, starting with a function and error analysis. The learning environment in the compound feed manufacturing use case has four different phases that add value to the learned model. Failures in these phases can affect the later use of the model for predicting and optimizing its outputs in compound feed manufacturing. These failures do not necessarily only affect the process settings but can also have a fatal consequence for the feed quality and, therefore, on the health of the animals that consume this feed.

L3 Function and error analysis

In the scenario of feed processing, the learning environment consists of four phases as described:

Training data acquisition, which represents the data collections of relevant process and product features for training a machine learning model
Training data preparation, selecting relevant features for the model and labeling them, i.e., cereal type or/and (un-)crushed seeds.
IT-infrastructure and model training, which includes the possible different hardware the model is trained on and also the selection of hyperparameters for the model
Evaluation tools, which measure the model performance in the form of metrics, i.e., accuracy or/and losses

The component of each phase in the learning phase and examples of possible errors are illustrated in Fig. 5.

For example, a failure in outlier filtering during the training data preparation can lead to a wrong model that does not predict the production quality accurately. This can be a potential risk for the later use of the model in production.

L4 Learning effect analysis

In this second step of the analysis, the effects of the previous potential errors on the learning effect are illustrated. Possible errors in the learning phase in the compound feed affect the model creation and how well the model maps the underlying process. If the model behavior is not as close as the real process, the use of the model in the production environment will lead to errors caused by the wrong trained AI that affect the whole process. One example of the risks caused by potential errors and their relation to the model training errors is shown in Fig. 6.

L5 Measure analysis and evaluation

To mitigate the errors and reduce their risks of occurrence, actions can be defined for the four steps of training in the compound feed learning environment. These measures are shown in Fig. 7 for every step of the learning phase.

Some of the above examples are a matter of time and better infrastructure. A better understanding of the process and its correlations to the product quality require more data, that is, extensive data. On the other hand, treatment of large datasets requires more memory and powerful hardware.

Last, potential errors are rated due to the risk of occurrence, the meaning of the possible error and recognizability of the caused failures. The predefined scale is defined in Chap. 3 for both use cases. As an example, the results are shown in Fig. 8.

4.3 Risk Management (S6)

Today, the AI-based model in feed processing presented in this paper is deployed at the company side. Current results show the need for a continuous adaption of the model, that is, refreshing the model with actual data prevents potential errors, and at the same time, it offers constant learning of current settings, changing process environment, and new coming products (portfolios).

The business environment of machinery and equipment manufacturers is characterized by intense competitive pressure and tight market conditions. Suppose small and medium-sized enterprises (SMEs) want to be successful in this turbulent environment. In that case, they must be able to react quickly and confidently to the individual customer requirements and short-term changes, such as the establishment of trade barriers. On the other hand, quality and delivery reliability dynamics in cross-company value chains must be mastered. Establishing trusting and secure cooperation between partners is also a prerequisite. The opportunity lies in using process mining to analyze deviations from the target process – that are otherwise not identifiable – at an early stage, to create feasible alternative courses of action, and thus to be able to act more quickly and effectively. Furthermore, based on the analysis of actual processes, the process optimization potential can be identified and exploited along the value chain. The following is an excerpt of applying the procedure for risk assessment of AI in manufacturing to the assistant system for fast and reliable process optimization enabled by process mining.

5.1 Risk Assessment of the Trained AI

S1 Scoping

The process analysis and optimization system are designed to perform two main functions: The identification and analysis of process anomalies as well as the proposal of process alternatives based on the analysis. In order to identify process anomalies and influencing factors in the process environment, deviating process behavior and critical influences need to be monitored and evaluated. Indicators in the field of supply chain management could be deviating delivery time, quality, and quantity. Subsequently, feasible and suitable process alternatives need to be proposed. Possible measures to be taken in case of quality deficiencies of supplied material could be demanding suppliers comply with certain standards or purchasing goods only from certified suppliers. Another measure could be to intensify the incoming goods control or to change the supplier altogether.

The key benefit lies in the combination of process environment monitoring and analysis instruments with the automated generation of processes alternatives, which enables the transparent analysis of actual processes, the identification and exploitation of optimization potential, and faster reactions to disruptions and critical influences. The main potential restriction is the high requirements towards repetition rates. It is still open for production companies to apply repetition rate and variant complexity process mining effectively.

For a company-wide risk assessment of the use case at hand, the three main process areas, where the assistant system is to be implemented, will be considered: the supply chain management processes, the order management processes and the manufacturing processes.

S2 Observation

In terms of the supply chain management processes, one example of possible incidents caused by a process analysis and optimization system is the delayed delivery of components by a supplier in three consecutive months. Even though the supplier had prepared the delivery on time, the transport of the goods was delayed due to, for example, snow storms in December, January and February. However, in this case, the process mining system only takes influencing factors of the immediate process environment based on internal data of the Supply Chain Management System (SCM) into account, such as the delivery time, amount and quality as well as, for example, reaction time to complaints. As external influences from the corporate environment are not being considered, the root cause for the delay cannot be found. Hence, the assistance system may identify the outlier but derives a misleading pattern and concludes that the supplier had become unreliable and should be replaced. However, such replacement, especially if it is actually unnecessary, leads to great efforts and high costs for the search for a new supplier. It also represents the risk of not finding another reliable supplier with exactly the product portfolio needed.

A further example of a misleading pattern being derived from a random outlier without an adequate analysis of the root causes is the sudden increase of demand for a certain customer-individual product adaption by chance, including the certain product adaption into the product portfolio solely based on the fact that it had been ordered several times in a row risks high costs for the adjustments that are necessary along the value chain as well as high efforts for the maintenance of product data in relation to the low return generated by the additional product variant.

In terms of the manufacturing processes, the process analysis and optimization system could propose inadmissible optimizations. If most products meet the quality criteria, only very few products are being sorted out. Hence, the system could propose skipping the quality control process to improve the lead time and reduce costs. However, the misleading pattern derived from the standard processes without an adequate analysis of possible consequences of exceptions could lead to high costs and image loss if customer requirements are not met, and a return campaign is necessary.

T3 Component function and error analysis

The process analysis and optimization system being developed builds on process mining and is planned to consist of eight components, which functions are outlined in the enumeration below:

IT systems (e.g., SCM), which support the business and production processes of a manufacturing company and thereby provide log files
Interface/data, for mapping different log files to a common information structure
Filter rules, which reduce the input of data to a manageable amount (by, e.g., common logic or fuzzy logic)
Data preparation rules to design a uniform model
Analysis rules (Reasoner), for the analysis of consistency and identification of anomalies
Identification of measures, for the derivation of actions for managing the process anomalies
Realization of measures, for the semi-automatic application of identified actions
Representation of results, which illustrate the effects of measures and gaps

The component and error analysis results for key components of the process analysis and optimization system are illustrated in Fig. 9. Based on the observations described above, one example for possible errors that immediately comes to mind concerns the component “Analysis rules”: Too much weighing of outliers (such as the delay of delivery for three months, when all previous deliveries were on time) without an adequate root cause analysis (such as the analysis of possible environmental impacts). Another possible error that is supported by the observations is the disregard of the consequences of analysis results by the “Reasoner” (such as a product return campaign).

When the system proposes identified measures based on the analysis results, the inadequate questioning of proposals prior to implementing measures is a possible risk for the semi-automatic measure realization. Process managers may not fully understand the mechanisms behind the system and may also become too trusting of the system. In terms of the representation of results, the error analysis showed that the effective illustration for the user is not to be underestimated. Incorrect interpretation of the semantics of the representation may be misleading, even though the underlying analysis and measure derivation is adequate.

T4 Network function and error analysis

As described in the S1 Scoping section, the functions of the process analysis and optimization system are the identification and analysis of process anomalies as well as the proposal of process alternatives based on the analysis. In addition to the possible errors caused by the individual components, the AI systems need to be examined as a network. In Fig. 10, one possible error is illustrated using the method introduced in Chap. 3.5.

In this case, the AI system identified and analyzed the sudden and extraordinary increase in customer demands for a specific product. According to the data available to the AI system and the registered reaction measures taken in order to meet the demand, a pattern was derived: From this year on, every August, the demand for the condensate pump increases and needs to be completed. Hence, the AI system proactively proposes preparation measures in order to better meet the demands in the future. However, the root cause for the anomaly could not be identified and the anomaly was weighted too much. Moreover, the possible consequences, such as the storage costs as well as the costs for a short-term increase of production capacities in terms of set-up costs and the effects on other products and customer orders, were not taken into account and may lead to strong losses.

T5 Measure analysis and evaluation

The following Fig. 11 contains exemplary measures for two of the errors described above. As a change of supplier can lead to high costs and risks as well as the loss of trust in the supplier network, the exemplary possible error in the category component analysis in Fig. 12 has an important meaning to the company. The underlying risks are that, for example, the search for a new supplier interrupts the production and the goods and services of a new supplier are worse than those of the previous supplier – which was mistakenly replaced. However, this would not pose the company front of existential risks. Hence, the meaning does not reach the maximum level of 10, but the level of 7 instead due to the specific constellation of the incidents that may lead to the possible error. If the underlying measure unit for estimating the occurrence was one month, this possible error could occur every six months or more. However, without root cause analysis, it is estimated that the possible error is difficult to identify due to the automatic and correct processing of data,

For the exemplary possible error identified by the network analysis, the meaning would be even higher, as it would create high costs for resources and production and therefore have a significant impact on the company's profitability. The occurrence is also estimated to be high due to the high frequency and variety of environmental influences. Even though the incidents are not recognizable in advance, first signs are expected to be identifiable comparatively early, leading to moderate recognizability.

In terms of measures, the root cause analysis should be included as a further extension of the AI system in order to avoid too much weighting of outliers and the direct derivation of faulty patterns. This could start with a manual analysis of root causes before implementing automatic analysis mechanisms. In order to avoid the weighting of outliers without the consideration of possible consequences and their effects on the company, continuous clearance management of company related data should be implemented.

5.2 Risk Assessment for the Learning Environment

In contrast to the deep learning system described in the feed processing industry use case, the Process Mining System considered in this use case is rule based. Hence, the main purpose of the Learning Environment is to test if the rule system is operating according to design and expectation and make adjustments accordingly. In the following, the risk assessment results for the learning environment are presented, starting with a function and error analysis.

L3 Function and error analysis

The learning environment for the Process Mining System consists of four interconnected components (see Fig. 12):

Testing of inference, for checking the rule system for contradictions. Identified conflicts must be eliminated manually or – if available – with machine learning algorithms.
Definition of test cases, for the design of use cases that are as close as possible to the future application of the Process Mining System.
Testing of rule system with pre-defined test cases, for the analysis, if the rule system is operating according to the expectation (reasoning).
Adaptation of rule system, for further developing the Process Mining System based on the generated test results.

As illustrated in Fig. 13, the testing process repeats itself until no more adaptations are necessary.

The definition of test cases plays a vital role in the learning environment for the rule system because design errors of the Process Mining System may not become apparent in the testing process. If, applied to one of the mentioned observations, for example, the Process Mining System does not distinguish between manufacturing processes and quality control processes, and the sample processes in the test cases do not include quality control processes, it may not become apparent that optimization rules cannot be applied to manufacturing and quality control processes equally. However, if the test cases included quality control processes, the need for adaption would become clear. The Process Mining System would propose to skip an ineffective quality control process to improve the lead time and reduce costs.

Another possible error concerns the possible side effects of adaptions to the rule system. If, in the example described above, the rule system was adapted to distinguish quality control processes from other processes, the adaption may influence (or even contradict) another interconnected rule tree. In turn, depending on the designed test cases, this influence may or may not be identified in the subsequent test iteration.

L4 Learning effect analysis

The learning effect depends largely on the design of the test cases as well. The Fig. 14 depicts the influence of testing criteria on the learning effect.

As explained in the previous chapter, adaptions are made to the AI if the rule system is not operating according to expectation (reasoning). This is done based on testing criteria that are linked to the test cases to optimize the AI system. However, focusing solely on reaching the expected results may lead to the disregard of other parts of the results (side effects). If, for example, the test cases are designed to evaluate if the Process Mining System can identify inefficient production processes and instead propose alternative approaches, it may be overlooked that the proposed alternative methods require more complex internal logistics and are therefore not more efficient overall.

L5 Measure analysis and evaluation

Figure 15 below contains an exemplary measure for one of the possible errors described above. The incomplete or irrelevant definition of test cases can be regarded as one of the most problematic possible errors, as the entire learning process could be conducted effectively. What’s more, this possible error may often occur because a complete consideration of all factors is only possible in very few cases. The identification of the possible error is most difficult if the same development team, which developed the AI system, also creates the use cases.

In order to avoid the design of incomplete or irrelevant test cases and avoid the testing criteria being too focused on expected results and therefore overlooking side effects, one possible measure is the process-oriented definition of test cases. A process model of the application scenario could be used as a basis for defining a complete and relevant use case. This could be supported by taking historical data and possible future incidents into account. Furthermore, involving the designated users of the AI system into the test case and test criteria definition may enable an evaluation of the AI system from another perspective.

5.3 Risk Management (S6)

The AI system considered in this use case is being developed in the framework of a research project. A continuous analysis and evaluation process will follow the development and implementation of the AI system.

This paper presented a decision-making approach based on the FMEA, a popular technique in this field, to analyze and experiment with how an FMEA-based approach can help identify and evaluate the risks of AI applications in manufacturing environments to derive recommendations for prioritizing errors. On the foundation of this definition, we developed a procedure that identifies attainable errors and defines penalties associated with the potential drawbacks of the evaluated AI models. The approach lets practitioners evaluate the risks of an AI system formally. Further, an initial setup of deployment scenarios provides the first outcome to tackle the problem of design certificates for AI-based models in manufacturing environments.

The authors provided two case studies derived from research projects they performed in the industry. The case studies are used to analyze the applicability of the presented approach. It includes the definition and motivation for using risk indicators. Risk indicators provide priorities related to the importance of avoiding the identified incidences. These indicators also investigate the robustness of an AI model, as they measure the effectiveness of different perturbations. As an example, while incomplete test cases decrease accuracy under new events, an update in the training datasets increased robustness for future events. The application of the risk assessment method showed an extensive analysis of AI models in real scenarios, not only in the deployment phase but also in the learning phase. The learning phase of an AI model is crucial, as such models learn on the basis of previous events.

Despite the fact that the case studies come from various branches – machinery, feed processing- and the models belong to different areas of AI, we used the same factor characterization. It remains to create meaningful factors for other applications when it comes to the risk evaluation. The set of risks to occur after integrating the AI system into a real environment are not necessary of the same nature.

An important aspect for future research is reducing time, effort, and investment for the risk analysis within the AI development. The development of technical support of the specific risk management for AI would be needed to reduce its effort. This might be extended in later research with insurance of the investment and maintenance for continuous learning as well as service level agreements related to AI.

An upcoming point is a more straightforward integration of the approach in future AI development, also covering more the effect of the cooperation between humans and AIs. Aspects for the topic “How we stay in control and make the right decisions in a digital world” need to be added because currently, the focus is on AI in its prospected environment, but what does it mean for the required or expected human capabilities related to the AI.

Acknowledgments

The research and development project mentioned in this article is funded by the German Federal Ministry of Education and Research (BMBF) within the “Innovations for Tomorrow’s Production, Services, and Work” Program (02P18X000) and implemented by the Project Management Agency Karlsruhe (PTKA). Further, the authors would also like to thank the Federal Ministry for Economic Affairs and Climate Action (BMWK) and the Project Management Juelich (PtJ) for funding the use-case projects "AI-supported platform for the assistance of production control for improving energy efficiency" - KIPro (funding code 03ET1265A) and "Increasing energy efficiency in production through digitization and AI" - ecoKI (funding code 03EN2047A). The authors declare they have no financial interests and are responsible for the contents of this publication. All named authors agree the order of authorship.

Funding

Ethics approval

Not applicable.

Conflicts of interest/Competing interests

All authors contributed to the manuscript and approve the version of manuscript. The authors state that the submitted work is original. The manuscript in part or in full has not been submitted or published anywhere. It will not be submitted elsewhere until the editorial process is completed.

Consent to Publish

The authors affirm that the results are presented without fabrication and ensure that all the authors mentioned in the manuscript have agreed to authorship, read and approved the manuscript, and given consent for submission and subsequent publication.

Availability of data and material

The use cases supporting the described approach includes data that are available on request from the corresponding authors related to the specific use case from IPK or BIBA. The data are not publicly available due to industrial partners in the related projects. The use cases are only examples to reflect the usage of the developed method. References used in the other parts of the study such as web links are available under the reference section of the article.

Code availability

'Not applicable

Authors Contributions

All authors worked on the whole text of the article. The use case in chapter 4 has been elaborated and contributed by M. T. Alvela Nieto and D. Bode. The use case in chapter 5 has been elaborated and contributed by J.-A. Scholz. The method description in chapter 3 has been elaborated and contributed by Frank-Walter Jaekel.

Schwerdtner P, Greßner F, Kapoor N et al. (2020) Risk Assessment for Machine Learning Models. In: McAllister R, Weng X, Omeiza D et al. (eds) Machine Learning for Autonomous Driving Workshop at the 34th Conference on Neural Information Processing Systems (NeurIPS 2020)
Corbett-Davies S, Goel S (2018) The Measure and Mismeasure of Fairness: A Critical Review of Fair Machine Learning. ArXiv abs/1808.00023
Amodei D, Olah C, Steinhardt J et al. (2016) Concrete Problems in AI Safety. ArXiv abs/1606.06565
Cioffi R, Travaglioni M, Piscitelli G et al. (2020) Artificial intelligence and machine learning applications in smart production: Progress, trends, and directions. Sustainability 12:492. https://doi.org/10.3390/su12020492
Nabati EG, Alvela Nieto MT, Decker A et al. (2020) Application of Virtual Reality Technologies for Achieving Energy Efficient Manufacturing: Literature Analysis and Findings. In: Lalic B, Majstorovic V, Marjanovic U et al. (eds) Advances in Production Management Systems. The Path to Digital Transformation and Innovation of Production Management Systems, vol 591. Springer International Publishing, Cham, pp 479–486
Alvela Nieto MT, Nabati EG, Bode D et al. (2019) Enabling Energy Efficiency in Manufacturing Environments Through Deep Learning Approaches: Lessons Learned. In: Ameri F, Stecke KE, Cieminski G von et al. (eds) Advances in Production Management Systems. Towards Smart Production Management Systems, vol 567. Springer International Publishing, Cham, pp 567–574
Alvela Nieto MT, Nabati EG, Thoben K-D (2021) Energy Transparency in Compound Feed Production. In: Dolgui A, Bernard A, Lemoine D et al. (eds) Advances in Production Management Systems. Artificial Intelligence for Sustainable and Resilient Production Systems, vol 634. Springer International Publishing, Cham, pp 496–503
Samuel AL (1959) Some Studies in Machine Learning Using the Game of Checkers. IBM Journal of Research and Development 3:210–229. https://doi.org/10.1147/rd.33.0210
Kohavi R, Quinlan R (2002) Data mining tasks and methods: Classification: decision-tree discovery. In: Klösgen W, Zytkow JM (eds) Handbook of Data Mining and Knowledge Discovery. Oxford University Press, Inc, USA, pp 267–276
Wujek B, Hall P, Güneș F (2016) Best Practices for Machine Learning Applications. In: SAS Institute Inc. (ed) Proceedings of the SAS Global Forum 2016 Conference
Dakic D, Stefanovic D, Cosic I et al. (2018) Business Process Mining Application: A Literature Review. In: Katalinic B (ed) Proceedings of the 29th International DAAAM Symposium 2018, vol 1. DAAAM International Vienna, pp 866–875
van der Aalst WMP, La Rosa M, Santoro FM (2016) Business Process Management. Bus Inf Syst Eng 58:1–6. https://doi.org/10.1007/s12599-015-0409-x
van der Aalst W, Adriansyah A, Medeiros AKA de et al. (2012) Process Mining Manifesto. In: Daniel F, Barkaoui K, Dustdar S (eds) Business Process Management Workshops. Springer Berlin Heidelberg, Berlin, Heidelberg, pp 169–194
Aizenberg I, Aizenberg N, Butakov C et al. (2000) Image recognition on the neural network based on multi-valued neurons. Int C Patt Recog (International Conference on Pattern Recognition) 15:989–992. https://doi.org/10.1109/ICPR.2000.906241
Hinton G, Salakhutdinov R (2006) Reducing the Dimensionality of Data with Neural Networks. Science (AAAS) 313:504–507. https://doi.org/10.1126/science.1127647
Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet Classification with Deep Convolutional Neural Networks. In: F. Pereira, C. J. C. Burges, L. Bottou et al. (eds) Advances in Neural Information Processing Systems, vol 25. Curran Associates, Inc
Wu Z, Liu W, Nie W (2021) Literature review and prospect of the development and application of FMEA in manufacturing industry. Int J Adv Manuf Technol 112:1409–1436. https://doi.org/10.1007/s00170-020-06425-0
ISO (2015) ISO 9001:2015 Quality management systems(ISO 9001:2015). https://www.iso.org/standard/62085.html. Accessed 11 Jan 2022
United States federal law (2002) Sarbanes–Oxley Act Section 404: Management Assessment of Internal Controls: SOX 404
Kohl H (2021) White Paper »RESYST«: Resilente Wertschöpfung in der produzierenden Industrie - innovativ, erfolgreich, krisenfest, München

No competing interests reported.

Download PDF

Version 1

posted

You are reading this latest preprint version

Risk Assessment for Artificial Intelligence Applications in Manufacturing

Status:

Version 1

Abstract

Figures

1 Introduction

2 State Of The Art

2.1 Methods of AI in Manufacturing

2.2 Methods to Risks Assessment in Manufacturing

3 Method For Risk Assessment Of Ai In Manufacturing

3.1 Procedure

3.2 Scoping

3.3 Observation

3.4 Functional and error analysis

3.5 Network and error analysis

3.6 Measure analysis and evaluation

3.7 Management

3.8 Learning environment and learning effect analysis

4 Scenario 1: Feed Processing Industry

4.1 Risk Assessment of the Trained AI

4.2 Risk Assessment for the Learning Environment

4.3 Risk Management (S6)

5 Scenario 2: Machinery Industry

5.1 Risk Assessment of the Trained AI

5.2 Risk Assessment for the Learning Environment

5.3 Risk Management (S6)

6 Discussion And Outlook

Declarations

References

Additional Declarations

Status:

Version 1