10.2.1 Agent Systems for Production System Automation and Control

The application of agent technology for automation and control of production systems has been considered intensively for at least three decades, starting in the early 1980s of the last century (Shen et al., 2006; Lüder et al., 2004b). A large number of architectures have emerged, including holonic architectures (Deen, 2003) and PABADIS architectures (Ferrarini and Lüder, 2011), which are the most widespread from the authors’ point of view (see, for example, Barata et al., 2001; GRACE Consortium, 2011; IDEAS Consortium, 2012; Leitão and Restivo, 2006; Rodrigues et al., 2013). The latest developments are documented in Mařík et al. (2011) and Marík and Martinez Lastra (2013).

All these control architectures postulate the existence of agents as main system entities, established either as pure software agents or as combinations of hardware and software acting as agents, all with a set of important characteristics (VDI, 2010). Most discussed characteristics for the implementation of agent behavior are the use of an environment model, reactivity, proactivity, and mobility, all related to production system control.

The basis for the behavior of agents within production systems is a dedicated model of the environment of the agent (i.e., a model of relevant information about the production system) implemented within the agent. Based on this model, each agent can detect the behavior of its environment and respond to this behavior following its own aims and scope of actions. For the implementation of this structure within the field of agent-based automation and control architectures, the so-called believe-desire-intention (BDI) architecture, as presented in Weiss (1999), or special reductions of this structure targeting only a few of the agent characteristics (such as reactive agents) are exploited. If the environment of the agent will enter a state, which is of importance for the agent, the agent will react with a dedicated behavior (reactivity). Within production systems, reactivity enables the implementation of agents responding to requests with a dedicated set of actions because it is necessary for the execution of processing steps (resource agents). Likewise, agents can follow their own aims. To reach them, they will expose a certain purposeful behavior (proactivity). This enables the implementation of agents controlling the execution of orders of different natures emerging from outside the production system and therefore requesting the assistance (or process execution) of other agents in the production system. Examples of such agents are agents controlling the production orders in a production system (order agents) (Ferrarini and Lüder, 2011).

Mobility takes an interesting role within the characteristics of agents in production systems. They can be mobile in different ways. On the one hand, agents can get mobility features through their hardware as, for example, within agendized transport systems such as Automatic Guided Vehicles (AGVs). On the other hand, pure software agents can be mobile, exploiting special software technologies. An example is software agent migration as given in JAVA Agent DEvelopment Framework (JADE) (Bellifemine et al., 2007). A classification of migration capabilities of agents is given in Figure 10.1.

Agents responsible for equivalent control decisions necessary within production system automation and control are structured in similar ways and expose similar behavior across the different developed agent architectures. They have equivalent characteristics. Within most of the developed agent-based automation and control architectures, agents can be found responsible for control of production orders. These agents are proactive and partially mobile.

Agents can also be found responsible for the control of production resources. They are reactive and usually not mobile. In addition, you can find agents responsible for storing, maintaining, and providing information about possible products (including information on necessary materials and manufacturing steps and its control), as well as agents supporting necessary planning activities. Both types are typically reactive and not mobile.

In Table 10.1, the named agent types are opposed to each other. Their names within different agent architectures and their main characteristics are given. Here, only agent architectures following the holonic structuring paradigm, and architectures following the PABADIS structuring paradigm, are considered. Thus, it is far from complete.

To ensure the necessary flexibility of production systems, it is expected that agent systems can self-adapt to changing conditions and responsibilities. In general, there are different possible approaches to realizing this self-adaptability.

The most frequent applied approach for flexibility provisioning is the use of generic agent types. They are based on the behavior generation exploiting an environment model that consists of a basic model extended following the special purpose of the agent at runtime with additional knowledge about the environment.

Usually, agents are developed following the BDI principle (i.e., its behavior is based on an environment model and sets of long strategic and short-term aims that are sought with regard to the environment model). Therefore, the aims and the environment model can contain static and dynamic components which may vary during the life cycle of the agent. The dynamic components of the aims and environment models can be changed by the agents themselves, by other agents, or by entities outside the agent system. This changeability can be exploited to provide flexibility for the agent with respect to changing requirements and boundary conditions and thereby provide capabilities for the self-adaptation and self-learning of the agent system.

This possibility to provide flexibility within agent systems is used in the PABADIS’PROMISE architecture (Ferrarini and Lüder, 2011), as well as in the GRACE architecture (Leitão and Rodrigues, 2011; Rodrigues et al., 2013), to name two examples.

Within the PABADIS’PROMISE architecture, generic production order agents are equipped with a special description of the product to be produced and the order executed, including control code fragments required for the production resource control. At agent runtime, this information can be changed, leading to a change of the environment model. These changes can be executed either by other agents or from outside the agent system. In addition, responding to changing requirements and conditions, new agents with updated information sets can be started and old agents can be terminated. The execution of the necessary manufacturing process for the production order will be controlled by the production order agents in cooperation with the machine agents. These agents contain the second part of code fragments required for the production process control on the production resource associated with the machine agent. In cooperation, production order agents and machine agents will set up and execute an optimized control code for the specific product on the resource of interest (Ferrarini et al., 2006; Lüder et al., 2007).

A second possibility to increase the flexibility of agent systems is the changeability of the agent itself. Here, modern software technologies can be applied, enabling the change of the agent source code. One example is the class loading within Java. But, up to now, there are no agent architectures known to exploit this possibility. Here, further potential for interesting research questions are given.

For the design and engineering process of agent-based production system automation and control systems, there are different approaches available, such as the GAIA methodology (Wooldridge et al., 2000) and the GAIA-UML methodology (Lüder and Peschke, 2007) based on it, as well as the design of agent-based control systems (DACS) methodology (Bussmann et al., 2004). In the authors’ opinion, the DACS methodology is the most appropriate with respect to the combination of mechatronical thinking and mechatronical engineering.

The main aims of the DACS methodology are the design and specification of agent-based control systems for production systems (Bussmann et al., 2001, 2002, 2004). It is based on a sequence of three steps for the analysis and description of sets of necessary control decisions, the identification of agents executing these control decisions, and the specification of interactions required to enable the control decisions.

The first step of the DACS methodology aims at analyzing and describing the set of all necessary control decisions within the production system. Based on the definition of the level of the automation pyramid of the production system to be controlled, it will be considered which control decisions are required to ensure a proper behavior of the production system on the relevant level. In addition, all dependencies between control decisions will be summarized. It has to be identified which information must be exchanged between control decisions to make them decidable. As the main result of this first step, a decision model is established.

In the second step of the methodology, the set of agents required to execute all identified control decisions is specified. Therefore, the control decisions of the decision model are grouped and associated with agents. Consequently, necessary interactions between the agents will emerge, resulting from the necessary information exchange between control decisions in the decision model. Within the agent specification process, it is relevant to consider which characteristics the specified agents have to possess. If an agent is more a provider of functionality within the manufacturing system for other agents, it should be reactive. In contrast, an agent usually exploiting functionalities of other agents to ensure its own aims should be proactive. As a main result of the second step, an agent model is developed.

Within the final step of the DACS methodology, the interactions between the agents are detailed. Therefore, the special requirements and boundary conditions to the interactions between agents are identified and mapped to existing agent interaction protocols (which may establish a library). For each interaction, the best fitting interaction protocol is selected and, if necessary, adapted to the detailed requirements. As a main result, the agent model is extended by interaction protocols.

Based on the decision model and the extended agent model, the control system for a production system can be implemented. Therefore, an existing agent platform such as JADE (Bellifemine et al., 2007) or AMES (Theiss et al., 2009) can be used. Assistance for the selection of the right agent platform is given in Part 2 of VDI (2010), while the existing best practice is given in Part 3 of this guideline.

10.2.2 Mechatronical Engineering of Production Systems

Mechatronical thinking and engineering have their roots in similar developments in industrial countries in the late 1970s and early 1980s. In Germany, to give an example, the so-called Feinwerktechnik (precision engineering) emerged, covering the combination of mechanical and electrical engineering. Very quickly, the term mechatronic, originating from Japan, was adopted internationally for the advantageous combination of mechanical, electrical, and electronic engineering. Over the following years, many more engineering disciplines have been integrated, such as optics and information sciences (Harashima et al., 1996; Tomizuka, 2004).

Initially, mechatronics had been focused on the design and engineering of products where the meaningful combination of different engineering disciplines could provide an additional value for the functionality, stability, and so on, of the intended products (VDI, 2004; Panich, 2013). This combination has proven to be not only fast but helpful for the structuring, design, and engineering of production systems and beyond (Kiefer et al., 2006; Thramboulidis, 2008; Lüder et al., 2010b).

In recent years, a broad agreement about the definition of the term mechatronic has been reached. Following this agreement, it holds that:

A mechatronical unit is a closed system providing a dedicated (mostly physical) behavior within a production system utilizing sensors, actuators, and intelligent control devices in a closed-loop control structure. Thus, the mechatronical unit combines, on the one hand, software (for control program development) and hardware (mechanics, electrics, electronics, …) and on the other hand, different engineering disciplines to achieve an optimal function provision.

A mechatronical system is established using the systematic combination/interlinking of mechatronical units and/or mechatronical systems within a hierarchical structure. Thus, each mechatronical system will contain its own information processing, used for optimal control of the functionality and the interaction of the different interlinked mechatronical units and mechatronical systems of the lower layers.

The distinction between mechatronical units and mechatronical systems results from the consideration of the hierarchy of mechatronical units and mechatronical systems. Usually, the leaves of this hierarchy are regarded as mechatronical units, while all other objects in the layers above are regarded as mechatronical systems. But most important, the mechatronical units have direct access and control of the underlying physics of the production system. It depends on the system of interest as to whether a drive is seen as the mechatronical unit or a drive chain including the drive, gearbox, and frequency converter, or the complete conveyer with a lifting table.

The structure and interlinking of mechatronical units and mechatronical systems covering only two layers is depicted in Figure 10.2, to give a hierarchy example. In real application cases, sometimes only two layers, but often more than two, have to be considered.

Following Kiefer et al. (2006) and Lüder et, al. (2010b) the complete structure of a mechatronical-oriented production system can be represented by a six-layer hierarchy. The lowest of these six layers is formed by mechanical and electrical parts such as metal stiffeners, electrical wires, and screws. They are arranged in subfunction groups which, in combination with other subfunction groups, will provide basic functionalities of the production system. Thus, subfunction groups are grouped to function groups. For example, single clamping fixtures are combined to clamping fixture groups providing the production function “fixing material,” which is required in a robot-based welding cell or by combining a drive, a gearbox, a frequency converter, and some shafts within a drive chain to provide the function’s “motion.” Function groups will provide functionalities, which are of importance for the execution of production steps, but complete production steps, which are usually part of the bill of operation of a product, will be provided by main groups. Main groups integrate a set of function groups, as is the case for clamping fixture groups, drive chains, and other function groups within a milling machine. Together, they can execute a milling function on a work piece. The main function can now be combined to manufacturing cells able to execute sets of manufacturing steps. For example, a milling machine can be combined with a robot for material handling and storage for different milling tools in a milling cell. Finally, a set of cells can be combined to a site, just as a set of milling cells can be combined to an engine production site of a car manufacturer.

Usually sites, cells, main groups, and function groups can be considered as mechatronical systems, while cells, main groups, function groups, and subfunction groups can be regarded as mechatronical units. Here, the relevant viewpoint is essential for the definition of the lowest level of consideration, which will constitute the mechatronical units. The hierarchical structure is depicted in Figure 10.3.

The engineering process for mechatronic-structured production systems is executed with direct application of mechatronical units and mechatronical systems in the structure described earlier (Lüder et al., 2011; VDI, 2004, 2009).

Within this process, two main process parts can be observed. The first part of the engineering process is focused on the design, engineering, installation, and commissioning of a production system for a special production purpose (i.e., able to produce a special product portfolio). In the course of these activities, mechatronical units and systems (or parts of them) are exploited as a starting point for emerging from a library of reusable mechatronical units and mechatronical systems.

The second process part serves the design, engineering, and testing of reusable mechatronical units and systems and its integration in the named library (it should be reminded that this library is not a single entity but a distributed one exploiting different storing and management technologies). These mechatronical units and systems can be exploited within the first process part. The design, engineering, and testing of reusable mechatronical units and systems is based on the abstraction of engineering results of the first engineering process under inclusion of expert knowledge about the industrial domain the intended production systems should belong to (Maga et al., 2010).

Within mechatronic-structured production systems, the applications for production system automation and control are distributed among the different information processing components of the different mechatronical units and systems. This distribution can either be a physical distribution on different control devices or a virtual distribution on the same hardware but with different execution contexts. Thus, the control decisions executed on the different information processing units are oriented on the automation pyramid layers they belong to. Hence, on site-level enterprise resource planning (ERP) decisions and control functions are executed, on cell and main function-level manufacturing execution control (MES) decisions and functions are relevant, and on the different function layers factory floor control decisions are taken (Ferrarini and Lüder, 2011). For the design and engineering of the automation and control applications (as well as for the complete production system design and engineering), it is useful to specify a stable interface structure for the information process units as depicted in Figure 10.4 (Lüder et al., 2010b).

Mechatronical systems of higher layers of the mechatronical hierarchy can access lower-layer mechatronical systems using its own device interface and the execution interface of the lower-layer mechatronical systems. The lower-layer mechatronical systems will provide, over its execution interface, access points to its provided production functions (or parts of it), which can be accessed by other mechatronical systems. The higher-layer mechatronical systems know the required lower-layer production function and can access and parameterize it appropriately over its device interface. Thus, a distributed but clear control decision hierarchy is established.

10.2.3 Comparisons of Mechatronic to Other Approaches

As mechatronic has a long history, there are several approaches accompanying it. Currently, the most relevant are the holonic manufacturing system (HMS) approach, the IoT approach, and the CPSs approach.

The HMS concept has been developed in order to make the automation process (the “shop floor” as well as the upper layers of the automation systems) more flexible to changes and avoid a huge overhead on the control level. It assumes a distribution of control based on the single independent unit (holon), performing a certain task and communication with the other members of the community in order to manage production. HMSs do not have a specified architecture, but instead a set of terms, which have to be common for all systems within an HMS (VanBrussel, 1994; Deen, 2003).

Within an HMS, the holon is seen as an autonomous and cooperative building block of a manufacturing system for transforming, transporting, storing, and/or validating information and physical objects. The holon consists of an information processing part and often a physical processing part. A holon can be part of another holon. Thus, a holon hierarchy can be established.

Advances in HMSs can be found in Mařík et al. (2011) and Marík and Martinez Lastra (2013).

As such, the holon is very similar to a mechatronical unit because it is not easy to distinguish between the two. Nevertheless, the design focus is different because the HMS approach is mostly driven by control system engineering, while the mechatronic approach is driven by the engineering process of the complete system. It also covers more general information related to mechanical and electrical engineering, which is not relevant to the HMS approach.

The IoT approach and its child, the CPSs approach, target the integration of intelligence to physical objects. In the case of IoT, these are general objects (in most cases, consumer products), while in the CPS case these are production resources.

The IoT approach (Atzoria et al., 2010) enriches physical objects through communication interfaces and intelligence, enabling the unique identification of these objects and their object-driven supervision/control. The CPS approach takes this idea up a notch. Its special derivate of the CPPSs (Vogel-Heuser et al., 2014) intends to enable production system components to act independently and be self-contained within the production system, providing functionalities in a kind of service to the overall system.

Similar to holons, CPS is near to mechatronic ideas. But, as in an HMS, the design focus is mostly driven by control system architectures and the intention to reach a self-adaptable system.

Both holonic ideas as well as CPPS ideas can be applied in a similar way within mechatronical thinking as presented in the next section. However, this is outside the scope of this paper.

10.3.1 The Structure of Mechatronic-Oriented Agents

Agents will contain different behavioral pattern required to implement reactive or proactive behavior. Additionally, they contain an environment model that should be changeable to ensure system flexibility.

To be able to exploit the behavior capabilities of agents correctly and completely and to preserve the agent encapsulation, higher-layer mechatronical-oriented agents will access the agent behavior of lower-layer mechatronical-oriented agents by accessing the execution interface of the lower-layer agents over its device interface. They will transmit the necessary parameters and configurations and start, stop, and supervise the accessed functions of the lower-layer agents.

To enable the flexibility of mechatronical-oriented agents, its environment model will be divided into two submodels: a basic model and an advanced model. The basic model is considered to be stable over the lifetime of an agent. It covers the information required to enable the basic behavior of the agent and to ensure its fault-free interaction within the production system. The advanced model is seen as a changeable extension of the basic model. It can be updated during the agent lifetime by either the agent itself (learning and self-adaptation), by other agents (system adaptation), or by entities outside of the agent system. Also, for the change of the advanced model, the execution interface has to be used.

By exploiting this general architecture, three different types of combinations of mechatronical units/systems and agents can be considered: mechatronical units/systems without agents, mechatronical-oriented agents (i.e., mechatronical units/systems with agents), and agents without mechatronical units (SAAs—stand-alone agents). They can play different roles within the hierarchical production system architecture depicted in Figure 10.3.

Mechatronical units/systems without agents (BMUs—basic mechatronical units) can be exploited to implement subfunctions, functions, and/or main functions. They enable the design and implementation of static basic functions within a production system (equipment functions) as suggested in Lüder et al. (2010a). Such basic functions will constitute the controlled basic physical behavior of a production system, such as powering a drive with the right current to create and control a motion ramp, and the controlled basic measurements such as reading the current within an inductive sensor. Thus, the BMUs will provide the control of functions directly on top of the production system’s physics and be comparable to driver functions within a PC system.

Mechatronical units/systems with agents (AMUs—agendized mechatronical units) can be used to implement functions, main functions, cells, and sites. They provide capabilities to set up coordination control as suggested in Lüder et al. (2010a) and can enable extensive capabilities for flexibility, reactivity, and proactivity. They will exploit the basic functions of BMUs and AMUs to create and control/supervise its own complex production processes. Thus, they call the underlying functions of BMUs and AMUs over the device interface (the execution interface access path), coordinate them, and reach production processes of a wide-ranging complexity.

Agents without mechatronical units (SAAs) can be used to implement parts of control applications (information processing) of the higher layers of the architecture, shown in Figure 10.3, which usually cover control decisions of higher levels (MES, ERP, etc.) and need not be associated with a special physical hardware of the production system. Examples for such control functions are resource planning and allocation, monitoring, data collection and documentation, and many more. Usually SAAs will interact with AMUs in both directions (i.e., they will access AMUs and will be accessed by AMUs).

First applications of mechatronic-oriented agent systems can be found in Leitao et al. (2012).

10.3.2 The Engineering Process of Mechatronic-Oriented Agent Systems

The combination of agent systems and mechatronical units/systems will change the engineering process of production systems and its internal control systems. On the one hand, the use of mechatronical structures itself will change production system design, as shown in Wagner et al. (2010). Here, it is envisioned that the discipline-crossing design of function providing system components is essential. Thus, just as agents within mechatronic-oriented agent systems will control these functions, they are an integral part of the engineering process, and one main part of the engineering process is to define the correct modular/hierarchical structure of the production system. This can be depicted by considering the necessary changes of the DACS methodology to be used for a mechatronic-oriented agent system design.

Within the first step of the DACS methodology, the set of necessary control decisions will be modeled within the decision model. This action is based on the definition of the level of the control pyramid to be controlled.

Each production system consists of a set of production resources implementing complex production functions and covering a subset of the set of all relevant control decisions. Each mechatronical unit/system, which is controlled by an agent (AMU) and which is involved in the library of mechatronical units/systems, has its internal set of control application parts realizing the control decisions required for a proper behavior and function provision of the mechatronical units/system. Hence, there is bidirectional mapping between a part of the necessary control decisions and the set of mechatronical units/systems required within the production system. By identifying the necessary mechatronical units, at first also the related part of the necessary control decisions is identified. For them, it has to be decided as to whether they will be implemented by BMUs or AMUs. In addition, all necessary control decisions have to be identified and are required for the proper coordination of the functions of the identified mechatronical units. They can be derived from the sequences of the necessary production processes, the necessary safety and security activities within the production system, and the necessary higher-level control decisions for planning, supervision, etc.

Within the second step of the DACS method, the necessary control decision of the decision model will be associated with agents. For the AMUs, this mapping is done. Hence, the control decisions not emerging from AMUs have to be distributed to AMUs and SAAs. Hands on for this distribution can be the relevance for mechatronical units, the emerging necessary communication load to provide all necessary information for the control decision, the possibility of mobility of the control decision position, and the accessibility in case of implementation in an AMU (see Lüder et al., 2004b). If a control decision is mapped to an AMU, it should be implemented as an extension of the environment model.

In the last step, the interaction protocols between agents have to be defined. Here, the execution interfaces and device interface structure of agent-oriented mechatronical units/systems have to be a guideline and a bordering condition.

The resulting use of mechatronical units/systems and agent-oriented mechatronical units/systems within the DACS method is depicted in Figure 10.6.

One of the main advantages of the extended DACS method is the capability to be embedded within the engineering process of production systems. Following (Ferrarini and Lüder (2011) and Hundt and Lüder (2012), the identification of the necessary mechatronical units will be within the phases of the plant planning, the mechanical engineering, and the electrical engineering (both as part of the functional engineering). Thus, the set of necessary control decisions can be collected within the engineering steps of the plant planning, the mechanical engineering, and the electrical engineering, as well as the control system implementation.

10.3.3 The Advantages of Mechatronic-Oriented Agent Systems

Mechatronic-oriented agent systems can extend and improve the reuse capabilities of engineering results of the design and implementation of control systems of production systems. For production resources of the different layers of the mechatronical hierarchy of the production systems, mechatronical units/systems can be developed and controlled by a mechatronic-oriented agent. The layers cell, main group, and function group seem to be especially relevant. If these resources have been engineered and stored in a library, they can be exploited in an engineering process for production systems combining the DACS methodology with the PABADIS’PROMISE engineering process as white boxes or black boxes. In this way, they can significantly improve the efficiency of the engineering process itself, as well as improve the engineering result quality by fault prevention.

Another advantage of mechatronic-oriented agent systems controlling AMUs/systems is its capability to implement the plug-and-participate behavior of production system resources (Heinze et al., 2008). Thus, the interoperability of production resources at the control level can be ensured.

With respect to the agent capabilities reactivity, proactivity, and mobility, mechatronic-oriented agents have no limitations in general. However, it can be expected that mechatronic-oriented agents for special purposes will exploit some of these capabilities. For production resources, providing its production functions in the production system like a service, it can be expected that the controlling mechatronic-oriented agent will be reactive. Examples are machining centers. However, resources can also be imaginable, especially those showing proactive behavior. An example is AGVs exclusively transporting the materials associated with an order and being responsible for the supervision of the order execution. Such proactive and mobile mechatronic-oriented agents can be found in the semiconductor industry.

The ideas of structuring and implementing production systems by use of AMUs/systems and mechatronic-oriented agents also has a reverse impact on the design process of production systems. One of the current challenges within this process is the identification of appropriate mechatronical units, and thus the appropriate slicing of production systems into resources. The inherent production function and control decision orientation of AMUs/systems can support the identification of the best reusable AMUs/systems by processes such as the domain engineering described in Maga et al. (2010).