9.1 Introduction

Modern production systems are characterized by both increasingly complex structures and behaviors. They need to be flexible or even self-adaptable to address varying conditions of the systems’ applications (Kuehnle, 2007). The required flexibility may, for example, be necessary for the product portfolio to be manufactured, the amount of products to be produced, the resources being applied, or the production technologies involved. This mainly depends on the application case and the scope of interest concerned. In particular, optimization of the production, with regard to efficiency, should be allowed by the systems (Wünsch et al., 2010; Bader et al., 2014). Therefore, the production systems’ architectures, their control as well as the engineering process of such systems, are mostly affected by these challenges. As a means to tackle these challenges, modularization and distribution of control are considered (ElMaraghy, 2009; Lüder and Foehr, 2013). Here, decentralized control and supervision of flexible decentralized manufacturing resources are envisioned.

Agents and agent orientation has been established increasingly as novel architectures and behavior paradigms over the last several years. Initially emerging from the area of artificial intelligence (Weiss, 1999), the application of these architectures within various fields of production systems control has recently been introduced (Shen et al., 2006; Marik et al., 2011; Wagner, 2003). The agent-oriented approach thus became an adequate and efficient means to cope with complex problems concerning the engineering and application of control systems for production systems, especially in cases of decentralized control. One crucial point in applying these architectures is giving real physical entities of the production system a central role. Moreover, the purposes and aims of the production and thus the necessary behavior are considered explicitly. The agent-oriented approach therefore especially enables the solution of challenges with complex and highly volatile constraints and the issue of self-adaptability. Various examples of applying the agent-oriented approach are available—e.g., production planning systems (PPSs), logistic systems, and maintenance systems (Göhner, 2013). Designing decentralized control systems based on the agent-oriented approach allow control decisions to be taken locally based on local information. Thus, the overall complexity of decentralized control in highly flexible manufacturing systems can be tackled. Various surveys on applying agent orientation to production automation exist (Shen et al., 2006; Leitão, 2009; Metzger and Polakow, 2011). Instead of providing another state-of-the art survey, an in-depth analysis of selected applications of agent orientation is provided to finally derive open architectural challenges.

The remainder is structured as follows. First, terms and definitions used in the remainder of this chapter are presented. Subsequently, a procedure for defining agents and for engineering agent-based systems is described. Three application cases within different domains were selected in VDI/VDE (2012b) as a best practice for the use of agent orientation in production control systems. Therefore, these application cases are described in detail, their relations to the presented procedure are highlighted, and experiences during their implementation are highlighted. Based on these experiences, architectural challenges that still remain open are discussed before the chapter concludes.

9.2 Terms and Definitions

Within this section, basic terms and definitions for the application of agent orientation in production systems engineering, control, and application are given.

Software architecture in general is a set of structures that include software elements, the relations between them, and the properties of both elements and relations (Clements et al., 2010). A production system’s control system is nowadays mostly realized by software implementing necessary automation functions on different levels, from field-level control to operations management. A software architecture of a production system’s control system is, according to the definition, a set of software elements, their relations, and properties needed to realize required automation functions.

Agents are regarded as delimitable (hardware and software) units. Their purposes and aims are defined and each agent intends to reach its aims by behaving both independently and cooperatively. Therefore, agents interact with their environment. The agent’s environment can either be an arbitrary system (1-agent architecture) or a system containing further agents (agent system architecture) (VDI/VDE, 2012a).

As a consequence, an agent system contains a set of agents that interact to realize one common or several (possibly contradictory) aims, and thus follow (mostly shared) optimization criteria.

Agents and agent systems are considered to be modeling principles, which are applied for problem understanding, solution strategies, and structures. Therefore, they do not depend on a specific technical realization or implementation. Nevertheless, some realization technologies have been developed lately and are now established in practice. First, there are software agent systems (Bellifemine et al., 2007; Theiss et al., 2009) providing runtime environments, execution systems, and platforms for agents that can serve as a basis for implementing agent systems. These software agent systems are primarily applied in the field of information sciences. Second, hardware agents refer to a combination of hardware and software. These agents are applied, for example, to automatically guided vehicles (Vis, 2006).

The behavior of the production systems’ agents is based on dedicated models of the environment. These models need to be implemented within the agent so it can perceive the environment’s behavior and deduce reactions in order to realize its aims within its freedom of action. Thus, agents require means to interact with the environment. Depending on the implementation technology, these means have either a direct or indirect physical impact on the environment.

The impact on the environment through an agent’s capabilities is limited by the agent’s freedom of action. Consequently, both an agent’s freedom of action and its technical implementation define its degree of flexibility.

Autonomy is another important feature of an agent. Agents are supposed to be independent and have exclusive control over their internal states and therefore their behavior. Thus, an agent will solely make decisions concerning its actions and operations on the basis of its local knowledge and its environment model.

When implementing an agent’s behavior in the field of agent-based control architectures, the so-called believe-desire-intention architecture can be used (Weiss, 1999). In this case, the environment of an agent has a certain state, and an agent associates this state with a defined meaning to be reacted on with a defined activity. This is called an agent reactivity property, which is the most implemented kind of agent used in the automation domain. By implementing reactivity, agents controlling production system resources—so-called resource agents—can possibly react to environmental signals by executing dedicated behavior as necessary for a certain production step.

Similarly, agents are able to follow their own aims directly and actively. Hence, they can act in a special, purposeful way based on their own activities. This is called agent proactivity. Proactivity enables the implementation of agents controlling orders or tasks originating from outside the production system (order agents). Order agents actively ensure the execution of orders and tasks, and therefore access and use the behavior of other agents to do so. Examples of such agents are manufacturing order agents or maintenance agents (Ferrarini and Lüder, 2011).

In contrast to agent architectures, a variety of other architectural paradigms exists. Cyber-physical systems (CPSs) are an emerging topic and combine the vision of an intelligent, adaptive control system with seamless vertical and horizontal information integration. According to Rajkumar et al. (2010), Lee (2008), and acatech (2011), a CPS is an embedded system that is integrated within physical processes. CPSs are characterized by their capability to monitor and control physical processes, are networked for communicating with each other, and utilize information sources that might be distributed worldwide. The analogy between the definition of agents and an agent architecture, which was presented previously, and CPSs is obvious: For realizing CPSs for production systems, an agent-based control architecture, according to VDI/VDE (2012a), can be applied.

A detailed consideration of agent-related terms and definitions and the application of agent-oriented approaches within the area of production system control is described in VDI/VDE (2010).

9.3 Generic Engineering Processes

Multiple approaches to engineer agent-based control systems have been suggested recently. They cover various application cases and implementation technologies (Van Brussel et al., 1998; Lüder et al., 2004a,b; Colombo et al., 2004; Leitão and Rodrigues, 2011; Ferrarini et al., 2006; Wooldridge et al., 2000; Lüder and Peschke, 2007). Regarding the simplicity of application and the need for independence from a specific implementation technology, a combination of the designing agent-based control systems (DACS) method of Bussmann et al. (2004) with methods of model-based engineering seems to be the most promising.

VDI/VDE (2012a) contains a collection of agent systems’ design approaches—for example, ROADMAP (Juan et al., 2002), SODA (Omicini, 2001), AgentUML (Bauer et al., 2001), MaSE (DeLoach and Wood, 2000), PASSI (Cossentino and Potts, 2002), and others. All these approaches, as well as much newer and more advanced approaches such as Cossentino et al. (2010) and DeLoach and Garcia-Ojeda (2014), provide a set of models, as well as engineering steps, to be executed to reach a running agent system.

Even if the DACS method is now more than 10 years old, it remains the most abstract and, related to the engineering of manufacturing system control, adequate methodology. On the one hand, the engineering of distributed control systems requires the identification of necessary control decisions and their association to agents executing them. On the other hand, this engineering process is usually executed by engineers with a limited background in information sciences. The DACS methodology enables the consideration of necessary control decisions independently from implementation issues, resulting in an optimized decision structure. Based on this structure, the final implementation can be done easily. Thus, the DACS methodology will follow the usual functional thinking of control engineers in manufacturing and is easily combinable with manufacturing system engineering. In addition, it requires only a limited IT background, and thus does not tend to confuse control engineers by using formalisms they are not familiar with—as might be the case in approaches such as those by Cossentino et al. (2010) and DeLoach and Garcia-Ojeda (2014) that focus on efficient software engineering.

The first step of the DACS method is to analyze and describe the set of necessary control decisions for the intended control system executing the manufacturing system. The relevant layers of the control pyramid, which will be controlled by the agent system, need to be defined. Subsequently, it will be determined which control decisions are mandatory, so that the production system will function correctly on the scope of interest. Furthermore, dependencies among control decisions, which lead to necessary interactions between control decisions, are finally identified. The main focus should be on the amount of exchanged information. This step results in the creation of a decision model.

In the method’s next step, the agent set is specified. Thus, control decisions of the decision model are grouped and mapped to individual agents responsible for the corresponding decision group. Hence, the dependencies between control decisions within the decision model will lead to connections among agents calling for interactions between agents. To ensure that the system behaves correctly, it is essential to consider the necessary agent properties within the definition process of control decision sets and its related agents, as well as the definition of agents in interactions. Therefore, it is relevant to distinguish reactive agents, which provide functionalities to be accessed and used by other agents, from proactive agents, which use functionalities of other agents to fulfill their plans. The second step leads to an agent model describing all agents, along with their behavior and interactions.

The final step of the DACS method (see Figure 9.1) contains the precise design of the necessary agent interaction mechanisms. Thus, requirements on interactions are collected and checked against capabilities of interaction protocols within a protocol library. The most suitable protocol is chosen for each interaction and (if necessary) adapted. This final step leads to an extension of the agent model by interaction protocols forming the interaction model.

The DACS method itself has been designed without targeting special engineering tool chains or modeling means. Within practical applications, tools supporting the Unified Modeling Language (UML) are very often exploited to create a decision model, an agent model, and an interaction model. Finally, special domain-dependent languages have to be exploited for the implementation of the manufacturing system control. Thus, there is no preferable tool chain advisable.

The decision model, agent model, and interaction model form the basis for the implementation of the agent-based control system, using an adequate implementation technology. Consequently, the relevant implementation technology must be evaluated and selected in the first implementation step. Software agents or hardware and software agents are applied based on the requirements on hardware and software properties and, thus, possibly provide migration capabilities. In this context, migration capabilities enable agents to move between locations and/or execution context positions. This can lead to a physical, as well as a logical motion, between systems. This property is especially useful, if the quality of fulfillment of an agent’s aim or the quality of the overall system performance strongly depends on the execution position of control decisions. Support concerning technology selection is given in Wagner (2003) with regard to agent technology, and in Lüder and Foehr (2013) regarding the definition of dependencies between agent and production technology.

Based on the selection of implementation technologies, first the agents and afterward their interaction capabilities and corresponding protocols are realized. In these steps, both the software as well as the mechanical, electrical, and other implementation technologies have to be considered carefully. After the implementation is completed, system integration, testing, and commissioning will follow.

When developing software agents for a production system’s field-level automation control software, the technical characteristics of the particular production plant must be considered strongly. It is thus necessary to adapt the identification of software agents inside the development process in order to enable the consideration of these technical characteristics.

An applicable methodology focusing on field-level automation control software of production systems is described in Schütz et al. (2013). A model-based development and implementation of software agents on standard automation devices—i.e., programmable logic controllers (PLCs) conform to the standard IEC 61131 of the International Electrotechnical Commission—is proposed that is based on the meta model and diagrams of the Systems Modeling Language (SysML).

The proposed methodology consists of seven steps, in which different aspects of the software agents can be defined with the use of different diagrams of the SysML (Figure 9.2). These seven steps are a refinement of the DACS method’s second step. In steps 1-3, based on sensors and actuators of a production system, basic functions, function modules (Module), and process functions (Function) are identified (using the set of control decisions to be undertaken in the decision model) and assigned to corresponding software agents (Agent). According to the agent system architecture (see Figure 9.2, middle), these Functions are invoked by one or several Processes (step 4), which control a plant according to a production process that can be defined, for example, by modeling a SysML Activity Diagram. Again, the decision model is being used. A Process may contain the requirements of a production process and the links to Functions inside the SysML model of an agent system. In steps 5-7 of this refined DACS methodology, Agents and Modules are evaluated according to the requirements defined inside the Processes.

Code generators were developed, which enable the transformation of agent models into executable code according to the IEC 61131-3 standard for supporting an automated transformation between the agent systems modeled in SysML on PLCs (Frank et al., 2011).

9.4 Application Cases

Referring to different application domains and different layers of control, three application examples for agent-oriented control applications for production system control are introduced in this section. All of these systems have been engineered and built (at least partially) based on the approach described previously.

9.4.1 Agent-Based Production Planning

Within the area of PPSs in the manufacturing execution systems (MESs) domain, there exist three crucial problems: (i) to allocate the production orders to production resources following a set of optimization criteria, (ii) to control the execution flow of the production process of the orders, and (iii) to immediately handle occurring problems within the production system, such as resource faults or production failures. Orders vary from conventional production orders, resulting in the control of the production process to resource maintenance orders or data acquisition orders.

One possibility to control systems fulfilling these functionality requirements is to use agent-based control systems. Agents can act in an autonomous but cooperative manner. On the one hand, agents are able to encapsulate order- and product-related entities and, on the other hand, they can represent entities related to resources. By dividing the system into components, the system’s complexity is reduced to a manageable level and the application of negotiation-based control solutions becomes possible. For that matter, the different decision-making components of the control system can be designed, implemented, and tested independently because they are separated and decoupled from each other.

A solution for this challenge can be based on the implementation of an agent-based control system with three layers following the agent system design pattern described in Lüder et al. (2004a,b). It will contain product agents, product agent supervisors, resource agents, resource agent supervisors, ability brokers, and product data repositories. These agent types cover the different necessary control decisions and provide the respective required information sets. They can be derived following the DACS method described earlier by considering the sets of control decisions necessary for order allocation, manufacturing process control, and exception handling (see (i), (ii), and (iii) earlier). As a result, the agents introduced in the following, as well as their related control decisions and interactions, will emerge.

Product agents will manage and control the execution of a production order. Therefore, the agent will be initialized at a start time with all necessary order-related information. Moreover, it will autonomously collect necessary product-related information from product data repository agents, evaluate applicable resources by negotiating with ability broker agents, negotiate the time and conditions of the execution of production steps with resource agents, and finally, supervise the production process execution by resource agents.

Product agent supervisors control the life cycle of such product agents. It is their responsibility to create order agents in cooperation with the Enterprise Ressource Planning (ERP) system, to supervise their order execution progress, and to ensure (if necessary) changes in the order information on request. Agents for plant maintenance and similar activities can be defined analogously to product agents and product agent supervisors.

Resource agents are responsible for providing production process execution capabilities by encapsulating field-controlled manufacturing resources. They must register their production capabilities to at least one ability broker agent. They will cooperate with the product agents, so that they can plan production process schedules and exploit their underlying field control systems of the production resources to control the production process on behalf of the product agents.

The resource supervisor agents make sure that upper layer control systems, such as ERP and Supervisory Conrol And Data Aquisition (SCADA) systems, have access to the resource agents. Additionally, they ensure the start and stop sequences of resource agents and the integration of field control applications in the resource agents.

The ability broker agents enable the retrieval of production resource information by product agents. Therefore, they will compare the requested resource capabilities (product agents) with the provided resource capabilities (resource agents) and return information about all resource agents matching the requested capabilities to the product agents.

Finally, the product data repository agents are responsible for storing and distributing product information such as production process characteristics and control application parts on demand.

A combination of all these agents will establish a decentralized control process using agent-based interaction mechanisms implementing negotiation solutions, as depicted in Figure 9.3. A more detailed description of the agent system, its inherent flexibility properties, and its example implementation are given in Ferrarini and Lüder (2011), and an overview is covered in Lüder et al. (2008). Among others, the main benefits of using the described architecture are:

i. The simplification of the reconfiguration of assembly, production, and transport systems throughout their life cycle by enabling the integration of new equipment or the replacement of existing equipment in a plug-and-participate way and enabling an easy product integration.

ii. The facilitation of a dynamic reconfiguration of control and manufacturing equipment during production, allowing order-oriented control application design.

iii. The provision of a higher value of built-in equipment flexibility limiting the application of resources only by resource ability physics.

iv. The alteration of customer demands (product parameters) until their ultimate possible point.

All these benefits have been demonstrated in the PABADIS’PROMISE approach (Ferrarini and Lüder, 2011). Implementation-related experiences can be found in the project result documentation publically available online.¹

The described distribution of control decisions between resource agents and order agents (considering all other agents as part of the necessary infrastructure to make the system entirely operate and interact with the environment) provides a basic pattern applicable in many cases in industrial control. Applying the idea of CPS, this distribution leads to two different types of cyber-physical production system entities: (i) order entities exploiting system capabilities and controlling the execution of its orders, and (ii) resource entities providing the system’s useable capabilities. Following the agent system design pattern described in Lüder et al. (2004a,b), there are different further approaches available, resulting in a similar structure for agent-based control of PPSs within the MESs domain. Several are described in Shen et al. (2006) and Leitão (2009). The most recent work is given in Mařík et al. (2013).

9.4.2 Agent-Based Machine Configuration

Increased competition in the mass production of the automotive industry and between its suppliers made the use of extremely productive transfer centers even more appealing. Contrary to machining centers, transfer centers have an inverse kinematic structure. Due to the integration and exchangeability of one or several components, transfer centers are reconfigurable. Therefore, these machines are highly flexible, but this flexibility in turn increases the configuration’s complexity because many constraints must be met.

An agent-based planning tool was developed in order to ease the configuration of these machines for human production planners (Bader et al., 2014). The system was developed by using the DACS method described in Section 9.3. In addition to the described method, the physical design of a transfer center was considered in the design of the multi-agent system. The emerged concept of the multi-agent system is entity-based, which means that agents represent different (physical) entities. For this application, the work piece to be produced is represented by the so-called work piece agent. This agent will provide all tasks that are directly related to a work piece and contain information about the work piece—e.g., choosing the corresponding production steps or analyzing the work piece features. A production step agent is instantiated for the representation of each production step. These agents choose a tool to execute the related operation and determine adequate cutting parameters. Afterward, the production step agents communicate with each other in order to find similar production steps that can be parallelized by a multiple spindle head. If a group of production steps is created, an adequate multiple spindle head is chosen and a corresponding spindle agent is instantiated. The spindle agent represents the multiple spindle head and exchanges data with the transfer center agents in order to get a mounting position in a transfer center. These agents represent and manage a transfer center. A production line agent represents the whole production line—e.g., to coordinate the utilization of the transfer centers’ capacity. Figure 9.4 shows an overview of the agents involved and their interactions.

A prototype of the multi-agent planning tool was realized using the Java Agent Development Framework (JADE). This prototype reads features of the work piece and necessary production steps from a file and is then able to configure the transfer centers of a production line. Up to now, the prototype was unable to determine the production steps automatically. For the production of an exemplary work piece with medium complexity, approximately 50 production steps are necessary. With the current version of the prototype, the system consists—for such a work piece—of approximately 65 agents, and the configuration of the transfer centers takes about 10 minutes. Without this system, the configuration of a transfer center would take up to one month, because many technical boundaries and decisions have to be considered. This agent-based system is faster in its performance because many of the decisions are made by the autonomous agents.

For this system, the design of the decision model and the interaction model, described as step 1 and 3 of the DACS method in Section 9.3, need increased attention. This is because of the fact that the agents have to make many decisions that rely on decisions made by other agents. This can easily lead to a situation in which several agents mutually wait for a decision.

Similar to the described approach, Blecker et al. (2004) introduced an agent-based concept for the process configuration for mass customization. Therein, agents are employed to represent platforms, modules, production constraints, etc. In Malz et al. (2012), a system to support the test planning in software tests is presented. In this reference, the agents represent, for example, test cases and determine their importance. With the test importance and some other constraints, an ordering of the test cases is possible, in order to find as many bugs as possible in a given time span. In Badr and Göhner (2009), the agent technology is used for the execution planning in a flexible production system. In Urbas et al. (2011), the technological characteristics of multi-agent systems and middleware for usage in automated systems are discussed. Pech and Göhner (2010) describe an approach to combine qualitative models with a multi-agent system to improve the usability of multi-agent systems for the control of complex industrial automation processes. In Mubarak and Göhner (2010), an agent-based architecture for self-management in the domain of industrial automation systems is introduced. This architecture is explained by means of a real-world example of a passenger lift system, which is extended with self-healing functionality.

9.4.3 Agent-Based Field-Level Automation Control Software

There exist stringent requirements for production systems regarding their dependability and tolerance against defects and failures of automation devices—i.e., sensors and actuators. Moreover, there are technological or financial constraints that limit the possibilities of shutting down or pausing a production process (e.g., for unplanned maintenance). Therefore, concepts are required to enable continuous operation of a production system according to the maintenance schedule, at least until a safe state for shutting down the process has been reached, even if modules are defective or have had a breakdown.

These goals can be realized by the implementation of software agents that contain and use data on the technical structure and physical dependencies of a production system. This knowledge may include information regarding other sources of information (e.g., redundant sensors) and alternative operational parameters for production processes (e.g., alternative sequences of manufacturing operations).

Various research works focus on the IEC 61499 (IEC, 2012) as an enabler for intelligent, distributed field-level automation software (Vyatkin, 2011). In Lepuschitz et al. (2011), an approach for reconfiguring an inner logistic system’s control is proposed, which relies on a model of the system’s states, capabilities, and structure (Vallée et al., 2011) for reacting to contradictory material identification information. For compensating machine breakdowns, Khalgui et al. (2011) propose an approach for reconfiguring field control software based on a model describing the behavior of available control functionality (Khalgui and Hanisch, 2011). An agent platform for real-time operation is proposed in Theiss et al. (2009). However, it is not applicable to standard field control hardware following established standards. In process industry, several agent-based applications have been developed for controlling critical processes (Metzger and Polakow, 2011). Nevertheless, detailed knowledge about used sensors and actuators (Romanenko et al., 2007) is required for applying agent technologies to fault-tolerant automation. In a nutshell, approaches to agent-based systems for field-level software exist and are increasingly developed. However, none of them is applicable for IEC 61131 control software, which is the most widespread standard for PLC-based field-level software (ARC Advisory Group, 2013).

A concept to realize software agents on IEC 61131 platforms that are able to detect and compensate sensor failures based on knowledge regarding various sources of information is described in Schütz et al. (2013) and Wannagat (2010). This concept was evaluated using different application examples. Among them was a plant from the wood and paper industry that compresses and heats a mixture of wood fibers and glue in order to produce a continuous strand of particle board. The wood fiber and glue mixture is pressed by a total of 75 separate pressing frames, each containing several hydraulic cylinders. Hence, the pressing frames are equipped with hydraulic valves, pressure sensors, and distance sensors so that a gradual compression of the material is ensured, until it has the desired thickness.

To operate the press, an agent system was realized that assigns one agent to each pressing frame, which form the modules according to the SysML-based engineering process presented in Section 9.3. The physical dependencies between the sensors within a pressing frame are part of the agents’ knowledge. There are distance sensors that measure the material’s thickness before and after a pressing frame, respectively, and pressure sensors, which measure the pressure inside the hydraulic cylinders, and thus the force that is applied to the material. This knowledge model is developed within step 4 of the SysML-based engineering process defining the process and its restrictions.

The implemented software agents are able to calculate soft sensors based on these physical dependencies, which can be used as an alternative source of information. Evaluation experiments using a simulation of the continuous hydraulic press showed that the implemented software agents were able to both detect and compensate failures of the pressure and distance sensors. Thus, the operation of the hydraulic press could be continued based on the implemented soft sensors (Schütz et al., 2013). Application examples from manufacturing automation have been used to conduct similar evaluation experiments, proving the applicability of the concept for production systems with discrete behavior. The achieved improvement of a manufacturing system’s availability in case of sensor failures by applying this agent-based architecture depends on various parameters (e.g., the number of soft sensors and the mean time to repair of respective sensors). It was shown that in any case the availability can be increased in the presence of failures if soft sensors are available (Schütz et al., 2013). The number of existing soft sensors depends on both the physical sensors installed within a production system and the experience of the expert describing physical dependencies between them. The number of soft sensors to be considered by the agent directly influences the calculation effort of the PLC. Accordingly, an adequate balance between the number of soft sensors (i.e., physical redundancies) and the required cycle time for controlling the system is required (see Schütz et al., 2013, for a detailed discussion).

It is hard to consider all possible failures of a manufacturing system’s actuators a priori. Therefore, an agent-based control system was developed that can dynamically reconfigure the control software (Legat and Vogel-Heuser, 2014). It completely applies the concept of knowledge-based production systems (Legat et al., 2013a) by assigning to each role (i.e., the function according to the extended DACS methodology) a corresponding knowledge model encapsulating all data required for making decisions (i.e., for implementing the respective function). As depicted in Figure 9.5, an agent within this agent system architecture realizes at least one of the defined roles. The knowledge models of these roles are completely based on various strategies—i.e., compositions of atomic real-time functions (real-time services). Execution agents fulfill the task of executing and monitoring the real-time field control. If they detect an unexpected behavior of such a real-time service (e.g., caused by an actuator failure) during its execution, the corresponding supervision agent is notified. Supervision agents stabilize the production process by executing an adequate stabilization strategy, which transfers the manufacturing system into a state in which it remains stable until being reconfigured. Both execution agents and supervision agents have to perform their tasks in real time and are therefore realized on PLC runtime environments compliant with the IEC 61131 standard. When the system is stabilized, the supervision agent requests the reconfiguration agent to compute a reconfiguration. The reconfiguration agent is not realized inside the real-time kernel because the computation of required knowledge models—i.e., the strategy to restart the system—operating the manufacturing system without utilizing disturbed functions or components and corresponding stabilization strategies, is not real-time capable. As described in Legat et al. (2013b), the time to compute a new operation strategy in case of failure depends on the size and flexibility (i.e., the number of functions and possible sequences) of the manufacturing system. Within evaluated manufacturing systems, the time for computing a reconfiguration was in the range of 5-10 milliseconds.

In order to engineer the necessary knowledge for this agent system architecture, a specific UML profile was developed that lets it describe the state space of a manufacturing system in a consolidated way (Schütz et al., 2012). In Legat et al. (2013b), a description is provided on how to use it to automatically determine all possible strategies and identify the optimal ones. By that, the knowledge models used by this agent system’s roles are automatically generated.

An evaluation of the approach has been successfully conducted on two different manufacturing demonstrators, both equipped with standard industrial automation hardware. Based on an analytical model, the overall benefit of this approach was quantified mathematically and was proven—in which case, the approach is more suitable than existing approaches (Legat and Vogel-Heuser, 2014).

9.5 Architectural Challenges

It has been shown throughout this chapter that agent technology can be successfully applied to production systems in order to increase the flexibility of control systems. Specific benefits of selected approaches, which were presented in this chapter, have been described in detail. Beyond the existence of the first promising applications of agent technology in industrial practice, a broad application of agent technology is still missing. The most relevant reasons for this fact are the conservatism of control engineers, as well as the limited determinism of agent technology at design time (Lüder et al., 2013). Furthermore, questionable investment costs for teaching a novel paradigm to practitioners in order to achieve potential flexibility advantages hinders the adaptation of novel concepts in industrial practice (Schild and Bussmann, 2007; Leitão, 2008). Besides these challenges, there are still open issues to be investigated. Depending on specific, given flexibility requirements on a production system’s control system, different manifestations of agent technology exist (cp. Section 9.4). Each of the application examples follow (at least partially) the design methodology presented in Section 9.3. Nevertheless, the agent architectures differ significantly and their commonalities are not visible without a detailed study. Furthermore, a variety of different additional architectural manifestations of agent technology exist—e.g., holonic paradigm (McFarlane and Bussmann, 2003) or bio-inspiration (Leitão et al., 2012). As to which of these manifestations is best suited for the given flexibility requirements is not fully understood yet. Furthermore, the variety of other (competing) flexible software architectures and paradigms (e.g., service-orientation or CPSs) exist (cp. Section 9.2) or are already under investigation. Some first approaches are also available, which combine different architectural paradigms in order to unite their benefits—e.g., agent technology and service-orientation (Mendes et al., 2010).

In a nutshell, the specific benefits of an architectural paradigm or combinations of paradigms, as well as key requirements, which lead to a non-ambiguous decision on agent technology or alternative approaches, has not been conducted until now. In the early phases of an engineering project, not all nonfunctional requirements on a control system, such as flexibility requirements, might be known exactly. For this reason, keeping open the option to decide for a specific architectural paradigm or implementation as long as possible will enable users to make the correct architectural choice without too many presumptions.

For providing a first step toward this vision, an approach is required, which enables developing a production system’s control software independently of a specific architecture or realization but supports the definition of intelligent software building blocks, which are structural entities, and their allocated functionalities (cp. Figure 9.6). The three architectures described in the previous section can be applied here as a design pattern. Both structural entities and functionalities refer to different kinds of knowledge. An environment model enables the reasoning of intelligent, structural entities—e.g., about their actions and consequences. A description of functionality is required to facilitate its application by structural entities—e.g., information about its boundary conditions, behavior, etc. Furthermore (nonfunctional) requirements on the structural entities, functionality, design patterns, and overall control software have to be defined, such as, e.g., extensibility with respect to additional entities, the flexibility of redeployment during operation, etc.

Designing a control system based on such an abstract, analytic model will support the development of control software independently of a specific architectural paradigm. In the context of an agent system, for example, a structural entity might be implemented as an agent, whereas in the context of a CPS, it is realized as a cyber-physical module. In the end, the specific realizations of intelligent software building blocks depend on given flexibility requirements and system constraints given in the abstract model. By that, these flexibility requirements and system constraints, as well as a specific implementation technology’s capabilities, can be juxtaposed in order to identify an adequate realization. Because the modeling approach is completely independent of a specific implementation technology, modeling a production system’s control software based on this analytic model will provide the possibility to reuse a designed control software solution across different implementation technologies, even ones not existing at the design time of the abstract model. Consequently, no commitment to a specific architecture or implementation technology is required in (early) design phases. In the end, this approach will leverage the acceptance and adaptation of novel architectural paradigms, including agent technology by industry.

9.6 Conclusions and Overview

This chapter provided a summary of current best practices with respect to the application of agent orientation within the design and implementation of agent-oriented control systems for production system control. The current definitions and the use of agent-based systems for manufacturing system control have been discussed. Additionally, it was shown how agent-oriented systems can be engineered. The described definitions and processes have been validated on the basis of three examples. Furthermore, the needs and requirements for a novel approach on intelligent software control design have been discussed. A vision of an appropriate analytic model has been proposed, which subsumes existing and upcoming architectural paradigms and enables control system design independently from a decision on a specific architectural realization.

As there are different approaches to research on the use of agent-oriented control in various application areas, there are additional best practices exceeding the scope of this publication. Further applications providing best practices for multiple application domains are given in VDI/VDE (2012b).

An important and still open problem with respect to control systems is the development of implementation technologies fulfilling all requirements of field control systems. Beyond some initial promising research and development activities, such as discussed in Theiss et al. (2009) and Stroppa et al. (2012), the required real-time capabilities of agent technology, which limit the applicability of agent-based systems, are still an open problem. Nevertheless, beyond agent technology as a software technology, agent orientation as a modeling and engineering paradigm is entirely applicable now. This is especially valid for use cases dealing with limited hard real-time conditions, as given in cases of manufacturing execution control, maintenance, data allocation, and documentation. In particular, the combination of an agent-based control architecture and the ideas of CPSs seems promising. Consequently, it can be envisioned to reach agent-based highly flexible systems in which agent orientation will enable plug-and-participate resources and orders, and also special production functions, special diagnosis functions, or special planning functions on demand. Therefore, further research and development is needed for identifying relevant sets of control decisions and their structuring toward agent-based CPS entities.