9.1 Introduction

DevOps is a set of collaborative and multidisciplinary efforts with an organization that has emerged in software development organizations. Although the concept of DevOps has been introduced nearly a decade before, various researchers have defined DevOps in different ways [1]. According to Díaz et al. [2], DevOps is defined as “a collaborative and multidisciplinary effort within an organization to automate continuous delivery of new software versions, while guaranteeing their correctness and reliability.” And as defined by Lwakatare et al. [3], DevOps is “a portmanteau of ‘development’ and ‘operations’ and is an approach where software developers’ team and operation team work in a close collaborative environment.” DevOps was introduced initially to resolve the 10 conflicts between development and operations teams when they need to provide a quick response to customer requirements [2,3].

Nowadays, the software organizations are following DevOps practices to efficiently and effectively develop software product in order to develop a quality product to satisfy their clients. According to the State of DevOps Report [4] published by Puppet, the deployment rate in the DevOps environment is 30 times faster than the enterprises that have not adopted DevOps in their software development cycle. Moreover, CA Technologies conducted their study over the adoption of DevOps practices and they have predicted that 88 percent of 1892 software organizations will be using DevOps practices in the next five years [4, 5]. The goal in creating effective communication and integration between DevOps teams is to obtain the benefits of modern software development approaches that increase software development performance, process productivity, rapid deployment of new high-quality software features, and team effectiveness; resulting in faster delivery to market, better software quality, and better alignment of developers and operations with business goals and customer requirements. [6, 7, 19, 21-25]. Although DevOps offers multiple advantages and is being continuously adopted by the software development organizations, it is not a straightforward process to implement in software organizations because of a number of challenges faced by the development teams. This could be because of the limited attention given by the researchers to the development of a readiness model for effectively implementing in software organizations.

Models and framework have been developed by various organizations in order to effectively bridge the gap between development and operational teams for continuous integration, and deployment. The Unicorn framework is used for service improvement strategy, and IBM introduced the DORA platform for DevOps performance analysis, which is used for the assessment of software product delivery value stream; and an ontology-based DevOps maturity model is used to perform DevOps tasks [8, 7, 18]. However, even though these frameworks and models were developed to implement DevOps, few studies discuss how to implement them in software development activities. Therefore, the continuously increasing trend of using DevOps practices in software organizations motivated us to initiate the development of the DevOps Implementation and Management Model (DIMM), which can help software organizations measure and improve their practices for implementing DevOps. In this research work, an initial step is taken towards the development of DIMM by investigating the challenges which could have a negative impact on implementing DevOps practices. Moreover, understanding the challenges can also assist the software organizations in addressing the key areas which the organizations need to focus on to effectively implement DevOps before using the DevOps practices [9, 10, 12]. Moreover, the identified challenges also need to be classified in the context of the development team and operations team (Dev-Ops). The ultimate goal of this classification is to highlight the significance of each challenge for both the development and operation teams. Based on the above discussion, the following research questions have been designed:

• RQ1: What are the challenges identified in the literature to implementing DevOps practices in software development organizations?
• RQ2: What are the most significant challenges that need to be focused on to implement DevOps practices in the software development?
• RQ3: What are the challenges as identified in the literature related to the development team and operation team of DevOps?

This chapter is presented as follows: Section 9.2 discusses the SLR process, which is followed by the findings of the study discussed in Section 9.3. In Section 9.4, the discussions and summary about the findings are presented. The possible limitations of the study are discussed in Section 9.5. Finally, the conclusions and future directions of the study are discussed in Section 9.6.

9.2 Research Methodology

In this study, the systematic literature review (SLR) approach was selected in order to investigate the challenges for implementing DevOps in software organizations. An SLR is a type of secondary study that reviews all primary studies by analyzing and exploring all the evidences related to questions of interest [13, 15]. We have opted for the SLR process given by Kitchenham [13] in which SLR is implemented in three main stages:

• Stage-1: Planning the review,
• Stage-2: Conducting the review, and
• Stage-3: Reporting the review as briefly discussed in the following subsections. The SLR process has been used by various other researchers in a variety of domains [12, 14, 15, 20].

9.2.1 Stage-1: Planning the Review

This is the first phase of SLR in which a complete protocol is discussed, which includes developing the research questions, search process of primary studies, inclusion and exclusion criteria for the selection of primary studies, quality evaluation of primary studies, and data extraction from the finally selected articles [14].

• Research Questions (RQs): All the developed research questions are discussed in Section 9.1.
• Searching Process: A complete search process used to find the most relevant primary studies was conducted to address the mentioned research questions. The relevant keywords and their alternatives are extracted from the available literature. The major search strings have been developed using Boolean OR and AND operators. A complete detail of search strings for the selection of primary studies from the selected digital repositories are discussed in Table 9.1. The selection of digital libraries is based on the prior studies and suggestions provided by other researchers [14, 15].

• Inclusion and Exclusion Criteria: In order to include and exclude the manuscripts, the following inclusion and exclusion criteria, as given in Table 9.2, were considered.

• Quality Check of Selected Papers: The quality assessment of selected articles was evaluated using the quality criteria defined by Kitchenham et al. [13]. Five questions were used to assess how well the selected papers define the quality of the SLR, as shown in Table 9.3. The following points system was considered to determine the individual criteria scores: Yes (Y) = 1 point, Partial (P) = 0.5 point, No (N) = 0 point. After summing up the four individual criteria scores, a total quality score for each selected paper was calculated. Thus, the total quality score for each selected paper ranged between 0 (very poor) and 4 (very good). A paper with a quality score of more than 3 was considered a good quality paper and considered in our SLR study [1].

9.2.2 Stage-2: Conducting the Review

• Selecting the Primary Studies: The whole process of selecting relevant articles was performed in four sequential stages [14]. Initially, 1520 articles were extracted from the selected digital databases after using the developed search string (2.1.2) and the inclusion and exclusion criteria as described in Table 9.2. After reading the title and abstract of the papers, 304 papers were selected in Stage-2. In Stage-3, 96 related papers were extracted after reading the introduction and conclusion sections of the papers, which was followed by the final Stage-4 from the selected list of articles in Stage-3, based on reading the whole text of the papers. After the four scanning phases, the total 24 relevant papers were retrieved as primary studies to address the research questions, as discussed in Section 9.1. Finally, a quality assessment criterion (Table 9.3) was performed on the selected papers. A list of the papers finally selected along with their quality score is presented in Appendix A.

• Data Synthesis: Data synthesis was performed by the project teams and a list of challenges was created from 24 articles extracted from the final phase of the selection process. The research questions were evaluated using the data extracted from the selected articles.

9.2.3 Stage-3: Reporting the Review Process

9.2.3.1 Distribution of Final Selected Papers Based on Their Type

The 24 final selected articles included 12 journal articles, 10 conference papers, 01 workshop articles, and 01 book chapters. Most of the selected studies (65 out of 78) are journal and conference articles.

9.3 Results

In this section, the factors are extracted from the finally selected articles based on the research questions discussed in Section 9.1.

9.3.1 RQ1 (Challenges Identified in the Literature)

To answer RQ1, Table 9.3 provides a list of challenges investigated using the SLR approach. These factors were identified from the 24 extracted primary studies. The most frequent challenges are identified in Table 9.5.

9.3.3 RQ3 (Development and Operation Analysis)

We have categorized the identified challenges between the developers and operations teams. The categorization was carried out by taking responses of the practitioners who were part of the developers and operations teams of DevOps. A total of 12 practitioners (06 Developers, and 05 Operations) of DevOps were requested to take part in our study. After getting their consents, the process of categorization was started in which the list of the identified challenges was shared with them and they were requested to categorize the factors into Developers and Operations categories. The details of the practitioners who participated in the categorizations of the factors are given in Appendix B.

A Fisher’s exact test was performed to test the differences and similarities of challenges between the developers and operations teams of DevOps. The findings of the Fisher’s test shown that both developer and operation teams of DevOps have more similarities than differences for the identified challenges.

Following are the results of the categorization of challenges presented in Table 9.6: Cultural change (Ch1: 83% and 80%), lack of training (Ch3: 83% and 80%), shortage of tool knowledge (Ch4: 83% and 80%), optimizing deployment tool (Ch5: 83% and 80%), managing multiple environments (Ch7: 83% and 80%), lack of adopt DevSecOps model (Ch10: 83% and 80%), use of immature automated deployment tools (Ch12: 83% and 80%), Lack of motivation (Ch15: 83% and 80%), lack of planning for DevOps transformation (Ch17: 83% and 80%), lack of transparency (Ch18: 83% and 80%).

9.4 Discussion and Summary

The objective of this research was to investigate the challenges that need to be addressed for successful implementation of DevOps.

To answer RQ1, a total of 18 challenges for effective implementation of DevOps have been identified from the 24 selected primary studies. The identified challenges represent the key areas where the software practitioners need to focus to improve the DevOps management practices in the software development.

To address RQ2, the criticality of each of the investigated challenges is determined using the criteria discussed in Section 9.2. A total of eleven challenges were found as most critical challenges: Ch1: cultural change, Ch3: lack of training, Ch5: optimizing deployment pipeline, Ch6: lack of knowledge sharing, Ch7: managing multiple environments, Ch8: securing your infrastructure, Ch9: lack of infrastructure, Ch10: lack of adopt a DevSecOps model, Ch12: using unsuitable performance metrics for security evaluation, Ch15: lack of motivation (88%), Ch17: lack of planning for DevOps transformation.

We have classified the identified factors between developer and vendor categories in order to address RQ3. The results of the Fisher’s exact test analysis shows that both development and operations teams have more similarities than differences between the identified challenges. The most common challenges encountered between development and operations teams while implementing DevOps are identified as: cultural change (Ch1: 83% and 80%), lack of training (Ch3: 83% and 80%), shortage of tool knowledge (Ch4: 83% and 80%), optimizing deployment tool (Ch5: 83% and 80%), managing multiple environments (Ch7: 83% and 80%), lack of adopt DevSecOps model (Ch10: 83% and 80%), use of immature automated deployment tools (Ch12: 83% and 80%), Lack of motivation (Ch15: 83% and 80%), lack of planning for DevOps transformation (Ch17: 83% and 80%), lack of transparency (Ch18: 83% and 80%).

9.5 Threats to Validity

There are several possible threats to validity in this study [17, 18]. The first possible threat is related to the SLR process. All the steps involved in the selection of primary studies were collected by the first author. However, all the authors discussed how to improve this threat by observing any issues that are unclear. However, a higher risk of biases exists because a single researcher could be biased and extract wrong data continuously. One possible threat is related to the internal validity of the study, which is referred to as the overall discussion and evaluation of the results. The outcomes of the discussion with the DevOps experts and practitioners provides an acceptable level of internal validity of categorizations of the factors between the developers and operations categorizations. One possible limitation is related to the review process used to extract the challenges. We might miss some of the relevant challenges because the scope of SLR is limited to five digital libraries (IEEE, Science Direct, ACM, Springer, John Wiley and Google Scholar) and a limited number of search keywords could be missed in the number of articles related to the implementation of DevOps. However, this is not a systematic omission as the SLR process has supported search strategy based on the previous SLR studies [17, 18]. The authors believe that the selected digital libraries and search strings are sufficient enough to investigate the recent DevOps literature for investigating the challenges to implement DevOps.

9.6 Conclusions and Future Study

Presently, the software organizations are following the DevOps framework for efficiently and effectively developing software product in order to develop a commercially viable quality product to satisfy their clients. The increasing trend of implementing DevOps in software development motivated us to investigate the challenges that negatively impact the implementation of DevOps practices. The SLR approach has been used and a total of 24 primary studies have been selected. A total of 18 challenges were investigated from the selected primary studies. Moreover, the identified challenges were categorized between development-operations teams. The results indicated that most of the challenges are the same for both development and operations teams of DevOps. This categorization could help both DevOps researchers and practitioners effectively in software organizations by addressing both development and operation teams.

In a future study, a multivocal literature review study will be conducted for investigating additional factors that impact negatively on the DevOps practices. Moreover, we also plan to conduct an empirical study to investigate and validate the factors identified in the literature. In this study, we only investigated the challenges, but this study can be extended by conducting pairwise comparisons between the factors and developing a priority-based framework which could help the DevOps practitioners address these challenges based on their priorities.

Appendix A. Selected Primary Studies

Appendix B. DevOps Practitioners