A Study of Awareness and Practices in Pakistan’s Software Industry towards DevOps Readiness

Khadija Saleem¹, Muhammad Sohail Khan¹

¹Department of Computer Software Engineering, University of Engineering and Technology Mardan, Pakistan.

* Correspondence: Khadija Saleem, Email: khadijasaleem027@gmail.com

Citation | Saleem.K, Khan.SM “A Study of Awareness and Practices in Pakistan’s Software Industry towards DevOps Readiness” International Journal of Innovations in Science and Technology Vol 3 Issue 3 PP 102-115

Received | Aug 01, 2021; Revised | Aug 26, 2021 Accepted | Sep 17, 2021; Published | Sep 19, 2021

________________________________________________________________________

There are regular conflicts between the traditionally divided software organization i.e. dev and ops teams during the software development process for delivering the software to the end-user. DevOps overcome these conflicts by automating the processes between the development and operations team in such a way that they can build, test, and release the software successfully and efficiently to the end-user. Globally, more and more organizations are adopting DevOps. As Pakistan’s software industry is progressively growing while DevOps is a relatively new concept, there is a need for DevOps awareness and understanding towards its adoption and practices. This paper evaluated DevOps awareness and identified the practices adopted in Pakistan’s software organizations and suggested generic guidelines for DevOps transition. A questionnaire-based survey is conducted to collect data and various DevOps sub-activities being practiced. The survey analysis and results depicted that Pakistan’s Software Industry is making efforts towards the adoption of DevOps but due to lack of its awareness, most of the DevOps practices are not fully adopted yet. According to the DevOps evolution model, only one-eighth of Pakistan’s software organizations are at the self-service stage, while the rest of them are still struggling at the normalization and standardization stage.

Keywords: DevOps; development and IT operations; agile; continuous integration and continuous deployment.

INTRODUCTION

Globally, the need for software-intensive products is highly increasing, day by day. Software products that are reliable, smart, secure, useful, and scalable at all times during operational use. Pakistan’s software industry is also growing impressively over the past years[1]. Challenges emerge with such growth in the software industry due to the organizational teams involving in software delivery. Conventionally, software organizations are mainly divided into development and IT operations teams[2]. Due to their different roles and responsibilities, conflicts among them occur frequently. Operations tend towards more stable and secure delivery of software and imply that developers do not make changes frequently, on the other hand, developers are more interested in delivering new changes and features to end-users speedily[3]. Such conflicts can delay the software delivery[4] and make the two groups unaware of each other’s problems hence developing stress on work and communication barriers between them. To overcome this problem and bridge the gaps between them is where DevOps comes in, by adjusting the approaches shared for the entire development process.

DevOps is the fusion of words; development (that includes programmers, testers, and quality assurance analysts) and operations (that includes network technicians, sysadmins, and DBAs)[5]. To resolve the critical problems during software development DevOps involves such patterns that improve communication and collaboration among the dev and ops teams. Those critical problems include the fear of frequent changes and making the deployment risky. DevOps reduces the gap between the dev and ops teams, ensuring the detection of issues earlier and providing proper steps to address them in software delivery process[6]. Collaboration among everyone participating in delivering software is a key DevOps tenant[7]. Due to DevOps development, globally, many organizations start to implement DevOps during their development phase. The puppets lab[8 , 9,10,11, 12, 13, 14 15, 16] conducted surveys reporting the immense increase in respondents that are part of DevOps or having started DevOps efforts in their organizations.

Background – DevOps is founded on the idea of frequent agile development[17] changes and deployment cycles. Previously, DevOps had no academic literature but now over the past few years, studies and surveys are conducted globally to discuss DevOps in terms of adoption, benefits, and challenges.

The puppets lab formulated a DevOps evolution model[18] based on the results and analysis of their surveys. Following are the five stages mentioned in the model, based on the practices adopted by successful organizations in the journey of DevOps.

Stage 1 – Normalization: Teams adopting agile methods and properly implementing version control to be ready for continuous integration and deployment[18].

Stage 2 – Standardization: Dev and Ops teams are working together towards the same goal in a way to make sure the complexity of processes is reduced, enabling early collaboration opportunities[18].

Stage 3 – Expansion: At this point, any previously created discrepancies are removed, as the foundation is built for DevOps. The dev team works on the same level as the ops deployed version of the software. Also, the team member can work without wasting time on work approval from outside the team[18].

Stage 4 – Automated infrastructure delivery: The automation of code deployment helps organizations to lessen the conflicts among dev and ops teams. So, when things like security configurations, system configurations, and provisioning are automated, the DevOps team can deliver faster and is better set up for future self-service[18].

Stage 5 – Self-service: The capabilities of this stage are already created and defined in the previous stage, but they are further realized and more resources are made available in the fifth stage. It gives the application developers the authority to deploy testing environments on their own and success metrics are visible to the team[18].

In this global race, further usage of conventional practices may result in loss of business at a minimum, more stress between the teams, inefficiency to deliver the software product continuously on time, and hence collapse of the industry as a worst-case scenario. Researches and surveys are being conducted globally for DevOps regarding adoption, practices, challenges, and benefits, providing evidence that DevOps delivers value and statistics verify fundamental capabilities and techniques that help organizations to improve their software development and deployment[19]. Pakistan’s software industry, despite its rapid growth, there is no such account of DevOps literature about the level of DevOps awareness or readiness for the adoption of these practices. Organizations are unwilling in changing their conventional techniques due to lack of such awareness and academic studies and without proper awareness, people are afraid to change their ways because it may cause them a negative impact without knowledge.

The purpose of this study is to analyze the DevOps awareness in Pakistan’s software industry, its related practices implemented, and provide the generic guidelines for organizations. So, the researchers and practitioners can understand the importance of DevOps and adoption through this study.

Material and Methods.

A questionnaire-based survey methodology is used for this study for collecting quantitative data regarding the state of DevOps awareness, adoption, and implementation across Pakistan’s software industry. After thorough literature review and data collection, the questionnaire design is formulated which consists of questions regarding the software development processes currently practiced by the organizations i.e. about development methods, version control, code integration and deployment processes, about the software teams, and overall employees’ satisfaction regarding the work and workplace environment.

Figure 1 shows the overall research methodology of this study. The survey form (using online ‘Google Forms’) is distributed via different platforms as LinkedIn connections, email, through professional connections, coworkers, and colleagues currently working, in order to get maximum responses. Out of more than 500 practitioners being approached, only 191 responded back as a survey response. The data collected is then validated and processed in a form to be analyzed for generating results and recommendations.

Figure 1. Research methodology.

For this study, a questionnaire-based survey is conducted across Pakistan’s software industry. The survey questions are formulated in a way that respondents can easily understand and responds to the best of their knowledge. Survey questions are categorized under the following sections:

Generic – Basic questions about respondents as email, organization, and designation at work.

Company Size – Software organizations vary in different sizes depending upon the number of employees and their respective project needs. Company size plays a key role when it comes to adopting and improving DevOps, by evaluating the DevOps adoption in more or less to happen on its own or needs deliberate steps to implement it. Survey question includes; what is the number of employees in your organization?

DevOps – Survey question includes; rate your level of knowledge about DevOps. This information will help in determining the other following survey questions that what practices are followed by the organizations belonging to the respondents whether or not having DevOps knowledge.

Agile Methodology – Agile and DevOps involve different teams and departments, having different approaches for the development of software[20]. They are not mutually exclusive as DevOps is a culture, focusing more on continuous integration and deployment along with collaboration and sharing between all teams that are involved in the development and operation of the software. While agile is on the development side maintaining productivity and rapid releases of changes or new features. Even though, agile and DevOps are relatively different with respective goals, when they are used together in any organization, provides more reliable and efficient results[20]. Survey question includes:

Does your organization employ Agile development?
Which Agile development methodologies have been employed by your organization?

Working Environment – Working environment of any software organization can be identified by knowing the type of teams/groups, the team’s collaborations, and sharing of the tools, discoveries, and lessons among all team members. Survey question includes:

Does your organization have separate teams/groups for development, operations, quality assurance, etc.?
Is there any regular collaboration between your Development and Operation groups?
How well does your organization share?

Employee Work Satisfaction – Employee work satisfaction is all about the employee working in a comfortable environment that is stress-free and relaxed, and doing work that’s challenging and meaningful, and empowered to practice the skills and findings. Employee work satisfaction also highly correlates with DevOps practices and culture. Survey question includes:

Overall organizational environment?
Organizational working process?

Version Control – Version control is managed by software tools that help the software team to maintain track of changes to source code over time. One of the ways to successful DevOps teams is version control, reducing development time, and increasing the rate of successful deployments[21]. It helps software teams to collaborate more rapidly as required by today’s frequently shifting IT environment. According to [10] version control is one of the top 5 tools used to support DevOps initiatives. Survey question includes:

Is version control practiced?
Do you use Git or any other tool for version control?
How often do you use Git (or the other tool) for version control?

Continuous Integration & Continuous Deployment (CI/CD) – Version Control is a prerequisite for continuous integration and continuous deployment i.e. everything required to reproduce the production environment must be checked into version control.

Continuous integration makes developers merge more often their changes back to the main branch. These changes or updates are validated by creating a build, eliminating the integration challenges that can happen when changes are merged into the release branch on release time. Survey question includes:

Do you integrate code continuously?
How often do you merge all the changes into a shared control repository?

Continuous deployment is a step further from continuous integration. With continuous deployment, all the changes or updates are released to the end-user bypassing all the stages of the production pipeline. It eliminates the stress on team members so that developers can focus on building software, not on release stress, and they can see their work go live within minutes after they've finished working on it. Survey question includes:

Do you deploy the code manually?
Do you deploy the code automatically?

Results and Discussions.

The survey conducted involves IT professionals and practitioners belonging to different companies and groups/teams across Pakistan’s software industry. A total of 191 responses are collected, and the respondents belong to the following different categorizations, as shown in Figure 2.

Figure 2. Survey Respondents Categorization.

Table 1 shows the overall summary of all the responses collected. It is observed that:

68% of respondents belong to medium to large-scale companies.
42% of respondents have good enough knowledge about DevOps.
84% of respondents do practice agile in their organizations.
35% of respondents do have separate Ops teams in their organizations, and only 13% of them do collaborate and share regularly.
67% of respondents are satisfied by their working environment and working processes.
32% of respondents fully practice version control at all times for all projects, and 57% of them partially practice version control.
15% of respondents do automatic CI/CD, and 32% of them do both automatic and manual CI/CD.

Table 1. Survey Responses Summary

Survey Topics	Categories	Total Responses
Company Size	Small-scale	22%
	Medium-scale	35%
	Large-scale	33%
	Large Enterprises	9%
	Uncertain	1%
DevOps Knowledge	Good enough knowledge	42%
DevOps Knowledge	Not enough knowledge	58%
Agile Methodology	Do practice agile	84%
Agile Methodology	Do not practice agile	16%
Working Environment	No Ops Teams, Collaboration & Sharing Insignificant	65%
	Dev & Ops Teams with Collaboration & Sharing	13%
	Dev & Ops Teams with No Collaboration & Sharing	9%
	Dev & Ops Teams with Indeterminate Collaboration & Sharing	13%
Employee Work Satisfaction	Satisfied	67%
	Dissatisfied	18%
	Indeterminate	15%
Version Control	Yes, do practice (fully)	32%
	Yes, do practice (partially)	57%
	Do not practice	2%
	Indeterminate	9%
CI/CD	Automated	15%
	Manual	2%
	Both (Automated & Manual)	32%
	Indeterminate	51%

According to Table 1, the survey response for ‘DevOps Knowledge’ indicates that approximately more than half of respondents do not have enough DevOps knowledge. Respondents may be working along the practices of DevOps in their organization but due to no awareness, the respondent may not know about it and the other way around. To observe and analyze the data for both types of respondents, we divide overall responses data into 2 groups as shown in Figure 3. It is important to observe and analyze the 2 groups (Group A & B) separately to have better insight for DevOps awareness and their respective organization software development patterns.

Figure 3. Group A & B.

Figure 4 shows that the majority of respondents (Group A ~ 63% while Group B ~ 71%) belong to medium to large-scale companies, it is better to adopt DevOps practices much earlier. Small and medium-scale companies can adopt DevOps much faster than large-scale companies if provided with the resources and are likely to adopt cultural change. For large-scale companies, it is a must to start adopting DevOps practices in sub-pilot base projects if the old projects are reluctant to adopt change. Because large-scale companies are having more employees and teams that need to collaborate and they must be on one page to deliver successful software without any hindrances.

Figure 4. Company Size Comparison of Group A & B.

Figure 5 shows that the majority of respondents’ organizations are implementing agile (Group A ~ 94% while Group B ~ 77%). Implementing agile means more rapid changes in work and delivery to customers. Which is of great value only on the dev side. Doing planning with the dev and ops team has more chances of knowledge sharing and collaboration, which reduces the gap between the dev and ops team. The ops team gets a better idea of the logic and the end deliverable, and suggests a solution to the dev on which both teams agree to work. Agile with adopted DevOps makes it much more beneficial for organizations.

Figure 5. Agile Methodology Comparison of Group A & B.

Figure 6 shows that approx. one-third of the organizations have separate ops teams with limited collaboration and sharing among the teams (Group A ~ 17% while Group B ~ 8%). Organizations having only a dev team doing all the work with no-ops teams are more likely to exist in small-scale companies where resources are scarce. Hence there is no debate about the conflicts of dev and ops teams. But it makes a huge burden and pressure on one team if they are doing things the manual way. The statistical figures show that the organizations having ops teams also seems to have limited collaboration and sharing among them (which is a drawback as communication gap).

Figure 6. Working Environment Comparison of Group A & B.

Figure 7 shows that the majority of group A respondents (Group A ~ 83%) are satisfied by their organizational working processes, which reflects the fact that the practices adopted by their organizations are making them feel stress-free of the conflicts and they focus more on work rather than fixing and handling conflicts among the teams. Group B approx. half the respondents (Group B ~ 54%) are satisfied while the remaining half of them are dissatisfied or indeterminate about their organizational working processes and environment which is a straight fact for time to change the manual processes into continuous and automatic processes.

Figure 7. Employee Work Satisfaction Comparison of Group A & B.

Figure 8 shows that the majority of respondent’s organizations are practicing version control either fully (Group A ~ 37% while Group B ~ 28%) or partially (Group A ~ 54% while Group B ~ 59%), that is the most common and basic step towards the DevOps journey. It is a must to have 100% full version control being adopted for all the projects in all organizations, for fast code shipment and integration.

Figure 8. Version Control Comparison of Group A & B.

Figure 9 shows relatively low statistics for automated CI/CD for both groups as organizations practicing CI/CD either fully (Group A ~ 22% while Group B ~ 9%) or partially^[1] (Group A ~ 31% while Group B ~ 34%). Version control is linked with CI/CD as it provides the master repository for continuous integration and leads to continuous deployment. The organization must focus more on after version control to CI/CD to increase deployment rate and reduce manual resource effort.

Figure 9. CI/CD Comparison of Group A & B.

Based on the above discussion, it is observed that Group A is on right track^[2] towards the DevOps journey, and Group B is struggling at the initial stages of the DevOps journey. Even though group A is better than group B but the statistics show similarity between them (to some extend), clearly indicating the lack of DevOps awareness among group B. According to ‘DevOps Evolution Model’:

Group A – only 17% of organizations are at the self-service stage of the DevOps evolution model. The rest of them are still struggling at normalization to standardization.
Group B – only 8% of the organizations are at the expansion stage of the DevOps evolution model. The rest of them are still struggling at normalization to standardization.

Overall generic guidelines for all Pakistan’s software organizations are:

To adopt 100% version control as it is the backbone for dev and ops, their first step towards the DevOps journey should be version control, as its the code that will be deployed to the server by the ops team and it will be hard to rollback without version control, plus both teams can work on same code when there is version control.
Without CI/CD the DevOps life cycle is incomplete, it’s easy for a company to do CI/CD as they are already in the managed lifecycle. Automating continuous deployments provides several benefits that directly contribute to high performance. The more reliable, secure, automated CI/CD leads to a happy dev environment and it is only possible when the Ops team taking care of operation and the dev team only doing the code without worrying about anything else.

Conclusion.

In this study, we analyzed the state of DevOps awareness across Pakistan’s software Industry and the relative practices being adopted. DevOps is rarely an exposed concept to university and small to medium-scale companies. The division of data into two groups^[3] also shows that both the groups are adopting the same practices^[4] but the only difference is that they do not have the proper awareness of this concept. In this study, we analyzed the state of DevOps awareness across Pakistan’s Software Industry and the relative practices being adopted. According to [18], only 13% of the organizations are at the self-service stage of the DevOps evolution model while the rest of them are still struggling from the stage 1 (Normalization) to stage 2 (Standardization). Only 15% of them adopted DevOps practices including separate experts for handling the DevOps tools and practices across the dev and ops teams. 32% of them are partially adopting DevOps depending upon projects while the rest of 53% belong to the manual or the indeterminate state of DevOps indicates that they are still not adopting DevOps or not sure of practices adopted by their organizations, that is why collaboration and sharing among teams are important.

Based on observation and discussions with IT practitioners, we agree that people are getting aware and trying to follow up with the DevOps trends but there is still a lot of confusion about the concept ‘DevOps’ due to lack of awareness. The statistics also show that if provided with proper knowledge of DevOps and the readiness of organizations to adopt the changes, it can boost Pakistan’s software industry in a much better and faster way.

In the future work, we will extend the study to the evolution of DevOps across Pakistan’s software industry over the years and see the implications, benefits, and challenges involved in the DevOps journey.

Acknowledgement.

First of all, I wish to express my deepest gratitude to Almighty Allah, for endless blessings that gave me the strength to complete this research study. I would like to thank all DevOps professionals specially Mr. Adil who helped in refinement of the survey questionnaire. I wish to extend my special thanks to all the practitioners and participants involved in the survey, who helped in collecting the data. Lastly, I would like to pay special regards to my parents and family for their endless support and prayers.

Author’s Contribution.

The idea for this study was conceptualized by Dr. Muhammad Sohail Khan. The study was performed under his supervision and contributed in analyzing and writing up the paper. Khadija Saleem conducted the study i.e. survey questions design and formulation, data collection, data processing and article writeup.

Conflict of interest. No conflict of interest for publishing this manuscript in IJIST.

REFRENCES

[1] https://www.pasha.org.pk/knowledge-center/industry-stats/

[2] J. Hamunen, “Challenges in Adopting a Devops Approach to Software Development and Operations MSc program in Information and Service Management Maisterin tutkinnon tutkielma Joonas Hamunen 2016 Tieto-ja palvelutalouden laitos Aalto-yliopisto Kauppakorkeakoulu,” pp: 69, 2016.

[3] Y. Liu and Y. Zhou, “The Challenges and Mitigation Strategies of Using DevOps during Software Development,” pp: 8-78, 2017.

[4] M. Hüttermann, “Introducing DevOps ”, pp: 15-31, 2012.

[5] https://www.chef.io/blog/what-devops-means-to-me

[6] J. Wettinger, U. Breitenbücher, and F. Leymann, “DevOpSlang - Bridging the gap between development and operations,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 8745 , pp: 108–122, 2014.

[7] Ernest Mueller, “What Is DevOps? | the agile admin,” 2010.

https://theagileadmin.com/what-is-devops/.

[8] Puppet Labs, “Take the 2012 DevOps Survey | Puppet.com,” 2012. [Online]. Available: https://puppet.com/blog/take-2012-devops-survey/. [Accessed: 19-Mar-2020].

[9] Puppet Labs, “2013 State of DevOps Report | Puppet.com,” 2013. [Online]. Available: https://puppet.com/resources/report/2013-state-devops-report/. [Accessed: 19-Mar-2020].