Agil 2.0 - DevOps for Software Development

The term DevOps is composed of the following two words: Dev stands for Development and Ops for Operations. Both terms combined stand for a process improvement approach from software development and IT operations.

The goal is to unite people, processes and technologies to deliver consistently high quality products. DevOps stands for the ability to deliver a software product at short intervals. Through a better interaction between software development and IT operations, it is possible to accelerate software development and improve the quality of the products.

At first glance, DevOps seems to be purely technologically driven, but it all starts with the organizational culture and with the people involved in the processes. The introduction of a DevOps culture requires far-reaching changes in the way employees work and collaborate. At M&M we have established more efficient teams through the introduction of a DevOps culture, together with our agile software development process:

"Automation has reduced the error rate and we are able to deliver software faster and more effectively"

Daniel Marcek, Group Leader, Business Unit Software Applications at M&M

One of the most important factors in the DevOps culture is teamwork. Development and IT operations must disclose and reveal their DevOps processes, priorities and responsibilities to each other.

Previously separate roles such as software development, IT operations, quality control and security are combined and coordinated in the DevOps team. Where software developers used to be purely responsible for the development of software, the spectrum of their responsibilities is expanding in the direction of IT operations.

DevOps teams are flexible, because the software is released in short cycles. Shorter release cycles also simplify planning and risk management as it is an incremental process and can be readjusted permanently. Due to short feedback loops from the customer, it is possible to react very quickly to the changed requirements in order to create a competitive advantage.

By introducing agile ideas into the software life cycle, M&M has created the possibility to provide software deliveries in very short intervals. Trusting in the abilities of the team to find the best possible solution, enormous potentials were awakened. This cultural change in software development enables a team to identify more closely with the product. It also reduces release cycles and thereby increases software quality.

1. higher productivity / efficient
2. Automation reduces manual errors
3. shorter release cycles
4. improved customer satisfaction
5. shorter feedback loop
6. continuous information flow

The continuous DevOps cycle provides for the following methods and practices:

• Plan

  Agile planning and project management is used to break work steps into sprints, manage team capacity, and adapt teams to changing business objectives. During the planning phase, ideas for new features and characteristics of a given application (or system) is being collected, defined and described. Tracking progress at both low and high granularity, for example, individual tasks or entire portfolios of multiple products, helps to ensure continuous improvement during this stage.

• Code

  Using the plan created in the previous phase, the new features or properties of the application or system are being implemented. This includes coding, unit testing, code review, and integration into code written by other team members.

• Build

  Continuous Integration (CI) has long been an established term in software development and plays an important role at DevOps. In this stage, well-formed build definitions are used to compile written code in a pipeline, validate it via unit, integration or end-to-end tests and analyze it with static code analysis. If the build fails, tests report errors or the code quality does not meet project standards, the entire pipeline will fail, and the responsible developer will be notified to correct/improve their code. All steps in this stage should be fully automated and should be performed on each change to the code in the repository. The result of this stage is one or more versioned and deployable, integratable or executable build artifacts and KPIs for code quality and test coverage.

• Test

  As soon as new build artifacts are generated from the build phase, they can be deployed in a test or staging environment - optimally again fully automated. These environments can either already exist, i.e. they can be reused in each cycle, or they can be set up specifically for this stage. The latter can be mapped automatically through a pipeline using methods such as Infrastructure as Code (IaC). Manual tests, such as acceptance tests, as well as automated tests can be executed against this test or staging environment. Automated tests can include UI tests, for example, but also security scanners that examine the environment for gaps, such as unintentionally opened ports.

• Release

  The release stage is a milestone in the DevOps cycle. The code or build artifacts that have made it this far have now successfully passed through a series of analysis and testing and are ready for deployment into a production environment. Depending on how much DevOps is lived in an organization, this stage can also run automatically after the test phase. Alternatively, manual approvals from the product owner, project or product manager can trigger the next steps.

• Deploy

  Here the application is now deployed into the production system and launched live. Infrastructure as Code ensures that the production environment (except for the configuration) does not differ from the test or staging environment. In this way, fundamental errors in the infrastructure can be avoided. Deployment strategies, such as Blue/Green deployment, enable the release of a new version of the application without downtime. If, despite extensive testing in the previous stages, a critical bug in the new version of the production system is still noticed, it is even possible to switch back to the old version within seconds.

• Operate

  Once the changes in production, trouble-free operation must be ensured. In this stage, resources are scaled according to the application's workload, continuous availability is ensured and support requests from customers are responded to.

• Monitor

  Although this is the final stage of the DevOps cycle, it should be operated through all phases. On the one hand, the application itself must be monitored so that feedback regarding new features or system characteristics can be incorporated into the next planning phase. On the other hand, the individual processes within the DevOps cycle should also be considered in order to achieve continuous improvement in the next cycles.

DevSecOps will become increasingly important in the future. DevSecOps extends DevOps with a way to approach IT security with a "everyone is responsible for security" mindset. Security practices are introduced into the DevOps pipeline of an organization. The goal is to consider security in all stages of the software development workflow.

• DevOps should be implemented in projects from the beginning.
• The initial effort is higher but the daily work in the project and the whole process will be much more effective afterwards.
• It is recommended to work with templates. Templates facilitate automation and can be reused in other projects.
• A DevOps Engineer should be integrated into the team as a central contact person for processes and tooling.
• Significant reduction of the error rate: With the help of automation, manual errors are eliminated.
• A high degree of test automation is a must-have

DevOps only works with an effective use of tools. These tools are designed to automate manual tasks and support teams in managing complex and large environments. We at M&M use Microsoft Azure DevOps as standard. With the help of Azure DevOps, our teams can implement the complete DevOps process uniformly and with one tool.

• Boards
  Azure Boards offers popular tools for agile software planning, such as Kanban Boards, (Sprint) Backlogs and Scrum Boards. In addition, dashboards can be used to visualize KPIs tailored to the project to get an overview of the status of past or current sprints.  
• Repos
  M&M uses GIT repositories by default. Azure Repos extends the version control system Git with useful features that improve the DevOps workflow and increase the degree of automation:
  - Pull Requests: Allow easy collaboration between multiple developers and enable detailed code reviews before new code enters the shared code base.
  - Branch Policies: Targeted protection of important branches in the repository. For example, the changes of a pull request can be validated by builds before they are transferred to the common code base.
  - Build Trigger: Triggering CI/CD when changes are made in the repository to implement a fully automated DevOps pipeline.
• Pipelines
  The heart of DevOps. With Azure Pipelines, there are no limits to automating any actions within a software project. In principle, all programming languages or frameworks can be integrated here and deployed to many different cloud providers. Until now, build and release pipelines were separate. Now both steps are combined in a multi-stage pipeline, defined using YAML, and stored in the repository next to the application source code, in the spirit of DevOps .
• Test Plans
  Azure Test Plans provide an environment for planning, executing and tracking exploratory and manual tests. Test plans are grouped into test suites that contain the individual test cases. Furthermore, different system configurations can be defined, e.g. a subdivision into different operating systems or browsers. When executing manual tests, the results of the individual steps can be documented directly via Web UI with screenshot or video recording.
• Artifacts
  Azure Artifacts provides a fully managed package management system with native support for NuGet, npm, Maven and Python packages. Here, private and public feeds can be created in several levels (local, pre-release, release), which can be accessed by the developer PC with the standard tools as well as in Azure pipelines.