SIMULTANEOUS INTEGRATION: DevSecSafOps draws parallels with DevSecOps by automating and integrating software safety testing during software development. (Photo by ThisIsEngineering, Pexels)
Rapid software development with DevSecSafOps integrates safety into modern software development practices.
by Franklin J. Marotta, Camille E. Houston and Melissa Reinard Steffen, Ph.D.
Software is at the heart of many warfighting platforms. To meet Army transformation initiatives, software must be capable of being rapidly modified and deployed. The Department of Defense Instruction (DoDI) 5000.87, “Operation of the Software Acquisition Pathway,” describes how modern software development practices, such as agile and development, security and operations, or DevSecOps, deliver products at a speed of relevance. However, these development processes do not directly consider safety and can result in unsafe outcomes.
In an Army where autonomous, artificially intelligent and robotic systems operate with minimal human input, the ability of the software to operate safely and to notify users of an unsafe outcome is even more critical. Development, security, safety and operations, or DevSecSafOps, is a new approach for integrating software safety concepts into the DevSecOps process.
Reminiscent to how ‘security’ was inserted into the development and operations, or DevOps, process to improve the cyber resilience and timeliness of software delivery, DevSecSafOps ensures software impacts to safety are characterized and mitigated during product development. Software is a source of military advantage, a shift to a DevSecSafOps process is vital to keep pace with rapid software fielding timelines.
BETTER SAFE THAN SORRY: Ensuring software safety risks are identified and eliminated or mitigated early in development will ensure fielding timelines can be met. (Photo by Markus Spiske, Pexels)
EVOLUTION OF SOFTWARE DEVELOPMENT
Software development practices have evolved significantly over time. The Waterfall model of the 1980’s developed software through linear sequential phases of requirements definition, design, coding, testing and maintenance; a method best used for creating predictable and well-defined software products. One of the disadvantages of this model is that government software testing does not occur until development is complete, resulting in product delivery delays. The Waterfall model also lacks the flexibility and user engagement which is needed to keep up with the pace of change in the defense industry. As a result, Agile principles began to be incorporated into product development in the early 2000s. Agile actively engages the user and allows the requirements, or user stories, to be continually refined throughout product development. Since Agile delivers software in increments, the users can test and provide feedback on functionality and use during development. In the later 2000s, the DevOps software development method applied the Agile concepts of customer engagement during software development to the deployment and maintenance operations that follow product delivery. The DevOps concept removes the barriers between the developers and the deployment teams, ensuring customer feedback is used to improve the product throughout its life cycle. During the 2010s, the expansion of network and cloud-based platforms increased the need to incorporate security into software development. However, security testing was executed late in DevOps, causing delays. As a result, the DevSecOps model emerged (Figure 1).
DevSecOps ensures that security decisions are made at the same time as development and operational decisions, and that security testing is automated, improving software deployment. In 2020, the best practices of Agile and DevSecOps were described in the DoDI 5000.87. The DoDI requires capability needs statements, user engagement, value assessments and metrics and iterative delivery of capability to users within a year. The intent of the Software Acquisition Pathway is to deliver effective, resilient, supportable and affordable software solutions to the end user, adhering to safety critical software standards and guidance, ensuring execution at the speed of relevance.
COMMUNITY OF PRACTICE: The DevSecOps process aims to unify software development, security and operations. The main characteristic of DevSecOps is to automate, monitor and apply security at all phases of the software life cycle. (Image courtesy of Air Force DevSecOps)
SOFTWARE SAFETY
Software safety presents a parallel challenge to rapid software development as cybersecurity posed to DevOps in the 2010s. Software safety is not directly considered during DevSecOps and testing for safety is executed late in the development cycle or is not executed at all. This can result in significant program delays due to the late discovery of software safety defects, urgent fixes to software safety defects, the need for additional testing, delayed delivery of the software product and elevated risk for hazards as a result of software safety defects. The ability of software to facilitate safe operations is critical, especially as the Army moves toward human-machine integration of autonomous, artificially intelligent and robotic systems operating with minimal human input. Without proper management, software can create safety hazards that impact personnel and infrastructure. For example, software safety risks identified late in government developmental testing can limit Soldier involvement in operational test events and result in use restrictions when the software update is fielded. The DoDI 5000.87 requires system safety assessments, safety critical risk assessment and mitigations and safety critical implications as part of the test strategy. Military Standard-(MIL-STD) 882E, DOD Standard Practice System Safety, describes the method to identify and eliminate or mitigate software safety hazards and defines risk in terms of probability and severity. Software that can cause a hazard is considered safety-significant software.
DEVELOPMENT, SECURITY, SAFETY, OPERATIONS
Integrating software safety engineering processes into DevSecOps, or DevSecSafOps, is proposed to reduce programmatic risk for software development while meeting MIL-STD-882E safety objectives (Figure 2). DevSecSafOps draws parallels with DevSecOps by automating and integrating software safety testing during software development. To create the biggest benefit for the program manager, MIL-STD-882E processes must be applied to the initial software configuration to establish a baseline. Safety is evaluated in the DevSecSafOps process when software changes implement new functionality; functionality is modified, removed or disabled; or corrections to software defects are implemented.
Once the scope of a planned software release is defined, the developer’s software safety engineer addresses the following questions as derived from MIL-STD-882E:
- Are new hazards introduced by the software changes?
- Are new hazard causes introduced by the software changes?
- Are new hazard mitigations introduced by the software changes?
- Are new mitigation verifications required by the software changes?
- Has the Hazard Tracking System (HTS) been updated appropriately?
- Have Software Control Categories (SCC), Software Criticality Index (SwCI) and Level of Rigor (LOR) tasks been assigned for newly developed or modified software?
- Is there evidence that LOR tasks have been performed?
- Are new safety-significant test cases added to the software (SW) test suite?
- Are new software problems discovered in the software update in the Software Problem Reports (SPR)? If so, do they have safety impacts?
DevSecSafOps ensures that software safety documentation is updated, safety analysis is conducted and necessary test cases are developed and executed as the software is developed and released.
SOFTWARE SAFETY ENGINEERING PROCESSES: The DevSecSafOps process builds upon DevSecOps by integrating software safety engineering in the software development process automating safety-significant test cases, implementing continuous safety monitoring during operations and ensuring stakeholder engagement. (Image courtesy of the authors)
The DevSecOps principle of test case automation is also a key element in DevSecSafOps. All safety-significant software testing cases within the test suite are tagged to be reviewed for their robustness by the developer’s software safety engineer and other stakeholders. As new requirements are implemented for a given software release, the automated safety-significant test suite is also updated. Safety-significant test cases that cannot be automated are executed at the system level. Software safety is considered throughout the operations phase through real-time monitoring to ensure ongoing safety assurance during operation. Safety-significant outcomes during operations are automatically collected, reported and analyzed for resolution.
STAKEHOLDER ENGAGEMENT
The successful implementation of a DevSecSafOps environment requires the engagement stakeholders beyond the software developers and software safety engineers. The user, the U.S. Army Test and Evaluation Command (ATEC) and the program manager also play important roles in software safety. The user provides essential feedback on the software’s function, helping to identify potential issues and improvements. To ensure Soldiers can safely use the system, ATEC is tasked with developing safety release and safety confirmation documents. Program managers oversee the acquisition process, from development to fielding, ensuring the system meets its contractual requirements and adheres to Army, DOD and industry standards. To enable DevSecSafOps principles during software development, program managers should specify security standards, safety regulations, operational best practices and contractual responsibilities in their contracts. In doing so, they can ensure that vendors adhere to secure, safe and reliable development, testing and deployment practices for their software-intensive systems. By incorporating DevSecSafOps principles early in the development process, stakeholders can gain efficiencies, improve Soldier interactions, reduce programmatic risk and expedite fielding timelines.
CONCLUSION
To maintain a competitive edge on the battlefield, the Army must be able to rapidly develop, upgrade and deploy software to weapons and warfighting systems. Ensuring software safety risks are identified and eliminated or mitigated early in development will ensure fielding timelines can be met. DevSecSafOps combines best practices of Agile and DevSecOps software development processes with procedures for software safety found in MIL-STD-882E, resulting in an efficient software development process that mitigates safety risk, ultimately delivering software at a speed of relevance.
For more information, go to https://www.atec.army.mil/atc.
FRANKLIN J. MAROTTA is a senior mathematician for software safety and artificial intelligence within the Modeling, Simulation and Software Test Division, assigned to the U.S. Army Aberdeen Test Center. He holds a B.A. in mathematics from the University of Rochester. He is a DAWIA Certified Practitioner in test and evaluation.
CAMILLE E. HOUSTON is the director of the Virtual Test and Advanced Electronics Directorate assigned to the U.S. Army Aberdeen Test Center at Aberdeen Proving Ground, Maryland. She holds an M.S. in systems engineering from Johns Hopkins University and a B.S. in mechanical engineering from Lawrence Technological University. She is a member of the Army Acquisition Corps and is a DAWIA Certified Practitioner in test and evaluation.
MELISSA REINARD STEFFEN, PH.D., is the technical director of the U.S. Army Aberdeen Test Center at Aberdeen Proving Ground, Maryland. She holds a Ph.D. in analytical chemistry from the University of Delaware and a B.A. in chemistry from Franklin & Marshall College. She is a member of the Army Acquisition Corps and is a DAWIA Certified Practitioner in test and evaluation.
