JPEO-CBD takes a fresh look at the required analysis of alternatives and finds ways to make it a more effective, less time-consuming tool.
by Ms. Breena Berté and Mr. Marshall Kindred
A recent study by the Joint Program Executive Office for Chemical and Biological Defense (JPEO-CBD) showed that the analyses of alternatives (AoAs) conducted within the Chemical Biological Defense Program (CBDP) listed, on average, 23 measures of effectiveness and 72 measures of performance to be considered before Milestone A. The AoA is an essential and required step in establishing the most operationally suitable, cost-effective options to provide a needed, potentially joint capability. But, JPEO-CBD found, attempting to analyze the possible trades among that many factors required significant time and effort without improved results.
In particular, the study concluded that AoAs were taking about 18 months in attempts to collect and analyze an average of 72 measures of performance (MoPs) on every available technology, when, in the end, the JPEO found that it could not collect all those data points. Even after investing the time and resources to collect the necessary data, there were significant gaps in the available information on the cost, schedule and performance of viable alternatives. And because there was no prioritizing of MoPs, the process failed to provide the milestone decision authority (MDA) the information it needed to direct the program toward the most advantageous alternatives.
To move toward an AoA process more attuned to the needs of individual programs, the JPEO-CBD is leading an effort across the CBDP to streamline the AoA process for Acquisition Category (ACAT) II and ACAT III programs. The goal is a streamlined, tailored approach that offers program managers and the MDA the precise information needed to make decisions early in the program’s life cycle while minimizing wasted time and effort. To this end, the JPEO-CBD has pinpointed areas in the Joint Capabilities Integration and Development System (JCIDS) process, which focuses on identifying needed capabilities and the associated requirements, that are adaptable while still providing specific enough information for the program to ensure affordability from the beginning.
THIS GOES HERE
JPEO-CBD pinpointed areas in the JCIDS process that are adaptable while still providing the right information for program decisions and now uses screening criteria to prioritize the most relevant measures of effectiveness and performance. (Image courtesy of iStock and the authors)
In general, this information provides broader direction at the program’s outset, with more granular analysis added if necessary as the program progresses. For example, at Milestone A, it is not possible to do a robust cost analysis or to analyze specific performance parameters because the data are insufficient, but it is possible to establish a rough order of magnitude for cost and performance to guide the program. Customizing the analysis could be as simple as reviewing swatch data early in a program to acquire wearable gear, to guide design by identifying which materials show promise or could reach a higher technology readiness level by the time the program reaches production, rather than running full assessments on all possible materials from the start.
As the Hon. Frank Kendall, undersecretary of defense for acquisition, technology and logistics, has said, “Our world is complex. One-size-fits-all cook-book solutions simply don’t work in many cases. … at the end of the day we have to figure out the best course of action in a specific circumstance, balancing all the complex factors that apply to a given situation.” Kendall’s comments come from his article “Better Buying Power Principles: What Are They?” in the January-February issue of Defense AT&L magazine.
As the Defense Acquisition Workforce moves to ensure the best value for acquisitions, there has been a push to identify and employ options in DOD Instruction (DODI) 5000.02, Operation of the Defense Acquisition System, for tailoring processes to fit the needs of a program. While there is no prescriptive list of what areas should be tailored, the general tenets of Better Buying Power can be applied to look critically at what adds value to the program. Many of the standard practices in DODI 5000.02 are targeted at ACAT I programs, but there are opportunities to customize many areas in our traditional processes for ACAT II and ACAT III programs. ACAT I programs are high-budget, large acquisition programs that come with the highest level of oversight. ACAT II and ACAT III programs have fewer oversight requirements because of their lower budgets, and therefore offer more areas of adaptability from ACAT I regulations.
In fact, DOD 5000.02 explicitly allows room for modification, allowing the MDA to make decisions that produce the right information to move a program forward.
NOT QUITE BY THE BOOK
As the only published document across DOD that prescribes how to execute an AoA, the Air Force’s Analysis of Alternatives (AoA) Handbook has set the standard. The Air Force created this document to outline what is required from an AoA given the complex ACAT I programs that the service manages, requiring a mandatory and explicit approach to issues of safety and risk. However, the level of detail required in AoAs for ACAT I programs, such as the F-35 Joint Strike Fighter or the M1 Abrams tank, is usually excessive when dealing with smaller, simpler, lower-budget programs.
The Air Force handbook directs that the AoA be complete before Milestone A. The JPEO-CBD’s revised AoA process treats the analysis as an evolution of information that will enable the project manager and the MDA to expedite programmatic decisions. This new strategy does not have a set number of required phases, for example. Instead, the initial phase of each AoA will include user feedback, an increased emphasis on threat and using screening criteria to target the most realistic options, along with prioritizing the measures of effectiveness (MoEs) and MoPs that are most relevant early in the program.
This approach envisions additional studies, or phases, as follow-ons to the initial AoA as needed to provide greater specificity of cost, schedule and performance and ensure that the program leadership chooses the alternative that will provide the best product to the warfighter.
The AoA significantly influences all aspects of an acquisition program and is a key input to other acquisition documents such as the acquisition strategy, systems engineering plan, test and evaluation master plan and particularly the capability development document (CDD). JPEO-CPD’s new process provides input to these documents before Milestone A. With the standard AoA process taking upward of 18 months on average, previously it had minimal influence on acquisition documents because they were developed before publication of the AoA results. By using this new, phased approach, JPEO-CPD’s goal is to provide critical information in intervals of six months or less, allowing for input to key documents from the start of the acquisition process.
LAYERS OF ANALYSIS
To make the AoA process more efficient and effective, JPEO-CBD’s streamlined approach offers program managers and the MDA the information they need to make informed decisions early in the program’s life cycle. The overall approach is to provide broad direction at the beginning of a program and add more detailed analysis as needed in later stages. (Image courtesy of Olivier Le Moal/iStock and the authors)
KEY FACTORS FOR ANALYSIS
One key component of the new AoA process is soliciting warfighter input immediately after the materiel development decision to assess specific functional needs that will influence design, provide a guide for prioritizing the focus of the program, and address the unique cross-cutting nature of joint solutions. While those executing the AoA have always requested this information from the combat developers, this new approach aims to add interviews directly with warfighters from all services to capture the wealth of knowledge our joint forces provide.
Early attempts at these interviews have shown some differences between what the combat developers report and what current users are reporting. In one instance, Mission Oriented Protective Posture (MOPP) gear exchange times reported in initial interviews based on actual experience were much higher than were published in doctrine. In another interview, an audience of over 100 end users asserted that they always had their protective armor on beneath their MOPP gear, whereas requirements reported on behalf of that service stated that troops would be wearing no protective armor while in MOPP gear.
Combining the high-level input with ground-level accounts will maximize the program’s relevance to the user and help prioritize MoEs and MoPs based on the concepts of operations (CONOPS) of each service or mission specialty. This also will give insight to whether a solution that overlaps all services is feasible, or if a solution needs to focus on a specific mission area. High-level analysis early in the AoA process thus can reduce the program’s timeline by focusing on the highest-priority next steps.
Threat is another key consideration in JPEO-CBD’s AoA streamlining. While threat has always been a factor influencing the requirements identified in the CDD, the new process increases the emphasis on incorporating threat into the initial AoA. When providing a joint capability, the threat risks vary from service to service, so understanding the threat that each service faces—in collaboration with our intelligence, requirements, combat development and warfighter communities—is critical in determining how a potential solution closes the capability gap for that service. Various services are willing to assume different levels of performance risk based on their concepts of employment and their CONOPS, so including threat as an independent variable along with cost, schedule and performance ensures that the AoA provides the necessary information to evaluate when and how to address it.
In a third area of the AoA that the JPEO-CBD has found adaptable, traditional AoAs have required the identification of additional technologies for the study report, independent of the requests for information (RFIs) that CBDP project managers release. Often, the RFI respondents are already known to the program, as only a few companies have the technology to fill that particular capability gap. This is because the CBDP is a niche field that relies on close partnerships with government science and technologies agencies, industry and academia.
These ongoing partnerships make the additional survey for new technologies redundant. Removing the requirement to identify additional potential solution sets outside of the RFI response allows for faster execution of the AoA, since we are only analyzing alternatives that have shown that they have the maturity and ability to provide a technology to the warfighter.
Although eliminating this research may be a risk, it is one that the JPEO-CBD has found acceptable for many programs, especially those in well-developed fields with known vendors. The JPEO would rather pay more immediate attention to determining technology readiness levels and gaining a rough order of magnitude for the anticipated life cycle cost estimates, which will allow for a more robust trade space analysis of the cost, schedule and performance for each of the potential materiel solutions, including a close look at the threat.
PICKING AND CHOOSING
The tailored AoA approach developed by JPEO-CBD identifies the most relevant factors for the most vigorous early analysis while still providing sufficient information to the MDA. While relevant for larger ACAT I programs, the full AoA process prescribed in the Air Force handbook is excessively detailed for smaller, simpler, lower-budget ACAT II and III programs. (Image courtesy of the authors)
CONCLUSION
The new AoA process is a big leap ahead in executing a major step in the acquisition life cycle. Executing an AoA in this way will reinvigorate the traditional approach by encouraging outside-the-box thinking to find better solutions for acquisition programs. Focusing on inputs most vital to program success can reduce the time spent collecting and analyzing information for an ACAT II or ACAT III program. In various memos, policies, white papers and other guidance on best practices, senior leaders across DOD continuously emphasize tailoring and streamlining our acquisition processes. JPEO-CBD has taken this to heart, not only tailoring the standards of the Air Force handbook but also collaboratively identifying areas to further improve the value of the AoA.
For more information on the tailored AoA approach, contact the authors at breena.j.berte.civ@mail.mil and marshall.l.kindred2.ctr@mail.mil. For more information on JPEO-CBD, go to www.jpeocbd.osd.mil.
MS. BREENA BERTÉ works as a process improvement specialist in the Continuous Performance Improvement Office at JPEO-CBD. She received an M.Ed. from Widener University, a Master of Social Service Management from Bryn Mawr College and a B.A. in English (writing) from La Salle University. She is Level II certified in program management and is a Lean Six Sigma Black Belt candidate. She is a member of the Army Acquisition Corps.
MR. MARSHALL KINDRED has been supporting the JPEO-CBD in the Continuous Performance Improvement Office for Data Systems Analysts Inc. since 2011. He has led and implemented acquisition process improvement initiatives, focusing on integrating the tenets of Better Buying Power across the portfolio. He holds a B.A. in economics from the University of Maryland, Baltimore County, and is a certified Lean Six Sigma Master Black Belt.
This article will be published in the January – March 2017 issue of Army AL&T Magazine.
Subscribe to Army AL&T News, the premier online news source for the Acquisition, Logistics, and Technology (AL&T) Workforce.
