Tested by austerity

Forced by budget constraints to reduce costs, ATEC is applying a comprehensive strategy across developmental and operational testing, incorporating Soldier participation and feedback early in the process for better data.

By Col. Stephen Lutsky, Ms. Alicia Thomas and Mr. Michael Dillen

Just as the equipment of yesterday will not meet the needs of tomorrow, nor will the traditional way of conducting the test and evaluation (T&E) mission. As the Army continues to focus on modernization and postures to meet the needs of Force 2025, the U.S. Army Test and Evaluation Command (ATEC), the Army’s independent T&E component, must be ready to test and evaluate the systems that will provide America’s Soldiers with the tactical edge they need to prevent, shape and win conflicts in the complex world in which we operate.

To remain effective in an era of budgetary austerity, the T&E community must explore better ways to test, collect meaningful data and evaluate systems with less money and fewer personnel.

To overcome this austere reality, ATEC has been looking proactively at its processes and practices to find innovative ways to improve T&E and reduce costs. The command’s current efforts include encouraging customers to consider testing earlier in the acquisition process, fully leveraging historical test data for analysis and reducing costs by integrating developmental testing (DT) and operational testing (OT). For example, the long-standing practice of conducting sequential DT and OT events costs more than the Army can afford. ATEC has developed a new methodology known as the Comprehensive Test Strategy (CTS) that, if implemented in FY18, would enable ATEC to test more effectively, collect meaningful data and evaluate systems while saving time and money. CTS looks across all the developmental phases of the acquisition cycle for ways to use T&E resources—time, money and people—more efficiently while supporting all major milestones and fielding decisions to improve warfighting technologies.

This continuous improvement of T&E processes and attention to a system’s entire acquisition life cycle, from technology development through sustainment, is increasingly essential. In this way, ATEC can identify efficiencies, achieve test-cost savings, enhance data products and reduce T&E timelines to support fact-based decision-making. The ultimate goal is to ensure that Soldiers’ equipment is effective and suitable and will survive in even the toughest operational environments to which Soldiers deploy.

STRATEGIC PRINCIPLES
In compliance with current acquisition and T&E policy, CTS examines all aspects of the developmental phases of the acquisition life cycle in search of T&E efficiencies while supporting all major milestones and fielding decisions.

The CTS rests on four key tenets:

• Actively using Soldiers in DT.
• Leveraging DT data to support OT evaluations.
• Using maneuver force home station training for data collection.
• Using combat training center (CTC) rotations to evaluate operational impacts.

Although the individual tenets are not new, combining them into one strategy with greater emphasis on the use of alternate OT methods has not yet been fully explored.

USING SOLDIERS IN DT
Leveraging Soldier-user assessments during DT helps uncover early system design or functional issues. Program managers can leverage data from those early assessments to make decisions on system technology development in advance of production and fielding. Leveraging the data early mitigates the impact of system design changes on a program’s cost and schedule. To ensure Soldier availability and competence in testing, ATEC has re-energized its training mission so that Soldiers may perform multiple roles in the field: Soldier, operator, maintainer, tester and evaluator (SOMTE). However, as the Army continues to reduce its force structure, the availability of SOMTEs for participation in testing is uncertain.

Soldier feedback is invaluable and influences the program manager’s decision-making during early phases of acquisition and T&E. For example, in May 2013 during an ATEC test on a grenade ammunition pouch in partnership with the Rapid Equipping Force, Soldiers navigated an obstacle course during which grenades fell out of their ammunition pouches. ATEC provided data from the user assessment to the program manager, which resulted in the redesign of the pouch. Early Soldier involvement and data collection allowed for system improvements without which Soldier safety or readiness might have suffered.

The benefits of using Soldiers in DT extend beyond reducing T&E costs, though. Perhaps equally notable is gaining Soldiers’ understanding of the environment in which they will be using a system and how other Soldiers will interact with it. Capturing Soldier feedback is critical to the Army’s efficient development of effective, suitable and survivable systems.

LEVERAGING DT DATA
Soldier participation in operationally realistic scenarios throughout the DT environment has yet to be practiced, but it clearly can reduce the time and cost of OT later in the acquisition cycle by enabling discovery of potential opportunities for system improvement while the system is in development.

One key to the success of this approach is limiting the test scope to only what is necessary to accomplish the evaluation. This means removing all test facets of the OT that have no impact on the specific operational area of focus and accepting that this focus on operational realism will not negate test results. Operational data obtained during this process can be used to make improvements early in the program’s life cycle. Using Soldiers in an operationally realistic DT environment can also reduce the scope of a subsequent OT event because there is no need to gather specific operational data again.

One example of leveraging DT to support the OT evaluation is in the Joint Assault Bridge T&E master plan. ATEC will be able to create an operationally realistic environment during the production-­qualification test by adding Soldiers and realistic scenarios to the last portions of the launch-and-retrieve cycles and the reliability, availability and maintainability (RAM) miles of the operational-mode summary and mission profile testing.

In the past, civilians would conduct the total RAM event without adding Soldiers and realistic scenarios to the event. Thus the event was missing Soldier-conducted RAM mileage, and RAM miles could not be reduced in the actual OT event, which ultimately meant higher cost. The environment will be further improved by having the Soldiers perform their duties as vehicle crew members and execute their primary mission using the Joint Assault Bridge. Avoiding the use of an opposing force saves valuable resources without affecting the reliability assessment. We expect that this combined DT-OT strategy will reduce the duration of the initial OT, from six to five weeks.

HOME STATION TRAINING
With fewer deployments, priority for use of installation training areas is going to mission-essential task list (METL) validation and preparation for capstone events at CTCs, which are expecting organizational units to arrive prepared to execute their METL without requiring training. By identifying platoon- through battalion-level units that are preparing, ATEC can integrate system evaluations with scheduled maneuver home station training exercises to capitalize on opportunities for Soldier feedback.

Historically, an OT is a stand-alone event, forcing the program manager to provide funding for test assets, blue force, opposing force, support and sustainment costs. Using home station training for data collection benefits the program manager by reducing resource requirements to only the test assets, with the organization’s maneuver funds supporting personnel and equipment not provided by the program manager.

An additional benefit of using home station training for data collection is the increased opportunity for Soldiers to become familiar with the test asset.

CTC ROTATIONS AND OPERATIONAL IMPACT
Like NTC rotations at Fort Irwin, California, Army CTCs are DA-resourced, large-scale, realistic operational events that offer a rich environment for conducting program T&E. CTC formations are normally at the battalion or brigade level, which provide a better sample size for evaluation without additional cost to the program manager. School-trained and -certified observer controllers are present to provide feedback on mission success and data collection, reducing the customary number of data collectors.

By using the unit that participated in the home station training exercise, ATEC can eliminate the need to repeat new equipment training. In addition, the unit transports test assets to a CTC with its organic equipment, providing critical transportability data that can be used to support the system evaluation.

Evaluating operational effectiveness and mission performance is the primary objective during a CTC event. ATEC uses the data collected to complement data from the DT and the home station events, resulting in a comprehensive evaluation report to inform Army decision-makers.

Successful T&E requires senior leader buy-in and early cross-organizational planning. ATEC prepares a memorandum of agreement up to two years in advance, updating it periodically to reflect any changes in personnel or requirements intended to reduce risk. The objective is to structure the CTC rotation in such a way as to satisfy both the training objectives of the unit commander and the evaluation data needs of ATEC.

CONCLUSION
ATEC plans to implement the CTS at the start of FY18. Depending upon the selected unit, however, cost savings for the 2018 Stryker Engineering Change Proposal (ECP), using the provided rough-order-of-magnitude (ROM) cost estimate, are projected to possibly exceed $3.4 million, for a 37 percent cost reduction. Cost avoidance for the 2020 Abrams and Bradley ECP is projected from the provided ROM cost estimate to be more than $4.03 million, for a 22 percent cost reduction.

The CTS is only one example of how the T&E community is rapidly adjusting, adapting and responding to a constrained environment while remaining compliant with acquisition and T&E policies. The CTS is a solid plan for capitalizing on early Soldier interaction and feedback in data collection and the consolidation of T&E requirements within an existing training framework.

The benefits of CTS to the Army will include eliminating duplicative testing, optimizing the use of dwindling resources available to the test community, reducing T&E time and costs, earlier fielding of equipment with a reduced burden on training resources and the ability to provide better-evaluated equipment to the maneuver force through early identification of operational problems.

---

For more information, contact the authors at stephen.j.lutsky@afghan.swa.army.mil, alicia.j.thomas2.civ@mail.mil and michael.p.dillen.civ@mail.mil; or visit http://www.atec.army.mil/.

COL. STEPHEN LUTSKY is director of the U.S. Army Evaluation Center’s Mounted Systems Evaluation Directorate at Aberdeen Proving Ground (APG), Maryland. He holds an M.A. in administration from Central Michigan University and a B.S. in microbiology from Lock Haven University.

MS. ALICIA THOMAS is executive officer of the U.S. Army Aberdeen Test Center at APG. She holds an M.S. in program management from the Naval Postgraduate School and a B.S. in textile engineering technology from Southern Polytechnic University. She is Level III certified in test and evaluation.

MR. MICHAEL DILLEN is chief of the Stryker Brigade Combat Team Division, Mounted Systems Evaluation Directorate at APG. He holds an M.S. in technology management from University of Maryland University College and a B.S. in agricultural engineering from Pennsylvania State University. He is Level III certified in test and evaluation, a member of the Army Acquisition Corps and a Lean Six Sigma Black Belt.

This article was originally published in the October – December 2015 issue of Army AL&T magazine.

Subscribe to Army AL&T News, the premier online news source for the Acquisition, Logistics, and Technology (AL&T) Workforce. Subscribe

---