USAOTC looks to streamline testing and get new products to Soldiers in less time and at a lower cost
By CPT Lev L. Mazères
Every test effort faces its own set of challenges—the Air Soldier System (Air SS) included—but through creative problem-solving, a team from the U.S. Army Operational Test Command (USAOTC) delivered a thorough operational test (OT) that has positioned the evaluation effort for success as the Air SS Test and Evaluation Working Integrated Product Team prepares for an initial operational test (IOT) in late summer 2015.
USAOTC executed a series of test events over the course of two months last summer, in support of Product Manager (PdM) Air Warrior’s Air SS milestone C low-rate initial production decision. During this period, personnel from the U.S. Army Evaluation Center (USAEC), Army Aviation Concepts and Requirements Directorate, PdM Air Warrior, the Army Survival, Escape, Resistance, and Evasion (SERE) School at Fort Rucker, AL, and the 10th Combat Aviation Brigade (CAB) supported operational testers from the USAOTC Aviation Test Directorate. The goal of these events was to test the effectiveness, suitability and survivability of the Air SS in an operational environment.
While successful, the experience revealed the need to re-examine how an enterprise plans, resources and executes OTs—particularly with regard to programs in acquisition category (ACAT) II and below that are not on the Office of the Secretary of Defense (OSD) Test and Evaluation (T&E) Oversight List—in a new, more fiscally constrained environment. ACAT I and T&E Oversight List programs generally receive test unit priority.
Air SS is an integrated, modular, mission-tailorable aviation life support equipment and protective ensemble for aircrew Soldiers. Intended for crews of select manned aircraft in maneuver, maneuver support and maneuver sustainment roles involved in full-spectrum operations, the new system promises to provide existing capabilities while reducing weight and bulk and increasing situational awareness. As the Army’s only independent OT organization, USAOTC ensures such enhancements are adequately tested and worthy of being put in the hands of the Soldier. As is typical of most ACAT I and II programs, USAOTC used the Test Schedule and Review Committee (TSARC), in accordance with regulatory requirements, to source OT players and units from U.S. Army Forces Command to support operational testing.
STAYING ON COURSE
The chart depicts the paths of communication between test officers and unit action officers and their respective hierarchies. Greater collaboration and communication within the operational community can improve the effectiveness of operational testing while reducing the cost. (Image courtesy of CPT Lev Mazères)
PLANNING COMES UNDONE
Originally planned as a combined developmental test (DT) and OT event, with the intent to use the same test players for both the DT and follow-on initial OT (IOT) effort, TSARC leadership notified USAOTC in late 2013 to expect the 25th CAB to provide the test personnel and equipment necessary to execute the consecutive test events. Unfortunately, additional and competing higher priority taskings resulted in 25th CAB requesting a reclama (reconsideration of a decision) from the TSARC tasking in early 2014. As a result, the TSARC assigned the 10th CAB to take part in testing.
To meet program timeline constraints, PdM Air Warrior opted to conduct a separate DT event at the Redstone Test Center, located at Redstone Arsenal, AL, while shifting the IOT to the summer. System maturation issues resulted in a joint agreement between Army Test and Evaluation Command and Program Executive Office for Soldier to reduce the OT from an IOT to a limited user test and subsequently to a customer test (CT). The CT resulted in the test timeframe being cut in half and the loss of range reservations at Fort Drum, NY, which meant that the M4 and M9 weapon ranges and SERE lanes could not be conducted.
Challenged to collect adequate data to meet USAEC evaluator needs, but with less time and fewer resources, the situation brought to the surface previously overlooked opportunities to collect relevant feedback. Leveraging previous working relationships with personnel at the SERE School, while ensuring a simultaneous request through the TSARC process, kept all appropriate stakeholders in the loop.
The joint effort resulted in an extensive test of the Air SS 72-hour survival gear during a SERE situation, as well as testing the performance of the combat basic ensemble configuration during various stress weapon and break contact firing ranges. All testing occurred at minimal cost and within the timeframe for inclusion into the USAEC OT agency milestone assessment report. The test also served as a great learning event for test personnel on how to better prepare for the following year’s IOT, when the testing will have to be repeated with a test unit on their own installation. Additionally, the test shed light on shortcomings in policies, techniques and procedures.
These shortcomings became apparent during the Air SS CT when the one of the 10th CAB company commanders approached the test team with the offer to test components of the Air SS during already-slated aircrew progression training, for which flight hours were already paid. However, the short notice precluded the coordination required to ensure that airworthiness and safety releases were in place, PdM Air Warrior could properly field the required gear, and USAOTC could set up the right data collection methods to ensure data collected would be valid.
Ultimately, the original CT schedule was maintained, and the proposed testing could not be conducted. Issues that would have been identified earlier during the opportunity testing surfaced, and as a result, caused test disruptions that could have otherwise been mitigated or eliminated.
ASSESSING THE PRODUCT
A test player, left, a SERE School instructor and MAJ Howard Swanson, USAOTC test officer, conduct a stress fire range test while wearing the Air SS ensemble at the SERE School at Fort Rucker, AL, on Sept. 18, 2014. Tests of the Air SS highlighted the need to re-examine how an enterprise plans, resources and executes OTs in a more fiscally constrained environment. (Photo by Bradley Davis, U.S. Army Research Laboratory)
TEST, FIX, TEST
A review of the last 13 years of operational testing through the archival lens of Army AL&T magazine and Program Manager (PM) Online shows how the challenges of planning, resourcing and executing OTs have evolved—or rather, have not. At their core, the same challenges from 2002 exist today in 2015. The struggle to ensure requirements are current, adequate and attainable continues. Often, sufficient testing is not conducted and adequate time to test-fix-test is not observed, partially because the processes for doing so do not provide the flexibility required to overcome the challenges faced by PMs and PdMs when developing materiel solutions.
By the time test officers are finally assigned test units, a great deal of time and resources have been expended developing working relationships with the unit leadership and personnel during the critical last months leading up to test execution. The result is often that Soldiers and unit leadership view operational testing as a tasking burden rather than an opportunity to train and to influence the Army’s acquisition decision-making process.
One potential solution is to create a program where unit leadership, starting at the company level, can “opt into” operational testing. Essentially, that would consist of a calendar of upcoming test events on a secure portal that would list upcoming OTs and associated timeframes and requirements. In exchange for working training into the framework of OT, the operational unit would receive additional funding and resources to enhance their training program, as well as an opportunity to shape the materiel solutions of the future. The program would also serve as a forum to negotiate trade space for when testing occurred and what personnel and equipment were required in real time. In its current form, the TSARC process cannot provide this level of flexibility in an acceptable window of time.
The TSARC process by no means would be replaced or sidestepped using this method; rather, the opposite would occur. By engaging in these discussions and negotiations at the action officer level, the TSARC process would in fact be enhanced rather than marginalized.
Requests for resources would be better refined and more realistic, while the leadership of potential units being tasked would be better informed by their subordinate leaders on whether a tasking could be supported. Even if a unit ultimately could not support a tasking because of higher echelon mission requirements, at least the time to receive that decision would be reduced, allowing another unit to consider the testing requirement.
Another potential benefit is the creation of greater transparency among test efforts and ongoing programs. The forum would give leaders across the T&E community the ability to view upcoming test events and combine efforts where applicable. By potentially combining two or more test events, T&E members could increase efficiencies, reduce overhead requirements and increase cost-sharing while decreasing the financial impact to PdM shops. In the USAOTC Aviation Test Directorate, flight hours routinely are one of the greatest cost drivers for test events. Any opportunity to sync test schedules results in instant savings while reducing TSARC’s need to task multiple units.
MAKING IT POTABLE A Soldier purifies water using the 72-hour survival kit included in the Air SS during testing at the SERE School at Fort Rucker, AL, on Sept.17, 2014. The tests were conducted by USAOTC in support of PdM Air Warrior’s Air SS milestone C low-rate initial production decision and also shed light on shortcomings in test policies, techniques and procedures. (Photo by CPT Lev Mazères)
CONCLUSION
This potential program is not necessarily a “one size fits all” solution. ACAT I programs and other programs on the OSD T&E Oversight List (which represented less than half of the OTs executed by USAOTC in FY14) are likely not appropriate candidates for this potential program, given the attention they receive and the complexities of their tests. In addition, certain types of equipment may not be as well suited depending on the stringency of the requirements for the time period, personnel and the type of equipment. Again, this proposed program is more in the realm of knowledge sharing, encouraging crosstalk and an overall enhancement of the TSARC process. Just as the Air SS test team ensured all regulatory and procedural requirements were met to conduct the SERE excursion for the Air SS CT, the same would have to be done for any future tests.
By providing a mechanism to increase collaboration with and buy-in from the operational community, the effectiveness of operational testing can be increased while reducing the cost. If the cost is sufficiently reduced, the frequency of and opportunities for operational testing would potentially increase. Not only does this ensure better refined materiel solutions, but more importantly, better refined requirements. With input from the combat developers, materiel developers, testers and evaluators, this process could work. This process would allow for the resolution of issues identified over the last 13 years of working in a post-9/11 world while simultaneously moving forward in a more fiscally constrained environment, thereby ensuring the American Soldier is equipped with the best solutions financially possible.
For more information, contact the author at lev.l.mazeres.mil@mail.mil. Acquisition officers interested in working in an environment that seeks out critical thinkers and problem solvers should contact their branch manager about future assignment opportunities at USAOTC.
CPT LEV L. MAZÈRES is an operational test officer with USAOTC’s Aviation Test Directorate. He holds an MBA in business management from the Owen Graduate School of Management at Vanderbilt University and a B.S. in economics from the United States Military Academy at West Point. He is Level II certified in program management.
This article was originally published in the April – June 2015 issue of Army AL&T magazine.
