Skimping on test and evaluation in defense acquisition is not a recipe for effective reform, it’s a recipe for disaster. by Robert F. Mortlock, Ph.D., Col., USA (Ret.) From 2009 to 2015, the Army conducted the most comprehensive testing of combat uniform camouflage in history, leading up to its selection of the Operational Camouflage Pattern on the Army combat uniform. The testing included a combination of controlled testing—relying on photo-simulation techniques (Soldiers viewing photos of camouflage uniforms on computer screens) that used modeling and simulation—and Soldier field testing. The testing measured the performance of camouflage patterns based on probability of detection and blending, with performance scores depending primarily on distance, environmental background, movement, lighting and the actual camouflage pattern. The goal of the program was to select the best-performing camouflage pattern. The testing included a combination of developmental test and evaluation—highly controlled testing in an environment that typically is not combat-realistic, in this case the testing that relied on photo-simulation techniques—and operational test and evaluation, in this case the Soldier field testing, which takes place in a combat-realistic environment. The test and evaluation (T&E) strategy was considered a major shift in approach because it weighted developmental test and evaluation results more heavily than operational test and evaluation results, the reverse of a typical acquisition program. The developmental test and evaluation was statistically robust and tightly controlled the distance, background, lighting and movement variables, such that changes to detection and blending scores were attributed primarily to differences in the camouflage patterns. Yet senior Army leaders consistently placed more credibility on the field testing than the photo-simulation results. Each time the program came forward for a decision, leadership demanded more expensive and time-consuming field testing, unnecessarily driving up program costs and delaying schedules. The scope of the T&E effort grew not because it was necessary but because senior leaders remained risk-averse and unwilling to make a decision. The bottom line is that they never fully appreciated the T&E paradigm shift and never really trusted the project manager. That project manager was me, and the experience highlighted the importance of clearly understanding and synchronizing test objectives within the acquisition strategy, especially as T&E is a target for savings and efficiencies in current efforts to reform acquisition. It is a fool’s errand to try to chase the “right” amount of T&E cost for a particular acquisition effort. In each case, the T&E strategy must be optimal to provide the knowledge that decision-makers need for the most informed decision possible. An extensive, comprehensive T&E program is a big part of building trust with warfighters, and acquisition professionals cannot afford to sacrifice that trust. The last thing the defense acquisition profession wants is for a Soldier, Airman, Marine or Sailor in combat to question whether their equipment works as required. Warfighters must be focused on their mission and must know implicitly that their equipment works. Rigorous T&E in defense acquisition projects provides information and knowledge to decision-makers and warfighters about how the equipment will work in a combat environment, and provides confidence that the equipment is safe to operate and works as intended. Efforts to marginalize the importance of T&E within acquisition, along with the perception that acquisition programs have bloated, wasteful and redundant T&E activities, erode the integrity of defense acquisition as an institution—further compromising legitimate reform initiatives focused on process improvements. The muzzle blast from a M2 .50 caliber machine gun tears through camouflage as gunners from the 173rd Airborne Brigade Support Battalion conduct base defense training in March in Slovenia. Though the search for a new camouflage pattern was not a major defense acquisition program, it attracted a lot of high-profile interest from Army leadership and Congress, as well as from the Soldiers who would be wearing the new uniform. (Photo by Lt. Col. John Hall, 173rd Airborne Brigade) PUTTING COSTS INTO PERSPECTIVE A comprehensive and rigorous T&E program comes at a significant cost for any category of acquisition. A meaningful measure of the appropriate scope, value and return on investment of a T&E effort might be the T&E costs as a percentage of total life cycle costs—the sum of a program’s research, development, testing and engineering (RDT&E), procurement, and operations and support costs. The following are two examples of using total life cycle costs to measure the value of T&E: The Joint Common Missile (JCM) program—the predecessor to today’s Joint Air to Ground Missile program—was a major defense acquisition program under development in the mid-2000s. It sought to develop a missile that could be fired from Apaches, Cobras and Super Hornets, replacing HELLFIRE, Maverick and aviation tube-launched, optically tracked, wire-guided missiles. From the JCM joint cost position, the JCM total life cycle cost estimate was $7.275 billion ($970 million in RDT&E, $6.023 billion in procurement, $267 million in operations and support) with a T&E effort costing $293 million, or 4 percent of the total life cycle costs. The Ground Combat Vehicle (GCV) program sprang from the ashes of the Future Combat Systems Manned Ground Vehicle program and was meant to replace the Bradley Fighting Vehicle. From the GCV program office estimate, the GCV’s total life cycle cost estimate was $83.982 billion ($8.195 billion in RDT&E, $38.952 billion in procurement, $36.835 billion in operations and support) with a T&E effort costing $417 million, or 0.5 percent of the total life cycle cost estimate. Which program managed its test budget best? It’s not as simple as looking at which spent the least amount of money. Different kinds of equipment require different evaluations: testing a missile is always more expensive than testing a vehicle, since the missile is destroyed during the test process and the program goes through many of them. So simply imposing arbitrary caps on dollars spent or percent of total budget allocated to testing is not a smart way to lower acquisition costs, the author explains. (SOURCE: the author) Did the T&E effort for JCM provide a better value than that for GCV? One might conclude that the GCV program’s T&E was a better investment, at less than 1 percent of program costs compared with 4 percent for the JCM. However, the numbers alone don’t provide a complete picture of whether either T&E effort was appropriate in scope or a good return on investment. The T&E program for a missile inherently involves more expensive, destructive testing—the object being tested is generally destroyed during the test—and requires a great number of missile firings for statistical confidence. Additionally, the total life cycle cost profiles for a missile program and a vehicle acquisition are drastically different because the missile program requires far less in operations and support costs—the missiles are simply stored until fired and then destroyed. Returning to the camouflage T&E effort, which represented 0.167 percent of total program life cycle costs, did the Army get its money’s worth for $10 million spent over six years of testing camouflage patterns? The cost numbers don’t account for the other benefits of improving camouflage, including reduced casualties, increased mission effectiveness, greater force protection and safety and improved Soldier confidence. This program was not a major defense acquisition program, but it did have considerable interest from Soldiers, senior Army leaders and Congress. The proliferation of camouflage patterns on uniforms had garnered a lot of attention from Congress because of Government Accountability Office (GAO) concerns that the services were wasting money on duplicative research, development and testing of camouflage uniforms. With respect to combat camouflage uniforms specifically, the September 2012 GAO report (GAO-12-707), “Warfighter Support: DOD Should Improve Development of Camouflage Uniforms and Enhance Collaboration Among the Services,” highlighted the fragmented approach taken by the Army, Navy, Air Force and Marine Corps. The report stressed the potential to save tens of millions of dollars in development, testing, logistics and inventory control for combat uniforms. For the Army in particular, the decision to change camouflage patterns on uniforms and equipment affects an approximately $6 billion inventory of uniforms and equipment. So, how much in T&E costs is enough to properly support an acquisition? The best answer for a seasoned acquisition professional is, “It depends.” Photo simulation—where a camouflage pattern is viewed and manipulated on a computer—pattern selection has statistically significant data and controls distance, movement, background and brightness. In short, it’s a good way to select a camouflage pattern that will protect Soldiers. But senior leadership did not trust the lab tests, which highlights the importance of incorporating a robust, well-designed test plan from the very beginning of an acquisition. (SOURCE: The author) T&E IN ACQUISITION There is no silver bullet or secret untapped method to fix T&E within defense acquisition because radical change is not needed. Incremental improvements can and do work, and the T&E processes within defense acquisition already use the best practices in commercial industry. In my experience, T&E efforts increase in scope not because of the project manager or testing agencies but because senior leaders don’t understand either the program or the T&E strategy, and thus demand more testing before making a decision. Recent Army acquisition reform and directives targeting the reduction of T&E do not improve acquisition outcomes. In fact, they may increase risk and potentially jeopardize Soldiers’ safety. The Army doesn’t need additional directives stating goals to “streamline T&E and minimize redundant testing.” That’s inherent in the project manager charter to develop a strategy that meets the program’s objectives by balancing cost, schedule, performance and risk. The testing itself is all about gaining knowledge, and the independence of the T&E agencies is necessary so that command relationships don’t overshadow important technical perspectives. Comprehensive testing reduces risk and provides indications that there may be issues to address—perhaps requiring more testing. Cost overruns and schedule slips are the result of immature technology, poor cost estimating, integration and interoperability challenges, and Murphy’s Law. The business of acquisition involves risk, and it takes time to work through the multitude of issues that arise in the development of new capabilities. In the T&E arena, the best use of time and energy for DOD senior leaders and decision authorities is to become knowledgeable about the acquisition effort in question and about acquisition T&E best practices in general. Soldiers assigned to 2nd Armored Brigade Combat Team, 1st Infantry Division, at Fort Riley, Kansas, move into nearby woods during a platoon combined arms live-fire exercise at Presidenski Range in Trzebian, Poland, March 26. The author, who was program manager of the search for a new camouflage pattern, explains that there are two ways to go wrong with test and evaluation: Cutting back on testing too aggressively, in pursuit of cost savings, puts the program at risk, while demanding more expensive field testing in the mistaken belief that it’s more accurate drives the budget up. (U.S. Army photo by Spc. Dustin D. Biven, 22nd Mobile Public Affairs Detachment) CONCLUSION The perception that the testing community, particularly operational test agencies, are causing program cost overruns and schedule slips is misguided and counterproductive—undermining the credibility of a dedicated acquisition T&E workforce. The nature and extent of T&E is one of the main areas of policy emphasis in defense acquisition reform initiatives. On the one hand, senior leaders demand T&E efficiencies to fix the perceived high costs and long schedules of T&E efforts. On the other, decision-makers typically demand more, higher-quality T&E to satisfy their desires for the most complete information on a program’s progress and risks as they prepare to make the most informed decisions possible. Project managers must balance those demands with cost and schedule constraints while ensuring compliance with applicable statutes. Early engagement and collaboration with all stakeholders in T&E planning, execution and analysis can help allay criticisms of the T&E program’s adequacy. Thus, the best answer to the question, “How much T&E is enough?” is one that recognizes and accommodates the interests of all T&E stakeholders. The appropriate T&E acquisition reform initiative might be to empower project managers and then trust them to build appropriate T&E strategies—and, of course, hold project managers accountable if the program fails to meet cost, schedule and performance goals. Above all, the most appropriate metric for T&E efforts is to measure whether those efforts provided enough information about the critical technical parameters, critical operational issues, effectiveness and operational suitability for an informed decision by senior leaders. More oversight, new commands and additional reporting requirements are not effective acquisition T&E reform initiatives. For more information, contact the author at rfmortlo@nps.edu. ROBERT F. MORTLOCK, PH.D., COL., USA (Ret.), managed defense systems development and acquisition efforts for the last 15 of his 27 years in the U.S. Army, culminating in his assignment as the project manager for Soldier protection and individual equipment in the Program Executive Office for Soldier. He retired in September 2015 and is now a lecturer for defense acquisition and program management in the Graduate School of Business and Public Policy at the Naval Postgraduate School in Monterey, California. He holds a Ph.D. in chemical engineering from the University of California, Berkeley, an MBA from Webster University, an M.S. in national resource strategy from the Industrial College of the Armed Forces and a B.S. in chemical engineering from Lehigh University. He is also a recent graduate of the Post-Doctoral Bridge Program of the University of Florida’s Hough Graduate School of Business, with a management specialization. Related Links: GAO Sept 2012 report This article will be published in the July – September 2018 issue of Army AL&T magazine. Subscribe to Army AL&T News, the premier online news source for the Acquisition, Logistics, and Technology (AL&T) Workforce. Related posts: DACM Corner: AcqDemo: Rewarding Excellence DACM Corner: Growing the Acquisition Workforce New Product Office Focuses on Armored Multi-Purpose Vehicle OSD Unmanned Warfare Directorate Develops Online Unmanned Catalog Database

Mastering acquisition The DACM Office and the Naval Postgraduate School realign degree programs to boost technical education of civilians and officers. Earlier this year, I was in beautiful Monterey, California, to help put the final touches to a new effort for the Army Acquisition Workforce at the Naval Postgraduate School (NPS). Starting this fall, the civilians that the U.S. Army Acquisition Support Center’s Director for Acquisition Career Management (DACM) Office sends to NPS will be pursuing an M.S. in systems and program management. Officers will pursue an M.S. in systems engineering management starting this summer. Previously, the professionals we sent to NPS were seeking an MBA. The new programs that we’re sending our students through—Curriculum 522 for officers and Curriculum 722 for civilians—will focus on getting them greater exposure and training across multiple career fields with added emphasis on critical thinking across domains; however, the curriculum has much of the same content as the previous master’s program. In addition to their master’s degree, civilians in a distance learning program will earn training equivalent to Level III Defense Acquisition Workforce Improvement Act training in program management and in engineering, Level II in test and evaluation and at least Level I in contracting. Any time you can get training and education simultaneously, you’ve got a greater opportunity to employ them in the experiential environment. Sailors approach Herrmann Hall. NPS’ new curricula in systems and program management and in systems engineering management for students from the Army Acquisition Workforce support its overall mission as well as the specific needs of Army acquisition professionals to understand the technical aspects of their jobs. (Photo courtesy of NPS) Eligible resident officers completing the 522 degree program also will obtain their Joint Professional Military Education and Level III training in program management, engineering and contracting, plus Level II training in test and evaluation. The impetus behind the change is Lt. Gen. Paul A. Ostrowski’s belief that if you want to be a great program manager or a great contracting officer, you have to thoroughly understand the technical acumen that’s needed for both parts of our business. If you’re running a program and haven’t been trained in contracting, when your contracting officer starts reeling off contracting jargon, you’re just going to agree to whatever they say and maybe not make the best choices for a program. And if you’re on the contracting side and you don’t have a solid understanding of engineering and program management, a contractor can overwhelm you with details and you might not make the best decisions for the government. The push for the change began with Lt. Gen. Michael E. Williamson, Ostrowski’s predecessor as the principal military deputy to the assistant secretary of the Army for acquisition, logistics and technology and director of the Army Acquisition Corps. This change is about exposing more of our professional workforce to greater technical education. The key word here is “technical.” We’re very confident that throughout their careers, our civilians and our officers have a variety of opportunities to get leadership and management training and experience. But technical training is probably the most difficult to achieve. And so we’re trying to focus a lot of our education programs, NPS being one of them, to increase the number of people that have a greater exposure to technical content. A key component in this change is NPS. The true power that NPS can provide, and needs to focus on providing in the future, is its operational relevance. When you get an MBA or an M.S. in systems and program management, an M.S. in systems engineering management or any other graduate degree from NPS, you should have been exposed to enough experienced operators to flavor it so that a lot of what you’re getting is real-world practical experience that you can apply to DOD. We can send anybody anywhere for a systems engineering degree or an MBA. There are great institutions all over the world. But there are very few accredited institutions that can provide real-world practical operational experience from practitioners who have walked in the shoes you will be walking in as an acquisition professional. The faculty understands the pitfalls that you will face. They’ve worked inside the DOD system. Very few schools of higher education and learning can offer that. That’s the power that NPS has, and that’s the focus it needs to maintain. The faculty’s operational relevance is what sets NPS apart and makes it unique. If NPS strays from that, then it’s forced to compete with every other college and university in America. If it loses track of that niche, NPS will be of less value to us as the sponsor of these programs. Craig A. Spisak, left, the Army DACM, and Professor John T. Dillard, Col., USA (Ret.), NPS senior lecturer in systems acquisition management and technical representative for the new curricula, hold the memorandum of agreement signed by Lt. Gen. Paul A. Ostrowski and NPS President Ronald A. Route, Vice Adm., USN (Ret.), on May 18 at the Pentagon. The memorandum cements a partnership to provide relevant education to the Army’s military and civilian acquisition workforce. (Photo courtesy of John T. Dillard) We’ve had a longstanding relationship with NPS, and we look forward to continuing that through many, many years. NPS will face, as any college or university does, the difficulty of adjusting its curriculum to changing times. But the fact that NPS is a DOD institution allows it to translate our real-world needs into viable education programs. In the future we may have another emerging requirement for our students who go there to understand at the graduate level. And we will need to be able to transmit that requirement to NPS as the sponsor of the programs and have it adapt and adjust the curriculum in real time. We need that kind of responsiveness and partnership. They’re the professionals at educating people. We have to see the world today and into the future to determine what skill sets should be embedded in the NPS curriculum. The 411 on 522 and 722 at NPS Curriculum 522 Systems Engineering Management System Acquisition Audience: Army acquisition officers. Description: An interdisciplinary program combining systems engineering with acquisition management knowledge and skills. Intended to broaden the technical competence of officers with nontechnical backgrounds so that they can manage and lead acquisition programs for complex combat systems. Students learn the systems engineering process, from establishing system requirements through test and evaluation; and how to manage, schedule and budget programs and work with DOD suppliers through contracts to meet program obligations. Requirements: A baccalaureate degree with above-average grades is required. Completion of at least two semesters of college algebra or trigonometry is considered the minimum mathematical preparation. Program start dates: January and July. Program length: Six quarters. Training method: Resident. Degree: Master of Science in Systems Engineering Management. Certification training included: Joint Professional Military Education; Defense Acquisition University equivalencies for Defense Acquisition Workforce Improvement Act training for Level III in program management, Level III in engineering, Level III in contracting and Level II in test and evaluation.   Curriculum 722 Systems Engineering Management Systems and Program Management Audience: Army acquisition civilians. Description: An interdisciplinary program combining systems engineering with program management knowledge and skills. Intended to broaden the technical capabilities of acquisition workforce members with nontechnical backgrounds so that they can successfully manage and lead programs or projects in support of the defense acquisition system. Students learn the systems engineering process, from establishing system requirements through test and evaluation; and how to manage, schedule and budget programs and work with DOD suppliers through contracts to meet program obligations. Requirements for entry: Candidates for the program must have a baccalaureate degree. This program is available only through the Army DACM Office. Program start date: September. Program length: Eight quarters. Training method: Distance learning. Degree: Master of Science in Systems and Program Management. Certification training included (for civilian students already Level II certified in program management): Defense Acquisition University equivalencies for Defense Acquisition Workforce Improvement Act training for Level III in program management, Level III in engineering, Level II in test and evaluation, Level II in production, quality and manufacturing, and Level I in contracting. This article will be published in the July – September 2018 issue of Army AL&T magazine. Subscribe to Army AL&T News, the premier online news source for the Acquisition, Logistics, and Technology (AL&T) Workforce. Related posts: Advocate for Innovation Building A Better Mirror A different way of doing business Career Navigator: Where is the What?

RCO joins SOCOM in early assessments of new or emerging technology from defense industry, tech startups and academia. by Ms. Nancy Jones-Bonbrest On a rainy afternoon in March, technology developers gathered to show their latest position, navigation and timing (PNT) solutions. There were no traditional marketing brochures, trade show booths or giveaway trinkets. Instead, these developers—from big industry, tech startups and academia alike—were there to demonstrate emerging capabilities and get feedback directly from the operators who may one day use the technology on the battlefield if their GPS is ever jammed, tricked or dropped. To carry out the event, organizers found a former Indiana state hospital, now a National Guard base with a 1,000-acre urban training complex that developers could use to assess their PNT technologies. They trekked through concrete subterranean tunnels, filled with several inches of water, weaving for 1.5 miles under structures that were both abandoned and still in use. They drove across rocky terrain, on paved roads with traffic circles and overpasses, and through mock villages. They navigated their way on foot through an old prison complex, a collapsed parking garage and a five-story hospital. A variety of scenarios brought the technology to life, providing the next step in assessing it beyond a PowerPoint presentation or white paper. Dubbed a technical experimentation, this event was one of several that occur throughout the year to rapidly assess the technical maturity and possible use of new or emerging technology based on specifically identified areas of need. While technical experimentations are business as usual and have been for more than a decade at the U.S. Special Operations Command (SOCOM), for the Army it marked a pivotal first, with its Rapid Capabilities Office (RCO) leveraging the event to lead the PNT excursion. This is just one of many avenues the Army is looking at through the eyes of SOCOM to gain insight into streamlining acquisition processes. (For more on the SOCOM acquisition model, see “Aggressive. Innovative. Fast.”) “The Army participated in these events in the past as assessors or evaluators, but this is the first that we know of where we were able to lead a portion of the event, in this case PNT,” said Rob Monto, director of RCO’s Emerging Technologies Office. “SOCOM took a leap of faith and partnered with us because it was a technology they were interested in as well. So it set up this unique collaboration that was beneficial for both, and we hope will lead to future joint efforts.” Among the more than 200 buildings, abandoned cars and rubble that make up the Muscatatuck Urban Training Center in Butlerville, Indiana, the Army RCO led the PNT portion of SOCOM’s Technical Experimentation 18-2, which took place March 26-29. While the RCO evaluated PNT technologies, the SOCOM event was much larger, evaluating such things as optics, biometrics, advanced sniper rifles and cognitive enhancement as well. The technical experimentations bring together the users, the program offices and the technology developers to evaluate promising new capabilities in a flexible, unclassified, operational environment. These events, which SOCOM holds a few times each year, allow for detailed user feedback and discovery of new or emerging technologies while also promoting information exchange and risk reduction. “I talk about this three-legged stool, where, if you don’t have all three players involved, you’re going to have some level of disappointment,” said Dan Bernard, the SOCOM acquisition, technology and logistics lead for the technical experimentations. “The technology developers are essentially showing their kit to the user and the program offices at the same time. We’re looking at early development. It does no good to do this with finished products. That’s just shopping.” Mounted special operators drive through a village center at the Muscatatuck Urban Training Complex during the SOCOM-led technical experimentation event in March. The Army RCO was a partner in the event, assessing PNT technologies that operate in a GPS-denied environment. Nine technology developers participated in the PNT assessment with capabilities for both mounted and dismounted Soldiers. ‘LOW THRESHOLD OF ENTRY’ Falling under the Special Operations Forces Acquisition, Technology and Logistics Center, the technical experimentation events take place at military sites across the country. They focus on various technology areas of need, identified by the SOCOM service components and program offices. Having identified a need, SOCOM posts a request for information (RFI) for the experimentation event on the Federal Business Opportunities website. Anyone interested in participating simply responds to the RFI. After the event, the participating companies receive detailed assessments of how their technologies performed. “We treat everyone like they are our customers and we want everybody to go home satisfied, feeling like they got something out of it,” said Bernard. “We’re doing this early in the development process. So if there’s a company pursuing a technology and they get this sound bite from an operator—that can really help to shape their thinking. “Feedback could be, ‘It needs to be lighter,’ or, ‘This is great, but if I have to carry that, then something has to come out of my rucksack,’” he explained. “That’s news to a lot of people. If you are a tech developer, you’re thinking, ‘This is a cool thing, and I don’t understand why they don’t want to carry it.’ But they have to understand that it is going to have to be good enough to replace something [Soldiers] are already carrying.” This feedback, combined with the three-pronged approach of having the program offices, users and developers working together in one place at the same time, is what attracted the Army RCO to participate, Monto said. “This is a low threshold of entry, where you have very small tech companies standing shoulder to shoulder with traditional defense companies, and you can assess the technology in an operational environment with the actual users,” he said. “Being able to participate in this SOCOM-led event meant the Army could determine if the capabilities were tangible now, while also giving us a better understanding of what technologies are out there.” The SOCOM technical experimentation provided the ideal venue to host an initial RCO “burn-off” event. Both the SOCOM event and RCO burn-offs emphasize the value of bringing together commercial capabilities or emerging technologies in an operational demonstration to size them up against a set of criteria with a very low barrier to entry and without the pressure of a formal test. By being able to use the SOCOM event, the RCO can better prepare for its first solo burn-off, expected later this year. Muscatatuck Urban Training Complex in Butlerville, Indiana, offers an urban environment complete with more than 200 structures, subterranean tunnels, downed aircraft, a church and a bus station, among other useful features. The site served as the location for a SOCOM technical experimentation event March 26-29 in which the Army RCO led an assessment of new technologies to keep warfighters mobile and safe in environments where GPS doesn’t work. (U.S. Army photos by Nancy Jones-Bonbrest, Army RCO) ‘DOES SOMEBODY … WANT TO USE IT?’ Technologies demonstrated during the PNT portion of the technical experimentation included radio-frequency range finding, atomic clock systems and inertial navigation unit technology. Each presented innovative ways to overcome jamming, which occurs when an adversary overpowers signals from GPS satellites so that receivers in certain areas cannot operate, and spoofing, or tricking a GPS receiver into calculating a false position. In the driving rain at Muscatatuck, mounted operators attempted to keep their vehicles on course without GPS while moving through various scenarios and settings. The different scenarios helped the Army measure technology performance and run comparisons, since one solution might do well driving around a planned course but drop in performance on a rough patch of road or an unplanned detour. Similarly, while a dismounted system could perform well for Soldiers climbing the stairs of a parking garage, it might not do as well within an enclosed concrete tunnel. Yet for a Soldier using the technology, all situations are relevant. To track the results, the RCO provided a GPS logger that recorded the ground-truth data to compare against the log files of the demonstrated systems. Additionally, they took distance and location measurements for the buildings and tunnels where GPS was not available. In all, nine developers participated in the PNT portion of the technical experimentation at Muscatatuck. They demonstrated technology that included, for example, a mounted device with a PNT-reliant system that can operate despite GPS disruption by using inertial measurement units and precision timing technology. The device combines PNT functions typically achieved through multiple independent systems. Another company demonstrated a dismounted system that uses an inertial navigation unit that users wear on the foot. It communicates with a smartphone via Bluetooth and uses robust algorithms to communicate during failures and dropouts. A third showed how three antennas prepositioned on the rooftops of nearby buildings provided triangulation to enable radio-frequency ranging for both mounted and dismounted operators. The Georgia Tech Research Institute assisted in developing the various demonstration plans and provided quantitative analysis of the data collected. “At an event like this, you get two things. On one side, you get the quantitative analysis that we are doing, and that answers, does this system really work? Does it actually provide position with some reasonable amount of accuracy?” said James Perkins, principal research scientist with the institute. “But I think the other side you get is the operational side: So, does somebody who is a boots-on-the-ground Soldier actually want to use it? Seeing the operational perspective and seeing what real operators think about a system is important early in the development.” Having navigated to the top of a partially collapsed parking garage, an operator uses a dismounted device to check position and time. The scenarios for those demonstrating dismounted systems also incorporated a subterranean tunnel and multilevel jail. The technologies for dismounted service members included one that uses an inertial navigation unit worn on the foot and communicates with a smartphone via Bluetooth. CONCLUSION As the RCO uses different burn-off events throughout the year to determine if a new technology can be used to meet a specific need, it has gained important experience from the Muscatatuck event and anticipates partnering again with SOCOM. For industry, the burn-offs provide a chance to showcase capabilities and receive formal and informal feedback. For the Army, they yield a greater awareness of what promising new technology is available now and how it performs under different conditions. The PNT project manager and the Army Futures Command’s cross-functional team also participated at Muscatatuck to facilitate potential future capability efforts, which can help build unity of effort to enable faster, more streamlined modernization efforts. “Bringing the Soldier and developer together early on allows the Army to speed up the requirement development process,” said Benjamin Pinx, product director in the Emerging Capabilities Office of the Project Manager for PNT. “During this particular event, we received immediate feedback from our dismounted operators and learned a lot. What we learn in these early experiments will influence how the Army continues to modernize the force and enable faster development and fielding of enhanced PNT capabilities for Army platforms and the Soldier.” “This event allowed vendors to demonstrate both dismounted and mounted PNT capabilities our warfighters need today to fight and win against near-peer threats in an electromagnetic-warfare contested environment,” said Lt. Col. Brian Mack, the emerging technologies coordinator for the Army network cross-functional team. “Equally as important, it demonstrated a commitment of Army modernization change agents like the Rapid Capabilities Office, the Army’s Network Cross-Functional Team and the Army’s Position, Navigation and Timing Cross-Functional Team to come together, collaborate, team and solve some of our most challenging capability gaps facing the warfighter.” For more information on the technical experimentation events, go to http://www.socom.mil/sof-atl/pages/technical-experimentation.aspx. For more information on the Army RCO or its Emerging Technologies Office, go to http://rapidcapabilitiesoffice.army.mil/ or http://rapidcapabilitiesoffice.army.mil/eto/. NANCY JONES-BONBREST is a staff writer for Data Systems Analysts Inc., providing contract support to the Army Rapid Capabilities Office. She holds a B.S. in journalism from the University of Maryland, College Park. She has covered Army modernization for several years, including multiple training and testing events. This article will be published in the July – September 2018 issue of Army AL&T magazine. Subscribe to Army AL&T News, the premier online news source for the Acquisition, Logistics, and Technology (AL&T) Workforce. Related posts: Strengths and myths of what makes special operations forces acquisition special Solidifying the base Been There, Done That: Acquisition Reform The heat is on: Burn-offs move ideas to the field for assessment