Seventh in a series of articles on how the Combat Capabilities Development Command is supporting the Army’s “six plus two” modernization priorities.
by Maj. Gen. John A. George
Maj. Gen. John A. George is the commanding general of the U.S. Army Combat Capabilities Development Command.
Training is a way of life for Soldiers. Beginning with initial basic training, Soldiers participate in training throughout their military careers. For most of our history, training occurred in the field or in a classroom. As the Army looks to the future, it is modernizing how it trains Soldiers and units by exploring synthetic training technologies.
The Army’s Synthetic Training Environment (STE) aims to combine the latest advances from the virtual and gaming industries to create complex operational environments for immersive unit training. STE’s terrain tools will enable Soldiers to select any location on a virtual globe and train in an environment that they may encounter in combat.
Nestled under the Army’s Soldier lethality modernization priority, STE is a collaborative effort of the STE Cross-Functional Team, the U.S. Army Combat Capabilities Development Command Soldier Center and the Program Executive Office for Simulation, Training and Instrumentation (PEO STRI). The Soldier Center’s Simulation and Training Technology Center serves as the primary science and technology organization dedicated to providing modeling, simulation and training technology in direct support to the STE Cross-Functional Team and PEO STRI. Located within blocks of one another in Orlando, Florida, all three organizations capitalize on this prime location—at the heart of the Florida high-tech corridor, where more than 170 companies specialize in simulation- and training-related research—to establish critical partnerships across industry, academia, government organizations and international partners in order to develop state-of-the-art training technology.
KEEPING IT REAL
The STE, which is scheduled for initial operating capability in fourth quarter 2021, will provide a realistic operational environment for training using state-of-the-art mixed-reality technologies. One of STE’s major science and technology focus areas is improving dynamic occlusion, or the capability to “portray” or “hide” computer-generated characters and objects behind real things, and to do so in real time from multiple perspectives as actors, objects and users move around in the environment. The virtual objects and the real scene must be perfectly aligned in order to maintain high levels of realism and enable objects to behave as they would under normal conditions.
For example, the popular mobile device-based game, Pokémon Go, is a virtual example of dynamic occlusion that many people play. Players use their mobile phone or tablet to seek virtual characters that are hidden in the real world. The characters appear as if they were hiding in and between real-world objects. Sometimes, however, players will see a character that appears to be on the far side of a room, but is still visible in front. This technical issue removes aspects of realism, and when used for military training will have an impact on outcomes.
Augmented reality game fans may encounter dynamic occlusion errors when the view within the game is not layered or aligned appropriately with real-world objects, making the experience feel unnatural—an annoyance to the player. In military scenarios, the problem can adversely affect the learning experience and lead to negative habits if, for example, a Soldier can’t realistically take cover or if a vehicle crew can’t accurately aim and fire at an enemy.
Occlusion of live, moving objects is challenging, and doing so at long distances is even more so. Current augmented-reality head-mounted displays restrict Soldier training to small, indoor environments because of hardware limitations. For example today, a squad can execute a close combat urban warfare training event in a room about the size of a basketball court. To work this challenge, the Simulation and Training Technology Center is evaluating several depth sensors and deep-learning algorithms to improve the alignment of the virtual and real objects within the scenario. They have also developed a prototype depth mask using networked depth cameras that achieve greater than 80 percent occlusion accuracy at ranges beyond 30 meters.
Plans for the immediate future include the same occlusion accuracy at ranges greater than 60 meters—still not good enough for our Soldiers. The goal at the Simulation and Training Technology Center is to mature and demonstrate augmented reality algorithms and techniques that occlude dynamic objects in realistic, changing environments with extended ranges that enable the same squad or platoon to train outdoors in all conditions and at the distance of their organic weapon systems.
BREAKING IT DOWN
The Simulation and Training Technology Center is developing science and technology research that will enhance the realism, effectiveness and usability of the STE. The team is heavily focused on novel automation techniques and emerging state-of-the-art technologies—areas that current vendors and industry partners deem too risky to invest in, do not have the necessary in-house expertise or lack military-specific domain knowledge.
The STE will enable units and Soldiers to conduct realistic, multi-echelon, collective training anywhere in the world. The STE information system, the software backbone of this capability, includes One World Terrain, Training Simulation Software and the Training Management Tool. These three software capabilities will be integrated into the STE information system to manage, conduct and deliver synthetic training to the point of need via a ground vehicle, air vehicle and dismounted Soldier interfaces and simulators. The software operating system will enable “plug and play” components such as Reconfigurable Virtual Collective Trainers and Soldier/Squad Virtual Trainers for unit training while also supporting the future Live Training Environment (force-on-force and force-on-target training) and Next Generation Constructive simulation capabilities (exercises for divisions and above).
The Simulation and Training Technology Center’s One World Terrain research efforts include developing and demonstrating software tools and methods to improve the generation, processing and fidelity of synthetic terrains. This includes representing the complexities of multidomain environments and reducing or eliminating inefficiencies as the synthetic terrains are developed. An important component of achieving a dynamic 3D global terrain is using disparate data sources, which will enable Soldiers to train in a variety of settings, including forested areas, massive urban centers and dense rainforests.
Units currently require subject matter experts and a minimum of six months to manually create synthetic terrains unique to each event. In response, our One World Terrain efforts focus on new methods of using artificial intelligence to decrease the amount of human expertise and time needed to develop, edit and validate terrains used for training, while simultaneously producing them with better accuracy and fidelity.
Training Simulation Software science and technology efforts focus on intelligent character behaviors, scalability and warfare modeling, which are crucial to providing the realism necessary to immerse Soldiers in the training environment. Current military training simulations are based primarily on semi-automated behaviors that are reasonably predictable. The virtual opponent in the STE must have a level of cognition and unpredictability that also adheres to known adversarial doctrines. To achieve this effort, we are leveraging state-of-the-art artificial intelligence to better replicate multidomain operations and realistic environments that will enable units to train how they will fight.
The Training Management Tool component of the STE will focus on automated team assessments, automated feedback, after-action reviews and intelligent adaptive training for teams. Currently, experts need to be present during every key task of a training event to annotate and assess performance in real time. Details are often overlooked or missed entirely during this process. Therefore, we are coupling artificial intelligence with learning science to develop new ways to measure, assess and provide feedback to Soldiers and leadership automatically during and after STE training events. The feedback will also inform subsequent training.
Our research with the Soldier/Squad Virtual Trainers includes developing augmented reality and mixed reality technologies that enable virtual characters, systems and effects to be inserted into a live training environment. Augmented reality and mixed reality is most effective when it is realistic and creates disbelief for the user. Creating a sense of disbelief involves developing synthetic environments, including sensorial stimulants that are indiscernible from real environments. This allows users to virtually “transport” and conduct training operations in any environment without leaving their home station. The sense of disbelief is much easier to create when using virtual reality technologies that fully create the visual experience of the user without concern for the user’s real location. Augmented reality, however, is much more difficult because synthetic objects must seamlessly blend with the user’s real environment so that users cannot distinguish the real from the virtual. Merging real and virtual must be applied to both static objects and dynamic interactions between objects, including real and virtual.
We are also working to improve live training by finding alternatives to the Instrumentable- Multiple Integrated Laser Engagement System, or I-MILES. Built on technologies developed in the 1970s and 1980s, I-MILES is used to support live force-on-force and force-on-target training at Army training centers worldwide. While I-MILES has been enhanced throughout the years, laser-based systems are limited in their ability to realistically represent lethal effects during live exercises. For example, a shrub or cardboard box provides effective cover from a laser hit but would be useless in a firefight. Our team is seeking ways to more accurately depict the effects of direct and indirect fire and facilitate training on more sophisticated or longer-range weapons that can’t easily be integrated into a live training exercise.
Another goal at the Simulation and Training Technology Center is to demonstrate a dual-use eBullet system that will simulate tactical engagements for blue forces (friendly) and opposing forces weapon systems within both live and synthetic training environments. eBullet technologies offer a variety of options for Soldiers. For example, an artillery round would not simply be a hit or miss. Soldiers would be able to determine how accurate the firepower was and if more strikes were needed. Plans for the future include more advanced options with simulated electronic warfare, cyber, chemical and biological attacks.
SOLDIER TOUCH POINT AND TESTING
Soldiers are key to providing meaningful input for many of our research efforts. We use feedback from the Army Expeditionary Warrior Experiment, which is an annual training event where Soldiers give feedback on technology; the feedback is used to improve products to fit Soldiers’ needs.
We also have existing relationships with customers, such as the Joint Readiness Training Center, which uses our technology during unit training rotations. Technology at the Joint Readiness Training Center is used by thousands of Soldiers who provide direct and indirect feedback.
The STE Cross-Functional Team conducts user and technical assessments, which connect Soldiers with developers and engineers. The Soldier Center Simulation and Training Technology Center supports the STE Cross-Functional Team with these events, which will be scheduled each quarter until initial operating capability, planned for the fourth quarter of 2021. In addition to helping plan these events, our researchers work with vendors and Soldiers to observe how they interact with the systems, and facilitate technical discussion to help refine the direction of our science and technology efforts.
One of the challenges for vendors is the need to test their products on a military network. The Technology Integration Facility in Orlando enables vendors with promising technologies to conduct rigorous testing on a military test network. The Technology Integration Facility serves as a place for the cross-functional team to test the products, provide feedback to vendors, and quickly adopt and refine capabilities that meet STE requirements.
The Team Orlando Integration Lab is co-located with the Technology Integration Facility, offering the modeling and simulation community a place to collaborate on research to advance the development of training aids, devices, simulators and simulations.
TEAMING AROUND THE WORLD
Our strong partnership with industry is threefold. The first component includes contracts or agreements with industry to develop innovative simulation and training technology. Technology development is limited to problem sets with a military application that the commercial market otherwise would not develop. The second is the transition of technology to industry partners for commercialization, and the third is entering into exchange agreements, including cooperative research and development agreements, to enable research or development efforts with mutual benefits to both parties.
Academic partnerships include the University of Central Florida Institute for Simulation and Training, which supports STE technology development and independent technology assessment; the University of Southern California’s Institute for Creative Technologies University Affiliated Research Center, which supports One World Terrain research; Carnegie Mellon University, which develops artificial intelligence “free-thinking threat” algorithms; and the University of Texas, which develops One World Terrain and Training Simulation Software technologies.
Key partnerships with other government organizations include close relationships with the STE Cross-Functional Team, PEO STRI and the Combined Arms Center – Training. The STE Cross-Functional Team oversees development and sets requirements and priorities for the STE and related activities. PEO STRI is the materiel developer of the STE and primary transition recipient of the newly developed technologies. The U.S. Army Combined Arms Center – Training identifies the mid- and long-term gaps for the Army Simulation and Training Program. Other government partnerships include cross-collaboration within CCDC’s internal centers and labs, and with technical experts at the Engineer Research and Development Center, Army Geospatial Center and National Geospatial Agency.
We also maintain active relationships with international partners, including data and information exchange agreements with seven nations, future project agreements with two nations, one exchange engineer or scientist and participation in international standard activities with NATO and the Technical Cooperation Program.
CONCLUSION
Current and emerging technologies from the Army’s research centers and the virtual, gaming, data storage and network industries are enabling us to provide more accessible and realistic training for Soldiers. The Army is leveraging these advances in technology, as well as the experience of the warfighter, to provide a training capability that accelerates Soldier and unit readiness to win decisively in multidomain operations.
For more information, go to www.army.mil/ccdc.
MAJ. GEN. JOHN A. GEORGE is the commanding general of CCDC. Before assuming command on Nov. 1, he served as deputy director and chief of staff of the U.S. Army Futures Command Futures and Concepts Center. He graduated from the United States Military Academy at West Point and was commissioned into the Army in 1988. He has an M.S. in social psychology from Penn State University and an M.S. in national resource strategy from the Industrial College of the Armed Forces.
This article is published in the Spring 2020 issue of Army AL&T magazine.
Subscribe to Army AL&T News – the premier online news source for the Army Acquisition Workforce.
Subscribe
