ACQUISITION STRATEGY AT AUSA: Product Lead for Project Linchpin, Bharat Patel, second from right, goes over acquisition strategy with PEO IEW&S headquarters staff Oct. 11 at the AUSA symposium in Washington. PEO IEW&S staff from left: Mardel Wojciechowski, division chief, contract planning; Steven Rothenberg, contract planning; Mike Schwartz, chief engineer and system of systems engineering division chief; and Steve Gunther, division chief, program acquisition and cost efficiencies. (Photo provided by PEO IEW&S)
The Army is developing its first artificial intelligence pipeline to deliver AI and ML capabilities to sensors for faster and more accurate decision-making, marking the future of overmatch in multidomain operations.
by Cheryl Marino
A revolutionary initiative called Project Linchpin is developing the Army’s first artificial intelligence and machine learning (AI and ML) operations pipeline. The capability aims to solve the problem of continuous training, integration and delivery of AI and ML at the speed required for multidomain and large-scale combat operations.
The initiative, which is a collaboration between Army Futures Command’s Artificial Intelligence Integration Center, U.S. Army Combat Capabilities Development Command Army Research Laboratory and the Program Executive Officer for Intelligence, Electronic Warfare and Sensors (PEO IEW&S), is considered a pre-program activity with the first use case focused on the Army’s suite of modernized sensor systems (satellites, optics, signals receivers, etc.) that will take shape in the next one to two years. It not only allows the Army to establish the necessary infrastructure for AI-enabled systems, but the project also provides a mechanism to create new industry partnerships and foster a competitive environment for third-party integration into Army programs.
For the Soldier, the capability is an invisible backend that is equipped and manned by technical analysts using specialized tools. A Soldier at the tactical edge receiving a sensor feed and leveraging AI would be able to identify enemy equipment in time and space (e.g., differentiate between a tank or a bus); be given enemy course of actions based on terrain, time and space; receive real-time alerts about enemy situation or movement and, with a human in the loop, send messages to the commander and warfighter for action. Current apps would still be used to conduct missions, but now with a little AI sprinkled on top to improve the output of the app itself (e.g., find the needle in the haystack in seconds or minutes not weeks, months or years).
Mark Kitz, program executive officer for IEW&S, said that enabling those same tools and systems with AI and ML capabilities is a game changer. “Today, our sensors collect far more data than human analysts can exploit and analyze manually. AI and ML models can be trained to detect objects and alert analysts to possible targets in a way that can close the gap on the amount of data we process, and how commanders and decision-makers see the battlefield. Making sure we have the infrastructure and a pipeline to facilitate that delivery is a key component for mission success.”
GOING WITH THE FLOW: The differences and benefits between a traditional weapons system pipeline versus an AI- and ML-enabled weapons system pipeline.
The U.S. Army’s Joint Light Tactical Vehicle showcases it’s operational capacity while being driven through an off-road obstacle course at Fort McCoy, Wisconsin Aug. 17, 2022. The JLTV’s modernized design provides advanced protection, mobility and capability for constantly changing landscapes of battle. The Army established Army Futures Command to drive Army modernization by leading a campaign of learning to understand how technology will change how Army forces organize and fight to deliver speed, range, convergence and decision dominance in the future operational environment. (U.S. Army Reserve photo by Spc. Brittney Joy)
(U.S. Army photo by Spc. Collin MacKown)
IT’S ALL IN THE DELIVERY
The concept of Project Linchpin is just like software pipelines used on cellphones for operating systems and apps. User data is provided back to the software developer, which helps generate the next iteration of patches, security upgrades and features. Both software and AI and ML pipelines require an infrastructure to enable that feedback and delivery—with a few key differences. Project Linchpin’s plan will establish this infrastructure and an environment that will allow successful deployment of AI and ML capabilities to intelligence, cyber and electronic warfare sensor systems like high altitude aerial platforms that capture imagery and other signals using an end-to-end pipeline. The delivery of these AI and ML capabilities is critical as it supports the work intelligence analysts do and will lead to faster and more accurate decision-making.
“These sensors gather all the information used to understand the operational environment in all domains, including imagery, video and signals,” said Bharat Patel, product lead for Project Linchpin, Project Manager Intel Systems and Analytics (PM IS&A), PEO IEW&S. “The goal of the sensor AI pipeline is to deliver trained algorithms [i.e., models] to sensors and sensor data-processing platforms.” For example, AI aboard future aerial sensors could assess battle damage to increase situational awareness of enemy forces and critical infrastructure on the battlefield, and new intelligence ground stations could automatically correlate and detect targets to alert decision-makers for more rapid targeting.
Col. Christopher Anderson, project manager for IS&A, said that there are dozens of examples to demonstrate the capability. “It would be like looking at Google Earth’s satellite imagery and quickly being able to identify all of the Ford F-350s in Maryland,” he said. “Normally an analyst has an area of interest assigned to them, and they have to look for a bunch of things; it could be enemy equipment, friendly forces, schools, etc. The analyst zooms in, out, rotates imagery and stares at the screen for hours. With AI, systems can quickly [within seconds] help Soldiers identify objects of interest that can be turned into intelligence for faster decisions and give Soldiers those hours back to work on more important tasks and analysis.”
Project Linchpin, denoting the name of the pin that keeps a wheel on its axle, organizes all the components needed to deliver these leap-ahead AI and ML capabilities to sensor systems. For PEO IEW&S, an operational pipeline helps solve the problem of delivering AI that Army sensor systems need to maximize their capabilities.
TAKING THE INITIATIVE: The Project Linchpin initiative.
SENSORY PERCEPTION
Army sensors (satellites, optics, signals receivers, etc.), like cellphone sensors, collect, process and analyze information instantly. They are used on land, under water, on aerial platforms and in satellites for things like explosive-detection systems, chemical warfare detection, missile systems and target and weapon seekers to gather all the information used to understand the operational environment and make collected data available to analysts through a feedback loop.
“Today, human analysts use tools to process, exploit and analyze the information to create intelligence. This intelligence is disseminated to decision-makers and commanders that can drive operations, inform targeting and support tactical, operational and strategic decisions,” said Patel. The only limitation is how much of the data that analysts can process. The more data that can be consumed and processed, the more complete the picture of the operational environment becomes. “When more advanced sensors are developed and fielded by PEO IEW&S, the volume of the data will grow exponentially.”
PIPELINE EXPRESS
According to Maj. Nick Bono, Department of the Army systems coordinator, in the Office of the Assistant Secretary of the Army for Acquisition, Logistics and Technology (OASA(ALT)), it’s the feedback loop within a Machine Learning Operationalization Management (MLOps) pipeline that ensures the delivery is continuous and performing.
“The pipeline is really an infrastructure that allows technical and functional teams to integrate and deliver AI and ML models to sensor systems,” Bono said. “This is an environment that accounts for live data in the operational environment and feedback from Soldiers using these tools and systems to keep improving the performance of the AI.” Within the pipeline, data scientists will interact with components that manage sensor data to train models, as well as test, evaluate, verify, optimize and deploy them onto the systems.
“Each one of those steps is a critical part of the process to ensure Soldiers trust the AI that is helping them,” Bono said. “Today, there is so much data that analysts can’t process all of it. To provide the most accurate and complete assessments of the operational environment, PEO IEW&S recognized that the process of delivering AI and ML capabilities to sensors continuously and rapidly is just as important as the models themselves.”
Kitz noted the importance of this investment. “Linchpin is the right name because this is a hard problem for an absolutely vital product.” Recognizing the need for a solution that focuses on the Soldier is a model for early adoption and acceleration in the use of AI for sensor systems, he said. “Soldiers have to understand where AI and ML models come from, how they are trained, that they are ready to perform their tasks and that Soldier feedback is baked into the entire process. With a pipeline like this, we’re delivering trust.”
AUGMENTED VISION: Spc. Mitchell Mcneil, assigned to 82nd Airborne, 3rd Brigade Combat Team trains with the Integrated Visual Augmentation System as a part of Project Convergence 2022 at Camp Talega, California, on Oct. 11. During Project Convergence, many systems are tested to determine how future command and control capabilities can be integrated with all-service multinational partners. (Photo by Sgt. Thiem Huynh, Army Futures Command)
DOLLARS AND SENSE
AI is unique from software in that there must be continuous integration and continuous delivery to systems. AI and ML capabilities require “pipelines” for delivery. Project Linchpin will result in a program of record for the Army’s first MLOps pipeline that manages all the unique aspects of AI and ML projects. As new algorithms are developed for specific tasks, and existing algorithms are trained on new data, the pipeline integrates and delivers them at the speed required for multidomain operations.
Emerging AI- and ML-enabled systems and the algorithms and models they use are expensive, but “consolidating the ‘development and deployment pipeline’ under a single program of record creates efficiencies that reduce costs,” Patel said. “It reduces the burden on individual programs that employ AI to manage their own pipelines and mitigates redundancy in like efforts for the broad range of sensors that may collect similar data types or use similar models.”
The pipeline also creates an opportunity for industry partners to compete, and for the Army to select the best capabilities. Since sensor AI requirements are continuous, and change with the operational environment, there is a constant need to identify and integrate new models. “Using the adaptive acquisition framework to employ the most flexible acquisition and contracting strategies will allow a competitive environment for industry,” said Kitz. “It also allows an opportunity to transition the Army’s own S&T [science and technology] investments in AI and ML capabilities directly into a program of record. Competition and effective use of S&T efforts all contribute to cost savings.”
MODERNIZED MOBILITY: The U.S. Army’s Joint Light Tactical Vehicle showcases its operational capacity while being driven through an offroad obstacle course at Fort McCoy, Wisconsin, on Aug. 17. The JLTV’s modernized design provides advanced protection, mobility and capability for constantly changing landscapes of battle. (Photo by Spc. Brittney Joy, U.S. Army Reserve)
DEMONSTRATED ABILITIES
The Army already possesses some of the tools and components required for an end-to-end solution. Initiatives like Arcane Fire demonstrated the ability to deliver AI to programs during Project Convergence 22 and other experiments. Patel said that future development will focus on key tasks like data management, continuous integration and continuous delivery, test and evaluation, quality control, explainability (also referred to as interpretability) and trust.
PEO IEW&S has laid the ground for Project Linchpin to be successful, having worked with the Office of the Secretary of Defense’s Project Maven—the DOD’s most visible artificial intelligence office, designed to process imagery and full-motion video from drones and to automatically detect potential targets—since 2018, to understand how to deliver AI capabilities to the Army.
THE FUTURE OF UAS: A Jump 20 Group 3 Unmanned Aerial System takes off on Oct.13 at San Clemente Island, California, during Project Convergence 22. At Project Convergence, all U.S. military services, as well as multinational partners, are experimenting with distributed sustainment of unmanned aerial systems over extended distances in a maritime environment. (Photo by Spc. Collin MacKown, 14th Public Affairs Detachment)
CONCLUSION
Project Linchpin aims to provide an AI infrastructure. “The goal is to create a complete and efficient AI and ML development and delivery pipeline for sensor programs within PEO IEW&S to provide needed capability while managing cost and risk.” said Kitz. “And it has the potential to create a new, dynamic marketplace for industry partners to compete and deliver the best tools, algorithms and models.” He said that, pending OASA(ALT) concept approval in fiscal year 2023, a campaign of learning and formal initiation of Project Linchpin would follow. After that, an S&T transition plan could be implemented along with future contract activities.
In terms of overmatch, Anderson compares AI and ML capabilities to the advantage night vision provided the Army when it was first fielded. “You can make a direct comparison in that night-vision goggles and optics allowed the Army to ‘see’ where there was no visibility. AI and ML capabilities for sensors, like object detection, provide overmatch by going one step further—AI allows sensors that can see to detect, recognize and identify. They can perform functions that only human eyes could do, but they can do it faster and at scale across the entire battlespace.” Project Linchpin is critical because it provides the mechanism to continuously deliver that advantage to sensors so the Army can make decisions faster than its adversaries and enable targeting and overmatch with long-range precision fires and other effects.
For more information, go to https://peoiews.army.mil/ or contact Larry Glidewell at larry.d.glidewell.ctr@army.mil.
CHERYL MARINO provides contract support to the U.S. Army Acquisition Support Center at Fort Belvoir, Virginia, as a writer and editor for Network Runners Inc. and Army AL&T magazine. She holds a B.A. in communications from Seton Hall University and has more than 20 years of writing and editing experience in both the government and commercial sectors. In addition to corporate communications, she is a feature writer and photojournalist for a biannual New Jersey travel magazine.
