The Army is bringing together many sources of data to enable better decision-making with “data fabric.”
by Nihar Patel, Upesh Patel, Col. Evert R. Hawk II and Krupal Kapadia
For a battlefield commander making a pivotal decision, too much data can be just as bad as too little. Reports, imagery, raw message traffic and other content arrive via different applications and compete for attention. Meanwhile, the information the decision-maker really needs may be buried deep in another digital file, yet to be discovered.
Finding the balance—harnessing and providing the relevant data at the right moment—is the driving force behind the Army’s work with data fabric. As the name implies, data fabric technology “stitches” together a variety of information sources and unique data formats, ranging from modern sensor arrays to legacy systems designed for specific functions like logistics or aviation. It overlays them with common interfaces and services using open standards so the systems can share information without custom translators in the mix. In doing so, the fabric creates a larger pool of data that is more enriched, synchronized and transparent, and routes the right data to the operator who needs it.
As Army senior leaders and operational commanders increasingly emphasize data as a strategic asset, it is clear that accessing, managing and protecting data will be key to achieving the Army’s modernization priorities. Implementing a tactical data fabric is foundational to these efforts—and to realizing the vision of Joint All-Domain Command and Control (JADC2), the Department of Defense-wide effort to integrate sensors, shooters and command and control nodes for resilient, adaptable communications across the joint services. For JADC2 to achieve this potential, useable data must flow quickly and accurately through the data infrastructures used by the Army and the joint services.
To reach that future objective state, the Army is following an incremental approach to deliver data fabric capability over time through the network capability set construct. Working across the Army modernization enterprise, and in collaboration with our joint service and industry partners, the Army has matured its own tactical data fabric and is now experimenting with additional commercial capabilities, with the goal of initial fielding in fiscal year 2023.
COMMUNICATIONS EXERCISE: The Army’s Joint Systems Integration Laboratory hosted its second communications exercise April 5-16 with numerous tactical systems at Aberdeen Proving Ground, Maryland, in preparation for Project Convergence 21 this fall. Data fabric helps address the challenges of connecting and translating stovepiped data that exists throughout different warfighting function systems. (Photo by Kathryn Bailey, PEO C3T Public Affairs)
THE WHY AND WHAT
Access to the right information at the right time is a critical element to achieving future battlefield advantage at the operational and tactical levels. Timely access to data depends on agility in how data is managed. As Chief of Staff of the Army Gen. James McConville put it, “We’ve got to be able to move data machine to machine very, very quickly. That is the secret if you want to get to speed, range and convergence.”
To achieve that goal, there are several challenges the Army needs to overcome. First, data in warfighting functional systems (such as fires, maneuver or sustainment) is often stovepiped or isolated, which can result in inconsistent and missing data in the common operational picture. When data is exchanged between warfighting function systems, it is frequently compressed due to message exchange formats, and it loses some of the information from the original source. Additionally, data is unnecessarily restricted from potential consumers when systems inherit the classification of the network they reside on, rather than managing the classification of the data itself. Finally, artificial intelligence (AI) capabilities—a necessity in order to operate at the speed of digital war—are starved for data, because warfighting systems generate information at high volume but without great value, as much of the collected data remains unprocessed and difficult to find, unbeknownst to the operator.
These technical challenges, coupled with the rising number of data sources available to operators, underscores the need for autonomous, flexible data integration capabilities that harvest and deliver the required data when needed. Data fabric provides that common layer to enable data discovery, synchronization and security across multiple silos and platforms—addressing the conundrum of having both too much data and too little.
So how does it work? Recently, the JADC2 Cross-Functional Team, run by the Joint Staff J-6, codified data fabric as a “Department of Defense federated data environment for sharing information through interfaces and services to discover, understand, and exchange data with partners across all domains, security levels and echelons.” The key capability components to achieve this outcome include metadata tagging, common interfaces, and security and access controls.
Metadata tagging describes characteristics about the data such as security or origin, and supports automated processing. Applying these tags using data fabric’s common design patterns and discoverable infrastructure helps make data visible, accessible, understandable, trusted, interoperable and secure.
Common data interfaces are standards that ensure data can cross boundaries where independent systems interact without losing its meaning. These interfaces are essential to getting data to and from data producers and consumers, including machine-to-machine accessibility. In both military and commercial instances of data fabric, this framework is achieved through application programming interfaces (APIs), which are software intermediaries that allow distinct applications to interact. Data fabric APIs are open and standards-based, which allows the Army to more quickly implement new and existing capabilities from multiple services and industry partners.
The final components of data fabric focus on access controls and security. Access controls determine whether an individual, or an AI algorithm, is permitted to search, retrieve, read, create or manipulate data. Data security protects the data itself—at rest and in transit, regardless of the system hosting environment—from unauthorized discovery, modification or destruction. Together, access controls and security support efficient access to, and sharing across, varied data sources without compromising the integrity of the information.
PROJECT RAINMAKER
The Army’s major contribution to data fabric development thus far is called Project Rainmaker, a science and technology effort spearheaded by the Command, Control, Communications, Computers, Cyber, Intelligence, Surveillance and Reconnaissance (C5ISR) Center—a component of Army Futures Command’s Combat Capabilities Development Command (DEVCOM).
Rainmaker approaches data fabric from a distinctly Army tactical vantage point, prioritizing the need to mediate data between existing programs of record, warfighting functions and echelons. It targets data synchronization across disadvantaged, disconnected, intermittent and latent communications environments that commanders and Soldier encounter at the tactical edge. Rainmaker also seeks to enable AI and machine learning tools to better access and process data to support commanders’ decision-making.
Through APIs, open standards and a modular, scalable design, Rainmaker aims to be deployable across the enterprise, tactical and edge networks. It is intended to leverage a wide range of Army network hardware, from fixed-site data centers and servers at the corps and division level; to tactical server infrastructure (both on- and off-premises) and laptops at brigade, battalion and company command posts; to software on mounted platforms and dismounted Soldier devices. This tailorable approach helps to account for diminishing throughput of data at lower levels and the tactical edge.
After conducting initial lab development, Army scientists took Rainmaker to last fall’s Project Convergence 20 at Yuma Proving Ground, Arizona, to experiment with data fabric in real-world conditions. Operating as a proof-of-concept, Rainmaker demonstrated improved sensor-to-shooter data sharing at the brigade and battalion levels, and contributed to a richer common operating picture to support detailed targeting. For Project Convergence 21 this October and November, Rainmaker will be integrated with other Army network and mission command capabilities to support mission scenarios in competition, crisis and conflict.
EXPERIMENTATION AND STRATEGY
In parallel to Rainmaker development, the Army is executing complementary prototyping efforts with vendor-designed data fabric technologies. In late 2019, the Army issued a request for information to industry on data fabric as part of the recurring technical exchange meeting process, which is led by the Army Futures Command Network Cross-Functional Team (CFT) and the Program Executive Office for Command, Control, Communications-Tactical (PEO C3T) to shape and target commercial innovation for potential Capability Set fielding.
After reviewing white paper submissions, the Army selected two vendors for data fabric prototyping contract awards, funded with dedicated 6.4 CFT research funds. (6.4 is the DOD research, development, test & evaluation budget activity code for advanced component development and prototypes.) CFT and PEO experts conducted lab-based experimentation with the vendor technologies under realistic threat conditions this spring, which led to a follow-on contract award to one vendor for field experimentation at Project Convergence 21.
In addition to Rainmaker and vendor technologies, the Army is also looking at Army Cyber Command’s big data platform called Gabriel Nimbus, particularly the Lower Echelon Analytic Platform, which is optimized to run at the tactical edge.
The Army’s ongoing experimentation with these technologies will continue to inform and optimize the tactical data fabric, which the Army will field to operational units through an incremental approach. Although some Rainmaker data management elements are integrated into the Command Post Computing Environment (CPCE) version the Army is currently fielding with Capability Set 21, Capability Set 23 is considered the data fabric benchmark. It will expand capabilities to deliver basic analytics, initial edge cloud capability, new API interfaces and management of structured, semi-structured and unstructured data. In Capability Set 25, the data fabric, as part of CPCE, will introduce advanced analytics using AI and machine learning, a full edge cloud capability and significantly increased capacity.
SPEED OF RELEVANCE: U.S. Army Soldiers from V Corps participate in the DEFENDER-Europe 21 Command Post Exercise at the Grafenwoehr Training Area in Grafenwoehr, Germany, in June. The goal of data fabric is to identify and provide the relevant data to the right decision-maker at the speed of relevance. (Photo by Spc. Denice Lopez, Training Support Activity Europe)
CONCLUSION
Data is a force multiplier if it can be found, made available and secured. As the Army moves toward a multi-domain operations approach to warfighting—with a larger scale and a compressed timeline for leaders to understand, decide and act—the requirement to process more data, and faster, intensifies. Implementing data fabric will better equip commanders and Soldiers to access, aggregate, share and secure mission-critical data against near-peer threats. It will improve human-machine teaming, including through AI, to enhance decision-making and speed reaction time.
Data sharing also must be optimized with joint and coalition partners. Under the JADC2 initiative, the services are modernizing command-and-control technologies and procedures to process information, make decisions and direct actions of the joint force across all warfighting domains, faster than our adversaries. As one aspect of JADC2 implementation, the JADC2 CFT is federating, or connecting, data fabric solutions across the Army, Navy, Air Force and other joint force elements to ensure interoperability. This “fabric of fabrics” effort will also include a clearly defined and documented API framework and standards, with the necessary security and governance to evolve data architectures over time as technology advances.
For all of the technical complexities and incremental steps, the end goal of data fabric is simple: give warfighters what they need make better decisions. By getting the right data, to the right people, at the right time and place—at the speed of relevance—the Army and joint force will gain an edge to prevail in the future fight.
For more information, contact Claire Heininger, Army Network CFT, at Claire.s.heininger.ctr@mail.mil.
NIHAR PATEL is the cybersecurity engineer at the Network Cross-Functional Team, under Army Futures Command. He holds an M.S. in computer engineering from Johns Hopkins University and a B.S. in computer engineering from the New Jersey Institute of Technology. He is Level III certified in engineering.
UPESH PATEL is a senior engineer in the operations-intelligence division at DEVCOM’s C5ISR Center. He holds a B.S. in electrical engineering from New Jersey Institute of Technology and an M.S. in software engineering from Monmouth University. He is Level III certified in systems acquisition and a member of the Army Acquisition Corps.
COL. EVERT R. HAWK II is the Line of Effort 2/3/4 team lead at the Network Cross-Functional Team. He holds a Master of Strategic Studies from the United States Army War College, an MBA in finance from Saint Joseph’s University, and a B.S. in business administration and finance from Penn State University. He is a certified Project Management Professional and PMI – Agile Certified Practitioner from the Project Management Institute and a Lean Six Sigma Black Belt.
KRUPAL KAPADIA is the deputy for Technical Management Division at the Project Manager Mission Command, under PEO C3T. He holds a M.S. in systems engineering from Stevens Institute of Technology and a B.S. in computer engineering from the New Jersey Institute of Technology. He is Level III certified in engineering.
