Army scientists pursue advances in situational awareness, 3-D modeling and real-time data updating to help units make better decisions faster in urban combat.
by Dr. Robert E. Davis, Dr. Kevin R. Slocum, Dr. Tad T. Brunye and Dr. Aaron L. Gardony
Today’s Army leaders consider it inevitable that U.S. ground forces will engage in combat in dense urban environments, including building interiors and subterranean spaces. These settings eliminate or severely degrade many of the technological advantages that U.S. forces and their global (near-) peers have developed over several decades, and they also may provide sanctuary to friend or foe. Dense urban environments also heighten broader risks of unintended consequences in combat.
A broad spectrum of existing and emerging research topic areas has shown the potential to develop significant capability for providing small disaggregated mounted and dismounted teams the ability to act independently, to outthink and to outmaneuver the enemy in close combat despite limited and intermittent access to higher-echelon command and control. Most of the promising science and technology (S&T) development focuses on major advances in situational awareness in urban settings and how they can lead to better decisions faster, presenting dilemmas to an adversary.
The Army S&T community has adopted the premise that urban combat, considered as a flowing series of tactical unit decisions and actions, will greatly benefit from rich and intuitive space and event and trend context. Accordingly, near-term and emerging research areas at the U.S. Army Engineer Research and Development Center (ERDC), the U.S. Army Research Laboratory (ARL) and the U.S. Army Natick Soldier Research, Development and Engineering Center (NSRDEC) include investigations into the design and formulation of new urban terrain data models, frameworks and cognitive display approaches. The goal is to identify solutions compact enough that many Soldiers and every vehicle can carry them along for sharing and analysis, while meeting a variety of needs for display on different equipment. Research interest across the ERDC and the U.S. Army Research, Development and Engineering Command also has focused on characterizing, moving and communicating within the confined space of building interiors and subterranean infrastructure.
Results of this research will shape design and development of techniques for much more rapid data generation, tailored dissemination, change analyses and visualization. In other words, Soldiers will learn as they go, and retain this spatial knowledge. This new direction, in most cases, markedly departs from the commonplace use of flat maps, in perspectives that may include 3-D features and immersive training environments.
The Army geospatial enterprise uses a concept called standard sharable geospatial foundation (data models and information architecture), which will allow for horizontal and vertical interoperability and sharing of geospatial information from the national level to the tactical edge. Built on this concept and the need for high-resolution 3-D topography of urban spaces, and working across Army platforms and applications, Army research efforts face technology-limited challenges from the near term to the far term. In the mid to far term, these include investigations of methods to introduce a user to semantically and digitally enriched information that is key to combat activities, from training through rehearsal to combat operations. Semantic markup may include street names, for example, or symbols referring to unit positions, while digital markup can refer to coordinates used in routing or targeting.
A critical line of research on visualization in a mobile context involves developing automated and rapid georegistration without delays during motion, which can distract attention or exacerbate motion distress. Some symbols and features added to displays, such as the user’s location, range to objects in the distance, etc., stay in the same position regardless of changing viewpoint. Other augmenting features, such as building outlines, need to remain fixed in the real 3-D world even when the user’s perspective moves.
We can expect improved and more diverse techniques to produce accurate, updatable geometries of urban infrastructure, including relevant materials and functions. The need to support decision-making and faster action also shapes research on what makes terrain visualization intuitive, cognitively low-cost, measurable by operators and interactive when needed. The following sections describe Army S&T research on the military-unique aspects of high-resolution 3-D enriched terrain data while leveraging commercial and academic innovation.
THREE DIMENSIONS, FOUR WAYS
These are some of the views possible with an enriched, 3-D visualization of a given area: Clockwise from upper left, a high-resolution representation; a version that filters out ephemeral objects such as passing cars; augmented reality view with edges and corners georegistered and attributed; and extracted edges and corners.
THE 3-D URBAN ‘MAP’
The goal of Army geospatial research is to design, develop and test a new, multidimensional 3-D “map” of urban infrastructure geometries, materials and functions. This capability would provide the context and baseline for a variety of Army operations. Current research efforts focus on some key attributes that such a map—really an information -architecture—would include:
- Available on demand to Soldiers and their applications, particularly in its small units.
- Measurable and supporting a variety of automated analyses.
- Updatable as conditions change.
- Intuitive displays for more rapid decision-making.
Let’s consider a requirement for 3-D urban terrain data available to the Soldier before deployment. First, by the time Soldiers deploy, the standard urban geospatial load may not have the most up-to-date geometries and other relatively static conditions in the area of operations. Second, units may need to know what has changed during the course of combat operations. Accordingly, we must consider the need for an organic capability to rapidly generate new 3-D data to upgrade gaps or other uncertainties in the standard geospatial load. This same function also becomes a change detection capability when comparing new data with existing information.
These two key considerations support sequential, in-stride rehearsal, movement and maneuver, targeting and battle damage assessment: Navigation, targeting and other sensing systems can “see” the real urban environment and compare that, in real time, with urban information on board to move, learn and assess. Think of three tiers in an open modular architecture for 3-D enriched urban terrain information, two of which involve inspecting the operational environment, while the third deals with improving support for decision-making and execution by analyzing data in hand.
At the highest level, we consider the creation of a 3-D, high-resolution geospatial foundation focused on complex and built-up areas. Currently, dedicated and analyst-intensive technologies exist to acquire, store, process, manage and distribute imagery and point clouds, a set of ranging data points that represent the surface of an object as obtained from either active or passive collection modes, from overhead or from ground-based sensing systems. From point clouds, mature exploitation technologies can identify shapes, edges and corners to model the urban objects, and can classify and label objects in the data space. The products have a variety of formats, nearly all requiring significant analyst involvement to make, large volumes to store, considerable cost to analyze, and substantial time delays related to access and visualization.
In the context of training environments, high-level mission planning and other activities, the main near-term research objectives involve designing and integrating a variety of tools for visualizing diverse data sets—such as point clouds, imagery, semantic descriptors and so forth—from diverse collectors. In the farther term, emphasis shifts to consolidating standard analyses and products into a common set of highly compact and manipulable formats.
For training purposes and tactical uses, the gross geometries of urban infrastructure, for example, should have a high level of precision, so that the buildings and other objects look right and building models overlay the real buildings in their actual locations. Important research investigates how to automate the functions of representing infrastructure as compact formats. Another area of research attacks the problem of automatically assigning functions (e.g., residential, commercial, industrial, transportation, etc.) and functional attribution of infrastructure objects and features in military and civilian context. For example, the interior complexity of a building and the building’s use are major factors in the efficiency of precision targeting or clearing.
Other near-term to midterm research seeks to dramatically reduce collection time, particularly since the U.S. cannot forecast with certainty where conflict will require our forces. Other areas of effort, also spanning the near term and midterm, investigate techniques to automatically filter and remove ephemeral features and objects—such as vehicles, people and urban clutter—to produce a time-stable foundation that contains only the horizontal and vertical infrastructure. Researchers also are designing and testing compact and common frameworks for adding back currently observed or projected ephemera, whether natural (e.g., ice, snow, water) or man-induced, like traffic on our day-to-day personal navigation applications.
A LABOR-INTENSIVE PICTURE
This information architecture represents a high-density point cloud, viewed obliquely. A point cloud is a set of points on a coordinate plane; in this case, the data points are the edges of buildings and other structures in this dense urban area. Taken together, they create a 3-D visualization of a space. ARL is looking for a way to collect this kind of data about an urban environment from multiple sources and craft it into a usable map, without the many human analysts currently required to do so. (Images courtesy of the authors)
CAPABILITY AT BRIGADE AND BELOW
When it comes to the next tier of 3-D, high-resolution terrain data generation, management and dissemination, the Army S&T community is exploring the need for tactical capability at brigade and below. In the 2030 to 2040 time frame, the brigade combat team will have a variety of laser, LIDAR and imaging systems for characterizing urban infrastructure. This expectation necessitates an organic capability to improve gaps in a standard geospatial load, to perform change analysis and to collect and overlay real-time information. Urban operations also may need purpose-designed kits for exploration of interior and subterranean spaces by Soldiers or robots.
To deal with all of this effectively and efficiently, the tactical user will need an operational data environment where generation, processing and dissemination of 3-D urban data can remain local for some period, avoiding the delays currently experienced in vetting and validating authoritative data.
As a third and directly actionable tier of a future capability, we can consider the examples of derived information layers and terrain reasoning related to METT-TC, the military acronym for mission, enemy, terrain and weather, troops and support available, time available and civil considerations. Many mature derived-information products have become standards, such as line-of-sight analysis, trafficability (the ability of a vehicle to traverse a specified terrain) and mobility analyses and the identification of helicopter landing zones. For near-real-time information to serve tactical operations, a capability would require accurate georegistration to the foundation data, despite GPS degradation or denial, and precision sufficient to automatically identify movement of people and platforms and an update rate of a few seconds. Movement assessment would require a latency of less than about 10 seconds and an update rate of only a few seconds.
Critical research objectives at the ERDC, ARL and elsewhere in the near term to midterm include developing algorithms and techniques for robust and fully automated collection, buffering, processing and tailored direct distribution, all in a -communications-challenged environment. Near-term research seeks to integrate on-demand and automated processing of products such as these with near-real-time updating, using data derived from battlespace sensing.
FORM, FIT AND FUNCTION
The prospect of 3-D enriched, high-resolution urban terrain with near-real-time updated tactical overlays does not necessarily constitute operational improvement and leap-ahead advantage. We can observe in the world every day the distraction and operational slowing caused by visual displays, personal and otherwise, as well as our dependence on them. To integrate and distill sufficient situational context—METT-TC—so that leaders of small units can make better decisions faster, a relatively new body of research is looking into the form, fit and function of visualization to catalyze a strengthening of intuitive understanding. From training to rehearsal to operational use, visualization requirements differ. With immersive training and research toward a fully synthetic training environment, and with mission planning at brigade and above, research challenges—near-term and enduring—appear well-defined.
For close-quarter combat in complex and especially dense urban environments, questions about what, when and how to visualize the data products described above become paramount. For example, the ability to move at will in dense urban environments and simultaneously force dilemmas on an adversary, as well as to manage risk, may depend on very short-lived multisensory (i.e., audio, visual, tactile) cues that bolster the retrieval and application of spatial memory. Can we train, rehearse and cue a Soldier to navigate in the city as effectively as the native city dweller?
Recent Army research at NSRDEC has demonstrated important trade-offs among the timing and type of information conveyed to a user, the attentional demands of the information and outcomes for individual and small unit performance. If, during mission planning and preparation, Soldiers visualize the intended operating area in 3-D from multiple perspectives and orientations, their spatial memory can improve; this increases their ability to move effectively through complex environments with constantly changing situations and demands on their attention.
NOT A SILVER BULLET
Army research has demonstrated that during combat operations, standard navigational displays can induce complacency, divide attention and disengage navigators from their environment. This can impair the development of flexible spatial memories Soldiers must rely on during times of heightened stress. These and other research outcomes present a challenging focal point for developing next-generation visualization technologies, such as chest-, helmet-, eyewear- and torso-mounted information systems that provide timely and relevant information without compromising the ability to think and act quickly and effectively. The Army’s geospatial, training and Soldier S&T communities are working collaboratively on this challenge, including developing scenario-based virtual test beds to predict and quantify performance outcomes of future systems, the development and application of which span from the near to the far term.
PLAN FROM ALL PERSPECTIVES Data-rich 3-D maps would let Soldiers spend time viewing terrain from a variety of perspectives to gain an intuitive sense of the battlespace before operations begin. That basic understanding of the physical environment and how to navigate it improves spatial memory.
CONCLUSION
With our current technology and doctrine, we can level the playing field in complex and congested environments—including dense urban and megacity domains—by degrading standoff and other advantages. Integrating capabilities like next-generation autonomous networked sensor platforms, heads-up situational awareness for small units and enhanced fusion and targeting has the potential to restore U.S. tactical advantage. Providing rich, detailed and actionable place and event context through analysis and visualization has great promise to give options to tactical commanders among integrated and available capabilities to make our adversaries’ intentions unattainable.
For more information, go to https://www.nsrdec.army.mil/ or http://www.erdc.usace.army.mil/.
ROBERT E. DAVIS is the chief scientist and senior scientific technical manager for geospatial research and engineering at ERDC, headquartered in Vicksburg, Mississippi, and with laboratories in New Hampshire, Virginia and Illinois. He holds a Ph.D. and an M.A. in geography, and a B.A in geology and geography, from the University of California, Santa Barbara.
KEVIN R. SLOCUM provides systems engineering and technical assistance support to the U.S. Army ERDC Cold Regions Research and Engineering Laboratory in Hanover, New Hampshire, and the Defense Advanced Research Projects Agency in Arlington, Virginia. He holds a Ph.D. in marine sciences from the College of William and Mary, an M.A. in applied geography from Texas State University and a B.A. in geography from James Madison University.]
TAD T. BRUNYE is a senior cognitive scientist at NSRDEC, a scientific manager at the Center for Applied Brain and Cognitive Sciences in Medford, Massachusetts, and a visiting associate professor at Tufts University. He holds a Ph.D. and an M.S. in experimental cognition from Tufts University, and a B.A. in psychology from Binghamton University.
AARON L. GARDONY is a cognitive scientist at NSRDEC and a visiting scholar at the Center for Applied Brain and Cognitive Sciences. He holds a Ph.D. in psychology and cognitive science, an M.S. in experimental psychology and a B.A. in psychology, all from Tufts University.
Related Links:
Center for Applied Brain and Cognitive Sciences: http://www.centerforabcs.org/
This article is published in the January – March 2018 issue of Army AL&T magazine.
Subscribe to Army AL&T News, the premier online news source for the Acquisition, Logistics, and Technology (AL&T) Workforce.
