Test & Evaluation/Science & Technology Program Testing Domains Supporting SoS T&E C4I & Software Intensive Systems Test (C4T) Test Technology Area (TTA) 13 December 2018 Mr. Kenneth Sanchez Phone: 805-989-0785 Email: Kenneth.s.sanchez@navy.mil
C4I & Software Intensive Systems Test (C4T) Mission Develops technologies to test C4I and Software Intensive Systems that operate in complex military environments. With emphasis on testing warfighter systems more thoroughly through earlier T&E in the acquisition lifecycle; improved interoperability of the DoD ranges and labs; and improved efficiency of DoD T&E resources. C4T is divided into the following three domains: 1. Test Automation: This domain addresses automation technologies to assess C4I and Software Intensive Systems within the Joint Information Environment Net-Centric Operational Environment (Theater to Tactical Edge). Test automation technologies will allow more thorough and optimized testing of such warfighter systems. 2. Distributed Testing: This domain addresses technologies to facilitate test quality environments utilizing a distributed infrastructure consisting of Services Major Range and Test Facility Bases. 3. Modeling & Simulation: This domain addresses technologies to increase the use of modeling and simulation for test and evaluation across a distributed infrastructure consisting of live and constructive warfighter systems/ environments; and validation of this infrastructure to test quality standards. 2
Innovative approaches to how we fight, posture our force, and leverage our asymmetric strengths and technological advantages* C4T Overview C4T Overview Complex Warfare Environments Battle increasingly sophisticated adversaries in increasingly complex environments* BIG DATA & CLOUD COMPUTING Modeling & Simulation (M&S) Test Automation (TA) Advance Big Data Collection, Analysis & Visualization Improve Test Execution thru Automation/Cloud Computing Advance Testing for Next Generation of Handhelds Improve Automated Control of Targets Distributed Testing (DT) Reduce Distributed T&E Effects Improve Security Posture of Systems Advance Mission Context Data Collection Analysis & Visualization Innovate T&E: Joint, Early, Often, and Agile Required Fidelity in Live and Simulated Environments V&V Techniques Aggregation Techniques Run-time Performance for RT Applications Systems, Communications, Environmental Representations 3
C4T Definitions Command, Control, Communications, Computers, and Intelligence Systems (C4I): Support net-centricity servicebased architecture pattern for information sharing via building joint architectures and roadmaps for integrating joint airborne networking capabilities with the evolving ground, maritime, and space networks. Software Intensive Systems (SIS): Increasingly complex systems characterized by thousands of platforms, sensors, decision nodes, weapons, and warfighters connected through heterogeneous wired and wireless networks. 4
C4T TTA Test Automation Domain 5
C4T: Test Automation Test Use Case 1: Improve Automated Data Collection, Visualization, Analysis & Reporting VIDEO CHAT UNSTRUCTURED DATA Use Smart Algorithms & Database Design Technologies to Structure Data TEXT STRUCTURED DATA VOICE SCREENS TACTICAL TSPI C2 Operations/Analysts Advanced Semantic Algorithms SEARCH: TARGET LOCATION Parallel Search Engine Technology Test Conductor Improve the test communities ability to analyze and make decisions from large volumes of structured and unstructured test data. 6
C4T: Software Intensive Systems Testing Test Use Case 2: Automated Testing in a Cloud Environment Automated Testing in a Cloud Environment Executing 1000 s of Test Case Scenarios in Parallel Software Intensive Systems (SIS) Typically Operate Within a System-of-System (SoS) to Assess Effectiveness Test Harness Technologies Support automated/agile testing (individual system) Minimize labor-intensive test procedures Support of regression, developmental, operational, and interoperability testing Virtualized HW Aspects of System(s) Under Test Virtual Test Harness Virtual Test Harness System Under Test Virtual Test Harness Virtual Test Harness Automated Tools For Testing System Effectiveness & Performance New virtualization and cloud-based test technologies are needed to automatically create, execute, and control a large number of test cases operating in parallel 7
C4T: Test Automation Test Use Case 3: NextGen Handhelds and Widgets 8
C4T: Test Automation Test Use Case 4: Automate Control of Tightly Maneuvering Attacking Targets Threat Air Swarm Agile Communications A2AD C2 Big Data Systems Under Test Target Control Stations Threat Land Swarm Threat Sea Swarm Coordinate and choreograph multiple targets with fewer operators 9
C4T TTA Distributed Testing Domain 10
C4T: Distributed Testing Test Use Case 5: Remove Test Infrastructure Biases Joint Mission Environment TACTICAL DATA INTEL DATA C2 DATA Joint Information Environment Target Positional Error: synchronization, terrain, and network Terrain Error: Tank in the Air Joint Information Environment Distributed Test Infrastructure Network Error: Latency & Packet Drop T&E Instrumentation Live Warfighter Systems Virtual and Constructive Systems/Environments Test Control & Analysis Improve distributed test infrastructure through advanced synchronization algorithms Distribution Statement to remove A. Approved test biases for public release: distribution unlimited. 11
C4T: Distributed Testing Test Use Case 6: Cross Domain Solutions and Multi-Level Security Multi-Level Security Joint Mission Environment CDS Test Control & Analysis CDS CDS CDS Distributed Test Infrastructure T&E Instrumentation Live Warfighter Systems Virtual and Constructive Systems/Environments Improve testers ability to handle multi-level security (data at rest) and cross domain solutions (multiple data classifications in motion) 12
C4T: Distributed Testing Test Use Case 7: Assess Big Data Warfighter Systems Test Control Test Control & Analysis Distributed Common Ground System (DCGS) - C4I cross Service System employing Data-to- Decisions Data generators and simulators for Velocity, Variety, and Volume Platform analytics & fusion assessments Big Data environment replication Effectiveness and Performance of Individual system within SoS Automate regression testing Joint Mission Environment Distributed Test Infrastructure T&E Instrumentation Live Warfighter Systems Virtual and Constructive Systems/Environments Improve ability to create and assess big data warfighter systems under emulated wartime modes of operation 13
C4T: Distributed Testing Test Use Case 8: Agile Communications & Contested/Dense Communications Environments Joint Mission Environment Evolving/Agile/Complex Warfighter Communications Networks Anti-Access/ Area Denial Joint Aerial Layer Network Moving Target Defenses Test Control Test Control & Analysis Contested/Dense Communication Networks Stimulation/Emulation to Replicate: Denied/degraded communications, Urban communications Cell Tower Command & Control Dense Urban Environment Distributed Test Infrastructure T&E Instrumentation Live Warfighter Systems Virtual and Constructive Systems/Environments Improve distributed test environments ability to assess agile communication networks and Contested/Dense Communication Environments at wartime modes of operation 14
C4T TTA Modeling & Simulation Domain 15
C4T: Modeling & Simulation Test Use Case 9: Battlespace Environments Aggregation & Determining Required Fidelity 75 Kilometers Low Resolution Objects in Mechanized Infantry Battalion 50 Kilometers High Resolution - Armored Personnel Carrier (50) - Wheeled Personnel Vehicle (10) - Ground Missiles (30) - Mortar (10) - Personnel (500) - Approx. Tactical Objects (600) Airborne Sensor Under Test Forces on Test Range Low Resolution High Resolution Representing a large numbers of target objects with tactical relevancy (200-300 high-resolution entities in a real-time test execution) Develop analytical aggregation techniques that will allow scaling of the test environment, while maintaining SUT required fidelity and update rates. 16
C4T: Modeling & Simulation Test Use Case 10: M&S for T&E of Systems in Complex Environments (Representations and V&V) Sensor Platform/ Aircraft Movement Simulation Update Rate (200 msec) Simulated Test Environment of Battlefield Robustness Rocket Fly Out Simulation Update Rate (100 msec) Non-Continuous Terrain Generator 100 Meters Per Grid Tile Rocket Sensor Guidance Software System Under Test (SUT) Ground Target/Vehicle Movement Simulation Update Rate (1 sec) Non-Continuous Sensor Update Rate (50 msec); Guidance Update Rate (100 msec); Accuracy 6m (CEP) Mature technologies that increase the scope of mission performance testing, that could occur across the Department of Defense (DoD) distributed T&E infrastructure. 17
C4T: Modeling and Simulation Test Use Case 11: M&S for T&E of Systems in Complex Environments (Run-Time Performance) High Fidelity Simulated Environments & Systems Distributed M&S Infrastructure for T&E Live Environments & Systems T&E Instrumentation Warfighter Systems Virtual and Constructive M&S M&S Technologies for Torpedo Testing Improve Simulation Runtime Performance 18
Closing C4T TTA investing in technologies to support SoS T&E Test Automation Distributed Testing Modeling and Simulation 19