Enhancing our Competitive Edge: T&E in a Time of Risk and Change Maj Gen Matt Zap Molloy Commander, Air Force Operational Test and Evaluation Center Albuquerque, NM Distribution A: Approved for public release; distribution is unlimited 1
Purpose Discuss why it is imperative that we continue to advance & adapt our testing acumen to maintain relevance in a changing acquisition environment and to also reduce warfighting risk 2
Overview CHANGE Agile and Rapid Acquisition n Testing: Some OT (DT, CT, IT!) Thoughts RISK Associated with our Space Systems, Specifically Space Threat Testing Closing Thoughts Q&A 3
Change Rapid & Agile Acq & Test Old wine in new bottles? Old wine in old bottles?? So what s different from the past bumper stickers?? 4
Cumulative effect of: Pace of Technology Rapid Acq and the Speed of Need Its nature & manifestation Moving from wars of attrition to wars of cognition The Threat Declining Resources SECDEF s Reformation of DoD Business Practices Think big, Think differently. Good luck The Nature of the Next-Generations Weapons Portfolio 5
Human-Machine Collaboration Network-Enabled, Cyber-Hardened Weapons 6
Traditional T&E Construct Speeding things up : Milestones C A B C Full Rate Production Decision Review Requirements Requirements Documents; Increased Detail and Complexity Acquisition Contracting Actions Requirements Documents; Increased Detail and Complexity Testing Developmental/Operational Testing IOC Contractor Test Developmental test activities OT&E 7
Rapid / Agile Training Wheels F-35 IOT&E ALIS & OFP releases Pre-IOT&E Activities IOT&E Activities Sept 17 Jun 18 MEI : Modernization Enhancements & Improvements 2019 OFP 3FR6.3 Basic Release OFP MEI SW Release OFP MEI SW Release Finish SDD OT&E-Identified Deficiencies Fix Deficiencies OT Verify Fixes ALIS 2.0.2 ALIS 3.0 ALIS 3.0.x 8
Rapid & Agile T&E Construct F-35 Follow-on Modernization (FoM): Transitioning to "Continuous 25 Capability Development and Delivery" (C2D2) Capability drops 5 7 9 9 11 13 15 17 19 21 23 Traditional OT&E: Significant Capability Oversight ACAT 1 3 Integrated Test (CT/DT/OT) MAJCOM-Level? OT&E Integrated Test (CT/DT/OT) MAJCOM-Level? Baseline Capability 2019 2020 Time 2021 2022 2023 Leverage non-traditional acq & test processes such as Agile (software) Switch between waterfall (traditional) or Agile as situation dictates Use RALOT-type analysis process to determine Level of OT Major capability drops, net-cumulative increases, new mission areas = Independent OT&E 9
Rapid & Agile T&E Construct F-35 C2D2: April 19 Oct 19 April 20 Oct 20 Hardware install Hardware install A/C Mod JSOW DRs (deficiency fixes) Countermeasure (CMD) Enhancements DRs PVI Improvements Weapons Enhancements DRs Weapons Enhancements DRs CT DT OT Cyber AA AA Integrated CT/DT/OT testing from the very beginning shift left (see AFI 99-103 encouraged) Joint approach with OT activities: Divvyed up between Services, much delegated to MAJCOM-level Leverage Combined Test Force OTA/DOT&E involvement & overwatch Independence DOES NOT mean isolation Advance the enterprise-level Cybersecurity Strategy Move more cyber testing into CT/DT; maximize use of cyber ranges 10
F-35 C2D2: Rapid & Agile T&E Construct April 19 Oct 19 April 20 Oct 20 Hardware install Hardware install A/C Mod JSOW DRs (deficiency fixes) Countermeasure (CMD) Enhancements DRs PVI Improvements Weapons Enhancements DRs Weapons Enhancements DRs CT DT OT Cyber AA AA Agile still must have clear, articulate performance requirements Articulate the performance floor and allow Moore s Law & serendipity to define the ceiling CoG = crafting the contract with manufacture/developer Test-driven approach: Early, repeatable testing is key; Fail forward : Test, fail, code, test, pass <rinse, repeat> release Automated testing Knowledge Mngt (KM) tools.make EVERY jet in Fleet/CAF a data provider! 11
Rapid & Agile T&E Construct Modeling and Simulation Key to success Maximize Joint Simulation Environment at PAX and Nellis A foundational element of risk reduction Must be a lead activity within the development cycle HW/SW-in-the-loop Must support cross-platform, multi-domain, multi-security, mission-level model environment 12
RISK Space, the Final Frontier 13
LEO, MEO, HEO, GEO, and Terrestrial: They re ALL vulnerable to the threat 14
2007 ASAT.A Chinese Space Odyssey 15
CRAFTING A VIABLE SPACE STRATEGY: - THE SPACE ENTERPRISE VISION ESV Force packages multi-platform resiliency Faster delivery cycles Commercial partnering SEARCHING FOR THE MOODY SUTER OF SPACE 16
We shake em, bake em, but what about robust threat testing??
Need to invest in a space test architecture A Nevada Test and Training Range (NTTR) for space?? The T&E community is coming together to advocate for dollars and resources to build Space M&S and test articles to replicate: Jamming Nuclear scintillation in the near term Common toolset Non-stovepiped testing tools Cyber Terrain & Space Domain: They re not the same, but they rhyme PDRR Protect, Detect, Respond, Restore Perhaps a helpful cognitive model for approaching space threat testing 18
Modeling & Simulation Integrated LVC Across Domains Space & Counter Space Space Threat Space Airborne C2 & BMC2 Kinetic & Non-Kinetic Weapons/Effects Emissions (Jamming, Inadvertent) Active (RF, Laser, etc.) Multi-Spectral Sensing & Emissions 5 th to 4 th Interactions Communication Air Integrated Sensing & Communication Architectures Access Storage TE Processing Reach Back Combat Cloud Future 5 th Gen Manned & unmanned (autonomy) Other source data Cyber Weather Cyber Current & Future Threat Air Red Threats C2/ISR C2/AOC JSPOC EW, TAR, TTR, SAMS, GCI, C2 DCGS COP C2/ISR & BMC2 Meta Correlation & Fusion Threat IADS 19
Questions & Comments 20