Sponsored by the U.S. Department of Defense 2006 by Carnegie Mellon University CMMI: The DoD Perspective Rick Barbour Chief Engineer Navy, Acquisition Support Program page 1
Report Documentation Page Form Approved OMB No. 0704-0188 Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. 1. REPORT DATE 2006 4. TITLE AND SUBTITLE CMMI: The DOD Perspective 2. REPORT TYPE 3. DATES COVERED 00-00-2006 to 00-00-2006 5a. CONTRACT NUMBER 5b. GRANT NUMBER 5c. PROGRAM ELEMENT NUMBER 6. AUTHOR(S) 5d. PROJECT NUMBER 5e. TASK NUMBER 5f. WORK UNIT NUMBER 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) Carnegie Mellon University,Software Engineering Institute,5000 Forbes Avenue,Pittsburgh,PA,15213-3890 8. PERFORMING ORGANIZATION REPORT NUMBER 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR S ACRONYM(S) 12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release; distribution unlimited 13. SUPPLEMENTARY NOTES 14. ABSTRACT 11. SPONSOR/MONITOR S REPORT NUMBER(S) 15. SUBJECT TERMS 16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF ABSTRACT a. REPORT unclassified b. ABSTRACT unclassified c. THIS PAGE unclassified Same as Report (SAR) 18. NUMBER OF PAGES 21 19a. NAME OF RESPONSIBLE PERSON Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18
Acknowledgement Presentation used with permission of Brian Gallagher Director, Acquisition Support Program, Software Engineering Institute 2006 by Carnegie Mellon University 2
DoD s Software Challenge DoD estimates that it spends about 40% of its RDT&E budget on software - $21B for FY2003 GAO F/A-22 SBIRS-High [Software] continues to grow in importance in our weapons systems - and remains a significant contributor to program cost, schedule and performance shortfalls. -- Pete Aldridge 2006 by Carnegie Mellon University 3
Today s Development Challenges Huge system/software engineering endeavors in aircraft, space vehicles, command and control, ground infrastructure, battle management, etc Several million SLOC programs Hybrid systems combining legacy re-use, COTS, new development Multi-contractor teams using different processes; Dispersed engineering & development locations New technologies/products rapid change and evolution; are they mature; obsolescence Business/operational needs change - often faster than full system capability can be implemented Skillset Shortfalls; Cost and schedule constraints Demands for increased integration, interoperability, system of system capabilities 2006 by Carnegie Mellon University 4
18000 Increasing System Complexity F/A-18E/F 17,101K JSF UAVs NCW Inter-System Operability 15000 F/A-18C/D SMUG/ RUG 14,268K 12000 KSLOC 9000 6000 3000 A-4 (ARBS) 16K F-14 80K E-A6B ICAP1 48K A-7E 16K A-6E 64K F/A-18 Night Attack 3054k F/A-18C/D XN-8 EA-6B ICAP2 6,629K BLK 86 779K F/A-18C/D 2130K AV-8B Radar 3,748K F-14D F-14B 364K 4160K F-14B 2866K EA-6B ICAP2 AV-8B Night EA-6B ICAP2 BLK 82 395K F/A-18A/B 943K Attack 1780K BLK 89 2203K AH-1 764K AH-1 NTS 1000K AV-8B 764K A-E SWIP 364K 0 66 70 74 78 82 86 90 94 98 02 Aircraft IOC, Year 06 10 2006 by Carnegie Mellon University 5
Capability Delivered in Software 100% % of functionality Software provides 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% F-22 B-2 F-16 F-15 F-111 F-4 A-7 1960 1964 1970 1975 1982 1990 2000 Ref: Defense Systems Management College 2006 by Carnegie Mellon University 6
Software is Even in Bullets! 150K SLOC - Weapon 2K SLOC - Ammunition Ada Wide Area Munition Infantry Combat Weapon 130K SLOC Ada, C++, C, Assembly 2006 by Carnegie Mellon University 7
And Software Connects Systems 2006 by Carnegie Mellon University 8
Environment Commitment Operational Need/Advocacy Predictable Performance Requirements Direction Operational Insight New Capabilities 2006 by Carnegie Mellon University 9
Introduction: Current Environment Providing enhanced capability to the warfighter is a complex and conflict-ridden endeavor. Operational forces demand war-winning systems. They need evolutionary enhancements to existing systems to maintain a cutting edge on the battlefield. Acquirers need to maintain cost, schedule, and technical baselines to uphold their duty as stewards of the taxpayers money and to satisfy oversight requirements. Contractors need to win contracts to stay in business and sustain the industry base. Underpinning these conflicts is an ever-increasing demand on systems and software engineering to solve the complexities of an interconnected battlespace. 2006 by Carnegie Mellon University 10
The Acquirer s Job Operational Need Requirements Management What are the key activities you perform when you acquire systems? Configuration Management Risk Management Verification and Validation Program Integration Project Planning Need to counter these attitudes: I'd rather have it wrong than have it late. Industry senior manager Ad hoc, catch as you can that s our motto. PMO We do not work problems until they re unrecoverable. PMO I don t want an ATAM [to reveal problems] on my watch. PMO 2006 by Carnegie Mellon University 11
Visibility into the Team s Capability Operational Need Acquirer CMMI-AM or CMMI-ACQ Acquisition Planning RFP Prep. Solicitation Source Selection Program Leadership Insight / Oversight System Acceptance Transition Developer Plan Design Develop Integrate & Test Deliver CMMI-SE/SW/IPPD/SS 2006 by Carnegie Mellon University 12
The Team Contractor A ML 3 My Program Contractor B ML 4 Contractor C ML 5 Acquirer ML? CMMI Math: 3 + 4 + 5 +? =? 2006 by Carnegie Mellon University 13
DoD s Problem Statement Many DoD contractors advertise high levels of process capability or organizational maturity as measured by either the Continuous or Staged representations of Capability Maturity Model Integration, yet from the perspective of acquisition program managers on some high visibility individual programs, strong systems engineering and project management practices still appear to be lacking. 2006 by Carnegie Mellon University 14
Example Large DoD program with multiple, geographically dispersed engineering locations. Multi-contractor teams (10+) using different processes. Several million lines of code. Systems engineering challenges. Combination of legacy, re-use, COTS integration and new development. All contractor sites are Maturity Level 3 or higher. 18 months after contract award, the program office conducted a CMMI Class B appraisal on the team. 2006 by Carnegie Mellon University 15
Characterizing Results 30 25 20 15 10 5 0 Project Mgmt Number of Strengths Engineering Processes Process Mgmt Number of Weaknesses Support Processes Project Mgmt Processes: - Project Planning - Project Monitoring & Control - Integrated Project Mgmt - Risk Management Engineering Processes - Requirements Mgmt - Requirements Definition - Technical Solution - Product Integration - Verification (Peer Reviews) Support Processes - Measurement & Analysis - Product and Process Quality Assurance - Configuration Mgmt - Decision Analysis Process Mgmt - Organizational Process Focus - Organizational Process Definition 2006 by Carnegie Mellon University 16
Issues Identified - Program Management Lack of project plans or having only incomplete, conflicting or out of date project plans Ineffective use of Integrated Master Schedule as basis for planning/tracking status across program Undefined engineering and management processes on program Inability to track and manage actions to closure Inadequate cost estimation processes, methods, data and tools Inadequate staffing and training project personnel Tracking dependencies between or across teams not defined Managing project data ad hoc Inability to proactively identify and manage risks 2006 by Carnegie Mellon University 17
Issues Identified - Engineering Lack of understanding of the program s requirements Inability to trace requirements to architecture/design or to test plans/procedures Poor linkage of functional and performance requirements Inconsistent requirements management at different levels No criteria for making architectural/design decisions among alternatives Not capturing entire technical data package (requirements, design and design rationale, test results, etc) Efficiency of design process/methods in question Late definition of integration and test procedures 2006 by Carnegie Mellon University 18
Issues Identified Support Processes Difficult to identify items in configuration management baselines Lack of ability to manage individual versions in incremental development Inability to effectively managing changes to work products throughout lifecycle Not conducting audits to establish/ensure integrity of baselines throughout incremental engineering and development Inefficient change management process (cycle time, volume of changes) Quality Assurance audits of products and processes not consistent QA involvement in system and software engineering processes not consistent No metrics to manage engineering activities (outside of cost/schedule data) 2006 by Carnegie Mellon University 19
CMMI v1.2 Part of the Solution! Increasing the integrity and credibility of the model Emphasizing project start-up and process deployment Increasing the integrity and credibility of the appraisal process Raising the bar for SCAMPI Lead Appraisers CMMI is a key enabler as the DoD acquires increasingly complex capabilities and systems 2006 by Carnegie Mellon University 20
Contact Information Rick Barbour Chief Engineer Navy, Acquisition Support Program Software Engineering Institute 4500 Fifth Ave. Pittsburgh, PA 15213-3890 (412) 268-7157 reb@sei.cmu.edu Acquisition Support Program: Director: Brian Gallagher bg@sei.cmu.edu Air Force:: John Foreman jtf@sei.cmu.edu Army: Cecilia Albert cca@sei.cmu.edu Intelligence Community: Rita Creel rc@sei.cmu.edu Civil Agencies: Steve Palmquist msp@sei.cmu.edu http://www.sei.cmu.edu/programs/acquisition-support/ 2006 by Carnegie Mellon University 21