SUMMARY Transition to USOAP Continuous Monitoring Approach (CMA) ICAO Secure websites:

SUMMARY Transition to USOAP Continuous Monitoring Approach (CMA) ICAO Secure websites: USOAP Continuous Monitoring Approach (CMA) Online Framework Secure Portal 1 Transition to Continuous Monitoring Approach (CMA) 1.1 Review of USOAP CMA 1.1.1 The CMA includes four distinct components and associated activities: Collection of Safety Information Determination of State Safety Risk Profile Prioritization and conduct of USOAP CMA activities Update of Lack of effective implementation (LEIs) and status of SSCs See image See Image I 1 1.1.2 Under the USOAP Comprehensive Systems Approach (CSA) a team of auditors would periodically visit a State to conduct a complete audit of the State safety oversight programme. Under the USOAP CMA ICAO will be conducting several main activities, with priorities set following the establishment of a State safety risk profile. 1.1.3 A State safety risk profile is established following review of information such as the implementation status of USOAP corrective action plans, accident and serious incidents, and the level of aviation activity. This analysis is supported by information provided by the State through submission and update of the State aviation activity questionnaire (SAAQ), compliance checklists (CCs), on-line completion of protocol question self-assessment, and the filing of differences, as well as information available to ICAO from other sources. 1.1.4 Following the determination of a State safety risk profile ICAO will monitor State progress and, as appropriate, schedule activities such as an ICAO Coordinated Validation Mission (ICVM), a comprehensive systems audit (CSA) or a Safety Audit 1. Since the full implementation of the USOAP CMA in January 2013 this includes on-line monitoring of information provided by States. 1.2 Transition to Protocol-based Findings 1.2.1 Under the USOAP Comprehensive Systems Approach (CSA), findings could identify one or more PQs. Corrective Action Plans (CAPs) were developed by the State to address each finding, including all PQs listed in that finding. ICAO Coordinated 1 Safety Audit is an activity conducted at the request of a State, on a cost-recovery basis. The Safety Audit employs the same methods and references as a comprehensive systems audit. file:\\coscap GUIDANCE - ICAO SECURE PORTAL AND CMA OLF R2 Page 1 of 16

Validation Missions (ICVM) then reviewed the implementation of each CAP in order to verify (validate) whether or not a finding could be closed. The ICVM could conclude that while certain PQs were successfully addressed others were not. In that case the finding would remain OPEN until all of the related PQs could be closed. 1.2.2 A significant difference introduced under the USOAP CMA and the CMA On-line Framework is that CAPs are prepared for individual PQs, rather than preparing a single CAP for each finding. This will benefit States because the implementation of individual PQs can be validated and closed by ICAO, thereby providing an improvement in the implementation score. Previously, if there were several PQs for a finding, the finding would remain open until every PQ related to that finding was successfully closed. 2 Corrective Action Plans, Compliance Checklists. Self-Assessment 2.1 Corrective Action Plan Implementation Progress Review 2.1.1 Corrective Action Plan (CAP) development and CAP implementation status reporting should be done by personnel in the technical area (e.g., PEL, AGA, etc.): Each technical area, with the assistance of the NCMC, should complete a CAP update for each open PQ. The NCMC may, if needed, review CAP development with COSCAP to obtain further assistance. 2.1.2 The proposed CAP updates should be approved by the Director General prior to uploading by the NCMC onto the ICAO USOAP Continuous Monitoring Approach (CMA) On-line framework. 2.1.3 When considering the development of a CAP, the same approach should be taken in each technical area: Review of the finding; Review of the proposed CAP; Review ICAO comments on the proposed CAP; Discussion of the implementation status of the approved CAP; Review of the related PQs; Review the ICAO requirement and protocol guidance; and, Development of CAP updates for each PQ. 2.1.4 Each CAP should include, as appropriate: the development of draft documentation (e.g., procedures, regulations, etc.); approval of final documents; training State personnel and / or informing the affected aviation industry; and, implementation actions. 2.1.5 Further, remedial action might also be needed to ensure that service providers comply with a revised or new requirement, or if oversight of the existing requirements had not been effectively implemented. 2.1.6 Each protocol question (PQ) includes references to the ICAO requirement that is being examined, as well as guidance on the type of information that will be needed to satisfy the question. The National Continuous Monitoring Coordinator (NCMC) can file:\\coscap GUIDANCE - ICAO SECURE PORTAL AND CMA OLF R2 Page 2 of 16

access and share this information either on-line or with downloaded files. Technical personnel should review the ICAO requirements and guidance for each PQ that they are addressing. 2.1.7 When developing CAPs it is important to carefully consider the wording of the protocol question. For example, a question that begins Does the State ensure that etc. will require three or four parts to respond: 1. A requirement for something from the service provider (e.g., training programme). This is usually a regulation. 2. A STATE procedure that makes sure that the regulatory requirement is addressed in the service provider manual (e.g., a training programme for staff) before the manual is approved by the STATE. 3. A STATE procedure such as conducting surveillance to make sure the service provider actually complies with the manual. 4. Evidence that all of this has been implemented. 2.1.8 It is also important when preparing a CAP to consider the critical element 2 that is provided along with each protocol question. This will indicate the type of action that is needed to address this question. For example: CE-1 implementing or amending the primary aviation legislation CE-2 implementing or amending specific operating regulations CE-3 Defining or revising State System and functions CE-4 Ensuring Qualified Technical personnel, including training CE-5 Addressing Technical guidance, tools and provision of safety critical information CE-6 Completion of Licensing, Certification, authorization / approval actions CE-7 Conduct of Surveillance, inspection, auditing, oversight CE-8 Ensuring the Resolution of safety concerns. 2.1.9 Partially Acceptable and Not Submitted The submission of revised corrective actions in order to fully address the ICAO comments on the original CAPs should be a priority. It is important that the opportunity of updating CAPs is used to demonstrate how the requirements of the related protocol questions are fully addressed. 2.2 Compliance Checklists and Electronic Filing of Differences 2.2.1 As part of the USOAP programme, each State has agreed to keep its compliance checklists for the Annexes to the Chicago Convention up to date. 2.2.2 The compliance checklists (CCs) assist each State to verify that it has appropriately considered each current ICAO standard and recommended practice. The on-line CCs are the foundation for filing differences (EFoD), avoiding a duplication of work. 2.2.3 To support review of the CCs by technical personnel and State management, they should be prepared as an off-line file. The NCMC can download the CCs into a 2 See Annex 19 Safety Management, s.3.2 and Appendix 1 State Safety Oversight System file:\\coscap GUIDANCE - ICAO SECURE PORTAL AND CMA OLF R2 Page 3 of 16

standard MS Word format file. Once they have been updated and approved by Management, the NCMC can upload the information to ICAO. 2.2.4 Once uploaded, any differences that must be notified to ICAO can be submitted by simply using the submit differences feature. 2.3 Self-assessment of Protocol Questions 2.3.1 The CMA on-line framework provides the ability for State to complete a self-assessment of its safety oversight programme in comparison to ICAO requirements and guidance. This will permit State to proactively ensure the compliance of its programme, independently of any USOAP activity conducted by ICAO. 2.3.2 As noted above at section 1.1.3, ICAO will use the self-assessment information provided by State as a factor in determining State s safety risk profile. Timely completion of the PQ self-assessment by STATE is an important element of the USOAP CMA. 3 CMA On-line Framework and ICAO Secure Portal ICAO is using two principal on-line points to share information: the CMA On-line Framework and the ICAO Secure Portal. Access to these areas is controlled and available to State personnel under individual user identification and password. Access rights are individually assigned according to the needs of the State users. 3.1 USOAP CMA On-line Framework 3.1.1 The USOAP CMA Online Framework is accessed at http://icao.int/usoap/ The CMA On-line Framework is the primary means for a State to provide updated information to ICAO, including the SAAQ, compliance checklists, electronic filing of differences (EFoD), USOAP corrective action implementation and self-assessment. 3 3.1.2 The CMA On-line Framework (image II 1) includes access to: State dashboard (summary) SAAQ Protocol questions Corrective Action Plan management Compliance Checklists and Electronic filing of Differences USOAP reports for all audited States Significant Safety Concerns 3.1.3 Access to this site should be widely available to State personnel including Directors, Inspectors and administrative officers, with individual access permissions established according to their specific needs. For example, while Directors and senior management might wish to have access to read information, it is probable that the responsibilities for actually updating State data will be assigned to subordinate 3 istars is no longer used to update corrective action plans file:\\coscap GUIDANCE - ICAO SECURE PORTAL AND CMA OLF R2 Page 4 of 16

personnel. In such cases, senior managers would have read access while trained subordinate personnel would have the access rights to update State information. 3.1.4 As an example of use, Inspectors with responsibilities for the approval and surveillance of foreign air operators would use the CMA On-line Framework to obtain USOAP report information, including any significant safety concerns, for the State of the operator. Understanding the safety oversight performance of the State of Operator is an important first step in assessing an application from a foreign operator wanting to enter State. 3.1.5 The ICAO internet security management system does not permit sharing of userid and passwords. This means that the access rights to update information are assigned directly to the personnel trained to do this, and not shared from a senior manager. 3.1.6 Training and help for the use of the CMA Online framework is available from the main page (Image II 1) by following the link [Tutorials & Help]. This will lead to the page shown at Image II 2. 3.1.7 To manage and control the update of State information, the State should establish a process so that proposed updates are approved by Management prior to being uploaded. This would be completed outside of the CMA framework, possibly using a spreadsheet file to develop and review proposed updates. The updates could then be entered into the CMA online framework using a copy & paste method. 3.1.8 The National Continuous Monitoring Coordinator (NCMC) may create new users and grant access rights as needed (read only, or read / write). This is done under the ACCESS CONTROL icon. 3.1.9 It will be important for the State to ensure that the information it submits via the CMA on-line framework is complete and up to date. ICAO will consider this information when it determines a risk profile for State. 3.2 ICAO Secure Portal 3.2.1 The ICAO Secure Portal is accessed at http://portal.icao.int/ 3.2.2 The ICAO Secure Portal (see image IV - 1) is used to provide access to information and applications for State personnel. Some applications, such as the ICAO-NET, are an on-line library that should be routinely available to a wide range of STATE personnel. Other applications such as, for example, the istars (see image IV - 2) and the Aircraft Registration System (ARS) should be widely accessible to Inspectors for read-access according to their assigned responsibilities. Editing (i.e., updating) permissions will also be provided to those personnel with specific responsibilities and training to manage this on behalf of the STATE and under management authority. For example, Operations and Airworthiness Inspectors responsible for the approval and surveillance of foreign air operators would be provided read-access to istars, ARS and the Air Operator Certificate (AOC) database (to be released soon by ICAO). Officials with responsibilities for managing the register of State civil aircraft would also have permissions to edit the ARS. (See image at Appendix III OASIS) file:\\coscap GUIDANCE - ICAO SECURE PORTAL AND CMA OLF R2 Page 5 of 16

3.2.3 To apply for access to the Secure Portal enter http://portal.icao.int/ and follow the link Request an Account 3.2.4 On initial registration from an account you will need to know the code for your State s Group name. This will be a code that begins with ST_ followed by two or three letters that designate your State. 3.2.5 To obtain the group name for your State contact your NCMC or Group Manager. If you do not know who these are, please contact your COSCAP for assistance. 3.2.6 Enter the Group name into the Group Subscription Request window (See Image IV-3). After this, you will receive a screen that requests name and position information (Image IV-4). Of particular importance is the question: Justification. Enter the reason you need access. Example: I am an Airworthiness Inspector and require access to current ICAO documents. 3.2.7 Once the application form is completed, click on Submit Request. Your application will be sent to your State s Group Manager. The Group Manager makes the decision to grant access. 3.2.8 Once access is granted, the user will receive an email from ICAOPortalAdmin@icao.int providing a userid and a password. 3.2.9 The initial access to the Secure Portal will provide access to ICAO-NET. To access other secure sites (e.g., istars, ARS) you will need to subscribe to each group. (See Image IV-6) Note that it is important to include a justification for each group. Example: I am an inspector responsible for surveillance of foreign air operators and I need access to istars to review foreign State USOAP results. 3.2.10 The request will be reviewed by ICAO headquarters and the user will receive a message from ICAOPortalAdmin@icao.int confirming access. 4 Recommendations R01 R02 R03 R04 ICAO-NET and istars Read access should be provided widely within the STATE. ICAO-NET and istars should be considered as an electronic library, providing easy access to a range of ICAO publications, Annex amendments and reference material. Read access should be provided to individual users, under their own user identification and password. CMA Online Framework Access (read-only and editing) should be provided as appropriate to assigned responsibilities. This will permit reviewing, for example, USOAP reports and significant safety concerns for other States, or entering updates for State s SAAQ, compliance checklists / electronic filing of differences. States should update its SAAQ, State Profile and compliance checklists. This should be undertaken in a methodical manner, and assigned to specific individuals under an implementation plan. States should undertake a methodical completion of the PQ self-assessment once the priority work to update the implementation status of corrective actions is completed. file:\\coscap GUIDANCE - ICAO SECURE PORTAL AND CMA OLF R2 Page 6 of 16

R05 R06 R07 States should identify lead personnel in each technical area to work with and support the NCMC. These personnel should initially be provided read access to the CMA On-line framework. Detailed training for the use of the on-line system should be provided to State personnel responsible for entering information such as SAAQ, compliance checklists, and developing and updating corrective actions. COSCAP can support this training. The user profile for each State person having access to the CMA On-line framework should use a unique userid for each user. The individual userid should be related to the user s name (e.g., John SMITH might use jsmith or johnsmith). file:\\coscap GUIDANCE - ICAO SECURE PORTAL AND CMA OLF R2 Page 7 of 16

Appendix I ICAO USOAP Continuous Monitoring Approach (CMA) Components Image I - 1 file:\\coscap GUIDANCE - ICAO SECURE PORTAL AND CMA OLF R2 Page 8 of 16

Appendix II USOAP CMA On-line Framework https://soa.icao.int/usoap/ Image II 1 file:\\coscap GUIDANCE - ICAO SECURE PORTAL AND CMA OLF R2 Page 9 of 16

Appendix II USOAP CMA On-line Framework Tutorials & Help Image II 2 file:\\coscap GUIDANCE - ICAO SECURE PORTAL AND CMA OLF R2 Page 10 of 16

Appendix III Online Aircraft Safety Information System OASIS Image III-1 file:\\coscap GUIDANCE - ICAO SECURE PORTAL AND CMA OLF R2 Page 11 of 16

Appendix IV ICAO Secure Portal (typical) http://portal.icao.int/ Image IV- 1 Image IV-2 file:\\coscap GUIDANCE - ICAO SECURE PORTAL AND CMA OLF R2 Page 12 of 16

Appendix IV Image IV-3 Image IV-4 file:\\coscap GUIDANCE - ICAO SECURE PORTAL AND CMA OLF R2 Page 13 of 16

Appendix IV Image IV-5 Image IV-6 file:\\coscap GUIDANCE - ICAO SECURE PORTAL AND CMA OLF R2 Page 14 of 16

Appendix V istars Image V-1 file:\\coscap GUIDANCE - ICAO SECURE PORTAL AND CMA OLF R2 Page 15 of 16

Appendix V istars Compliance Page Image V-2 file:\\coscap GUIDANCE - ICAO SECURE PORTAL AND CMA OLF R2 Page 16 of 16