ANNUAL REPORT ON OVERSIGHT

Transcription

1 ANNUAL REPORT ON OVERSIGHT Issued April 11, 2018

2 Copyright 2018 by American Institute of Certified Public Accountants, Inc. New York, NY All rights reserved. For information about the procedure for requesting permission to make copies of any part of this work, please with your request. Otherwise, requests should be written and mailed to the Permissions Department, 220 Leigh Farm Road, Durham, NC

3 AICPA Peer Review Board Annual Report on Oversight TABLE OF CONTENTS Page Introduction i Letter to the AICPA Peer Review Board 1 2 Peer Review Board Oversight Process 3 12 Exhibits 1. Administering Entities Approved to Administer the 2016 AICPA PRP On-Site Oversights of Administering Entities Performed by AICPA Oversight Task Force Observations From On-Site Oversights of Administering Entities Performed by AICPA Oversight Task Force Comments From RAB Observations Performed by AICPA Peer Review Program Staff and OTF Members Material Departures From Professional Standards Identified by SMEs Overall Nonconforming Identified During 2016 Enhanced 21 Oversights by Firm Size 7. Nonconforming Identified During 2016 Enhanced 22 Oversights Random Sample by Firm Size 8. Administrative Oversights Performed by Peer Review Committee of 23 Administering Entities 9. Summary of Oversights Performed by Administering Entities Summary of Reviewer Resumes Verified by Administering Entities 25 Appendix 1. History of Peer Review at the AICPA AICPA Peer Review Program Overview Glossary 30 33

4 Introduction Purpose of This Report The purpose of this Annual Report on Oversight (report) is to provide a general overview, including statistics and information, of the results of the AICPA Peer Review Program (AICPA PRP) oversight procedures and to conclude whether the objectives of the AICPA Peer Review Board s oversight processes performed in calendar year 2016 were in compliance with the requirements of the program. Changes to 2016 Oversight Report As a result of the transition to the Peer Review Information Management Application system, the software program is currently unable to generate certain quantitative statistics that were included in previous reports. Accordingly, this report only includes the results of AICPA PRP oversight procedures performed in calendar year Overall AICPA PRP statistics for 2016 are not included in the report. i

5 AICPA Peer Review Board Annual Report on Oversight Letter to the AICPA Peer Review Board To the members of the AICPA Peer Review Board: We have performed comprehensive oversight procedures during the 2016 calendar year. In planning and performing our procedures, we considered the objectives of the oversight program, which state that there should be reasonable assurance that (1) administering entities (AEs) are complying with the administrative procedures established by the Peer Review Board (PRB) as set forth in the AICPA Peer Review Program Administrative Manual; (2) the reviews are being conducted and reported upon in accordance with the AICPA Standards for Performing and Reporting on Peer Reviews (Standards); (3) the results of the reviews are being evaluated on a consistent basis by all AE peer review committees; and (4) the information provided via the internet or other media by AEs is accurate and timely. Our responsibility is to oversee the activities of AEs that elect and are approved to administer the AICPA Peer Review Program (AICPA PRP), including the establishment and results of each AE s oversight processes. As a result of the transition to the Peer Review Information Management Application, the software program is currently unable to generate complete program statistics for reviews performed in This report only includes the results of AICPA PRP oversight procedures performed in the 2016 calendar year. Our procedures were conducted in conformity with the guidance contained in the AICPA Peer Review Program Oversight Handbook and included the following procedures: Oversight visits of administering entities. Visits to the AEs, on a rotating basis, ordinarily every other year, by a member of the Oversight Task Force (OTF). The visits included testing the administrative and report acceptance procedures established by the PRB. OTF members visited 21 AEs in See pages 3 4, Oversight Visits of the Administering Entities. Report Acceptance Body (RAB) observations. RAB observations are performed by OTF members and AICPA PRP staff. The RAB observations increase the probability that the report acceptance process is being conducted in accordance with Standards and guidance. For 2016, 392 reviews were selected for RAB observations. See pages 4 5 for a detailed description of the RAB observation process. Enhanced oversight. Oversights performed by subject matter experts on must-select engagements that include the review of the financial statements and working papers for the must-select engagements. See pages 5 9 for a detailed description of the enhanced oversight process. Oversight procedures performed by the AEs in accordance with the AICPA Peer Review Program Oversight Handbook included the following procedures: Administrative oversight of the AE. Administrative oversight performed by a peer review committee member in the year in which there was no oversight visit by a member of the OTF. Sixteen administrative oversights were performed in See page 9 10, Administrative Oversight of the AE. Oversight of peer reviews and reviewers. Oversight of various reviews, selected based on reviewed firm or peer reviewer, subject to minimum oversight requirements of the PRB. For 2016, 277 reviews were selected for oversight at the AE level. See pages 10 11, Oversight of the Peer Reviews and Reviewers. 1

6 AICPA Peer Review Board Annual Report on Oversight Annual verification of reviewers resumes. Verification of accuracy of information included on peer reviewer resumes. For 2016, resumes were verified for 697 reviewers. See page 11 12, Annual Verification of Reviewers Resumes. Based on the results of the oversight procedures performed, the OTF has concluded, for the oversight initiatives performed in the 2016 calendar year, that the objectives of the PRB oversight program, taken as a whole, were met. Respectfully submitted, Brian Bluhm Brian Bluhm, Chair Oversight Task Force AICPA Peer Review Board 2

7 Peer Review Board Oversight Process The Peer Review Board (PRB) has the responsibility for the oversight of all administering entities (AEs). In addition, each AE is responsible for overseeing peer reviews and peer reviewers for each state to which they administer. This responsibility includes having written oversight policies and procedures. All state boards of accountancy (SBAs) that require peer review accept the AICPA Peer Review Program (AICPA PRP) as a program satisfying its peer review licensing requirements. Some SBAs have entered into an agreement with state CPA societies to perform oversight of their administration of the AICPA PRP. This report is not intended to describe or report on that process. Objectives of Peer Review Board Oversight Process The PRB has appointed the Oversight Task Force (OTF) to oversee the administration of the oversight program and make recommendations regarding oversight procedures. The main objectives of the OTF are to provide reasonable assurance that the AEs are complying with the administrative procedures established by the PRB, reviews are being conducted and results of reviews are being evaluated and reported on in accordance with the Standards and on a consistent basis in all jurisdictions, and information provided to firms and reviewers (via the internet or other media) by AEs is accurate and timely. The oversight program also establishes a communications link with AEs and builds a relationship that enables the PRB to accomplish the following: Obtain information about problems and concerns of AEs peer review committees Provide consultation on those matters to specific AEs Initiate the development of guidance on a national basis, when appropriate OTF Oversight Procedures The following oversight procedures were performed as a part of the OTF oversight program. Oversight Visits of the Administering Entities Description Each AE is visited by a member of the OTF (ordinarily, at least once every other year). No member of the OTF is permitted to visit the AE in the state that his or her main office is located, where he or she serves as a technical reviewer or may have a conflict of interest, or where he or she performed the most recently completed oversight visit. Oversight Visit Procedures During these visits, the member of the OTF will, at a minimum meet with the AE s peer review committee during its consideration of peer review documents, evaluate a sample of peer review documents and applicable working papers on a postacceptance basis, 3

8 perform face-to-face interviews with the administrator, committee chair, and technical reviewers, and evaluate the various policies and procedures for administering the AICPA PRP. As part of the visit, the OTF member will request that the AE complete an information sheet documenting policies and procedures in the areas of administration, technical review, peer review committee, report acceptance, and oversight processes in administering the AICPA PRP. The OTF member evaluates the information sheet, results of the prior oversight visit, and comments from report acceptance body (RAB) observations to develop a risk assessment. A comprehensive oversight work program that contains the various procedures performed during the oversight visit is completed with the OTF member s comments. At the conclusion of the visit, the OTF member discusses any comments and issues identified as a result of the visit with the AE s peer review committee. The OTF member then issues an AICPA Oversight Visit Report (report) to the AE that discusses the purpose of the oversight visit and that the objectives of the oversight program were considered in performing those procedures. The report also contains the OTF member s conclusion regarding whether the AE has complied with the administrative procedures and Standards in all material respects as established by the PRB. In addition to the aforementioned report, the OTF member issues the AE an AICPA Oversight Visit Letter of Procedures and Observations (letter) that details the oversight procedures performed and observations noted by the OTF member. The letter also includes recommendations that may enhance the entity s administration of the AICPA PRP. The AE is then required to respond to the chair of the OTF, in writing, to any findings reported in the oversight visit report and letter or, at a minimum, when there are no findings reported, an acknowledgement of the visit. The oversight documents, including the Oversight Visit Report, the letter of procedures and observations, and the AE s response, are presented to the OTF members for acceptance. The AE may be required to take corrective actions as a condition of acceptance. The acceptance letter would reflect corrective actions, if any. A copy of the acceptance letter, the oversight visit report, letter of procedures and observations, and the response are posted to the following AICPA PRP website: ( visitresults.html) Results For the years 2015 and 2016, a member of the OTF performed at least one on-site oversight visit to 39 AEs (excludes the National Peer Review Committee [NPRC]). See exhibit 2 for a listing of the AEs and the year of oversight. See exhibit 3 for a summary of observations from the on-site oversight visits performed during the two years. RAB Observations Description The purpose of the RAB observation is to determine whether the RAB is performing all of its responsibilities; technical reviewers are performing all of their responsibilities; reviews are being conducted and reported on in accordance with the Standards and guidance; 4

9 administrative procedures established by the PRB are being complied with; information is being entered into the computer system correctly; and results of reviews are being evaluated on a consistent basis within an AE and in all jurisdictions. The objective of RAB observations is to provide real-time feedback to the RABs to improve overall quality and consistency of the RAB process. The process for the increased RAB observations is similar to the process used during the oversight visits. The RAB observer receives the materials that will be presented to the RAB prior to the RAB meeting. The observer selects a sample of reviews of firms enrolled in the AICPA PRP from the package and reviews the materials that will be presented to the RAB. The observer notes any issues or items that are unclear for each review selected. During the RAB review, the observer allows the RAB to deliberate each review. If the RAB does not address the items noted by the observer, the observer will bring those items to the RAB s attention prior to the RAB voting on whether to accept the review. All significant items that were noted by the observer, but not noted by the RAB, are included as comments in the RAB observation report. The OTF approves the report, and it is submitted to the AE peer review committee for its consideration. The OTF may request a response from the peer review committee based on the results of the RAB observations. Peer review committees may also choose to respond to the report, if not requested by the OTF. Results For 2016, each AE had at least one RAB observation. RAB observations were performed by OTF members as well as AICPA PRP staff. Recurring comments generated by RAB observations are summarized in exhibit 4. Results for 2016 RAB observations are as follows: Year Ended December 31, 2016 RAB meetings observed 104 AEs observed 40 Peer reviews observed 392 Based on observers comments: Acceptance delayed or deferred 135 Feedback forms issued 24 Monitoring letter Issued 1 OTF requested AE response due to results 8 of RAB observation Enhanced Oversights Description In May 2014, the PRB approved the addition of enhanced oversights performed by subject matter experts (SMEs). For 2016, the SMEs consisted of members of the applicable Audit Quality Center executive committees and expert panels, PRB members, former PRB members, individuals from firms that perform a large number of engagements in a must-select category, and individuals recommended by the Audit Quality Center executive committee and expert panel members. The enhanced oversights are one element of the AICPA s Enhancing Audit Quality (EAQ) initiative. 5

10 The oversights increase confidence in the peer review process, identify areas that need improvement, and provide meaningful data to inform other EAQ activities. As a result of the oversights, the PRB has approved multiple initiatives to improve reviewer performance on must-select engagements, including additional training requirements for reviewers and additional RAB observations with emphasis on must-select engagements. The results of the enhanced oversight findings are shared with other teams at the AICPA to further the goal of improving audit quality. The most common quality issue encountered in the enhanced oversights was inadequate audit documentation, which indicates that auditing procedures were either not performed or, when they were performed, not documented in accordance with professional standards. The AICPA has developed targeted communications and education to raise awareness of the documentation issues, including blog posts, Journal of Accountancy articles, a special inhouse training program for firms, newsletters, communications to state CPA societies and SBAs, and a social media meme campaign. The objective of the enhanced oversight is to increase the probability that peer reviewers are identifying all material issues in must-select engagements, including whether engagements are properly identified as nonconforming. This objective is achieved through the selection of two samples. The first sample is a random sample that will achieve a 90 percent to 95 percent confidence level. The second sample is a risk-based sample (targeted) based on certain risk criteria established by the OTF. For 2016, the risk-based sample consisted of firms and peer reviewers that perform five or fewer engagements in the must-select category chosen for oversight. If an individual was selected in the random sample, the individual was not selected for the targeted sample. The enhanced oversights focus exclusively on must-select engagements (engagements performed under Government Auditing Standards, audits of employee benefit plans, audits performed under the Federal Deposit Insurance Corporation Improvement Act [FDICIA], audits of carrying broker-dealers, and examinations of service organizations). For Government Auditing Standards engagements with Single Audit Act/Uniform Guidance portions of the engagement, the oversight focused only on the Single Audit Act/Uniform Guidance portion of the audit. These oversights will neither replace nor reduce the number of oversights currently required by AEs. Enhanced Oversight Process The enhanced oversight process consists of the review of the financial statements and working papers by the SME for the engagement selected. AICPA PRP staff notifies the peer reviewer and the firm that they have been selected for oversight once the peer review working papers and peer review report have been submitted to the AE. In this manner, the peer reviewer is not aware he or she has been selected for oversight until after he or she has reviewed the engagement. The SME completes the relevant peer review checklist and compares the results to the results of the peer reviewer. The SME issues a report detailing any material items not identified by the peer reviewer. If the peer reviewer failed to identify a nonconforming engagement, the peer reviewer completes a letter of response (LOR) detailing whether he or she agrees with the oversight report and lists any additional procedures that he or she will perform. The report and LOR (if applicable) is provided to the AE for consideration during the report acceptance process. If the peer reviewer disagrees with the results of the oversight, the AE will follow the disagreement guidance in the RAB Handbook. AICPA staff 6

11 monitors the effects of the oversights on the peer review results and what type of reviewer feedback (feedback form, performance monitoring letter, or performance deficiency letter), if any, is issued to the peer reviewer. Enhanced Oversight Sample For the 2016 sample (selected from peer reviews with must-select engagements performed between August 1, 2015 and July 30, 2016), 190 reviews were selected for enhanced oversight (85 random, 81 single audit, and 24 targeted selections). One hundred fifty different team captains were selected for oversight through the random and targeted samples. The 190 must-select engagements selected for oversight consisted of the following (a larger number of single audit engagements were selected due to the single audit sample discussed in the following text): Employee Benefit Single Audit/Uniform Government Auditing SOC 1 Total Plans Guidance Standards /FDICIA Exhibit 5 provides a listing of items identified by SMEs that were not identified by the peer reviewer that, either individually or in the aggregate, led to a nonconforming engagement. Exhibits 6 and 7 show the percentage of non-conforming engagements identified based on the number of must-select engagements performed by the firm in the category selected. Only one engagement was reviewed for each firm selected and the SME did not expand the scope of the oversight. For exhibit 7, the detail of the random sample is provided because the sample is representative of the overall population. Refer to the following section for further discussion of the sample selection. The results of the subsequent samples will be presented in future oversight reports. Random (Baseline) Sample The random sample was selected in order to achieve a 95 percent confidence rating for the population as a whole. This means that the sample has a 95 percent chance of representing the overall population. The results of this random sample of oversights are used as a benchmark to measure audit quality improvements over time. 1 In 2017, the AICPA introduced the term system and organization controls (SOC) to refer to the suite of services practitioners may provide relating to system-level controls of a service organization and system or entity-level controls of other organizations. Formerly, SOC referred to service organization controls. By redefining that acronym, the AICPA enables the introduction of new internal control examinations that may be performed (a) for other types of organizations, in addition to service organizations, and (b) on either system-level or entity-level controls of such organizations. 7

12 2016 Random Sample Results: Selected Number of Nonconforming Identified by Peer Reviewer % Number of Nonconforming Not Identified by Peer Reviewer % Total Nonconformin g % % 21 25% 47 55% There was a significant increase in the percentage of nonconforming engagements identified by peer reviewers over the first two oversight years. For 2016, peer reviewers identified 31 percent of the engagements selected for oversight as nonconforming, whereas peer reviewers identified only 9 percent of the engagements as nonconforming in the first oversight sample performed in the prior year. For the 21 oversights in which the nonconforming engagement was not identified by the peer reviewer, the oversight resulted in a change in peer review rating on 7 of the 21 (33 percent) reviews. Single Audit Sample Due to the troubling single audit results from the first enhanced oversight sample, a specific single audit random sample was performed in 2016 in addition to the statistically valid sample previously discussed. This sample included a random selection of 81 single audits. The purpose of the special sample was to determine the nonconformity rate of single audits subject to peer review as well as obtained additional firm quality control data to inform EAQ initiatives. Single Audit Random Sample Results: Selected Number of Nonconformin g Identified by Peer Reviewer % Number of Nonconforming Not Identified by Peer % 34 Reviewer % Total Nonconformin g % 42 % 44 54% The overall nonconforming rate for the single audit sample was similar to the baseline random sample, but the peer reviewers did not identify nonconforming engagements at as high a rate. Targeted Sample The targeted sample for 2016 consisted of a sample of firms and peer reviewers that perform five or fewer engagements in the must-select category selected for oversight. If a team captain was selected during the random sample, he or she was not selected for the targeted sample. 8

13 Targeted Sample Results: Selected Number of Nonconformin g Identified by Peer Reviewer % Number of Nonconforming Not Identified by Peer % 7 Feedback Issued 9 Reviewer % Total Nonconformin g % 29 % 23 54% Overall, for the 2016 sample, there were 62 engagements selected for oversight that were not identified as nonconforming by the peer reviewer. Of those 62 oversights, 56 have completed the RAB process as of the date of this report, including consideration of feedback. The delay in consideration of feedback is either because the firm or peer reviewer is not cooperating with the process, or the peer reviewer disagreed with the result of the oversight, which resulted in a delay in the acceptance of the peer review and consideration of feedback. For the 56 oversights referred to previously, in which the nonconforming engagement was not identified by the peer reviewer, the following feedback was issued by the AE: 14 resulted in a significant reviewer performance deficiency 19 resulted in a feedback form 2 resulted in a monitoring letter 3 resulted in a deficiency letter 18 resulted in no feedback The OTF monitors the types of feedback issued as a result of the oversights. The OTF considers if any further actions are necessary, including whether to issue a feedback form, monitoring letter, or deficiency letter to the peer reviewer. Oversight by the Administering Entities Peer Review Committees The AEs peer review committees are responsible for monitoring and evaluating peer reviews of those firms whose main offices are located in their licensing jurisdiction(s). Committees may designate a task force to be responsible for the administration and monitoring of its oversight program. AEs are required to submit their oversight policies and procedures to the PRB on an annual basis. In conjunction with AE personnel, the peer review committee establishes oversight policies and procedures that meet the minimum requirements (discussed on pages 10 11, AE Oversight Procedures ) established by the PRB to provide reasonable assurance that reviews are administered in compliance with the administrative procedures established by the PRB; reviews are conducted and reported on in accordance with the Standards; results of reviews are evaluated on a consistent basis; and information disseminated by the AE is accurate and timely.

14 AE Oversight Procedures The following oversight procedures are performed as part of the AE oversight program. Administrative Oversight of the AE Description At a minimum, a committee member or a subcommittee of the AE s peer review committee should perform the administrative oversight in those years when there is no oversight visit by OTF. Procedures to be performed should cover the administrative requirements of administering the AICPA PRP. Results The AE submitted administrative oversight reports to the AICPA as part of the 2016 plan of administration (POA).Comments or suggestions resulting from the administrative oversights are summarized in exhibit 8. In addition, the OTF member reviewed the results of the administrative oversight during his or her oversight visit (described on pages 3 4, Oversight Visits of the Administering Entities ) and compared the results of the administrative oversight to those noted during the OTF oversight visit. Oversight of Peer Reviews and Reviewers Description Throughout the year, the AE selects various peer reviews for oversight. The selections can be on a random or targeted basis. The oversight may consist of completing a full working paper review after the review has been performed, but prior to presenting the peer review documents to the peer review committee. The oversight may also consist of having a peer review committee member or designee visit the firm, either while the peer review team is performing the review, or after the review, but prior to final committee acceptance. As part of its oversight process, the peer review committee oversees firms being reviewed as well as reviewers performing reviews. Minimum oversight selection requirements are also imposed by the PRB. Firms The selection of firms to be reviewed is based on several factors, including the types of peer review reports the firm has previously received, whether it is the firm s first system review (after previously having an engagement review), and whether the firm conducts engagements in high-risk industries. Reviewers All peer reviewers are subject to oversight and may be selected based on several factors, including random selection, any unusually high percentage of pass reports as compared to non-pass reports, conducting a significant number of reviews for firms with audits in high-risk industries, performance of his or her first peer review, or performing high volumes of reviews. Oversight of a reviewer can also occur due to previously noted performance deficiencies or a history of performance deficiencies, such as issuance of an inappropriate peer review report, not considering significant matters, or failure to select an appropriate number of engagements. When an AE oversees a reviewer from another state, the results are conveyed to the AE of that state. Minimum Requirements At a minimum, the AE is required to conduct oversight on 2 percent of all reviews performed in a 12-month period, and within the 2 percent selected, there must be at least 2 of each type of peer review evaluated (that is, system and engagement reviews). The 10

15 oversight involves completing a full working paper review, and it may be performed on-site in conjunction with the peer review or after the review has been performed. It is recommended that the oversight be performed prior to presenting the peer review documents to the peer review committee. This allows the committee to consider all the facts prior to acceptance of the review. At a minimum, 2 system review oversights are required to be performed on-site. Oversights may be random or a combination of a targeted and random selection. AEs that administer fewer than 100 reviews annually can apply for a waiver from the minimum requirements. The request for a waiver includes the reason(s) for the request and suggested alternatives to the minimum requirements. The waiver is to be submitted and approved by the PRB each year. Also, at least two engagement oversights must be performed by the AE s peer review committee or by its designee from a national list of qualified reviewers on an annual basis. An engagement oversight (performed either off- or on-site) is the review of all peer reviewer materials and the reviewed firm s financial statements and working papers on the engagement. The two engagement oversights must include audits of employee benefits plans subject to the Employee Retirement Income Security Act of 1974 (ERISA), engagements performed under generally accepted government auditing standards (GAGAS), audits of insured depository institutions subject to the FDICIA, audits of carrying broker-dealers, or examinations of SOC 1 engagements and SOC 2 engagements. Also, the two oversights selected should not be of the same types of audits. No waivers of oversight of these types of engagements are permitted. Results For 2016, the AEs conducted oversight on 277 reviews. There were 156 system and 121 engagement reviews oversighted. Approximately 53 percent of the system oversights were conducted on-site. In addition, 1 FDICIA, 73 ERISA, and 77 GAGAS engagements were oversighted. See exhibit 9 for a summary of oversights by AE. Annual Verification of Reviewers Resumes Description To qualify as a reviewer, an individual must be an AICPA member and have at least five years of recent experience in the practice of public accounting in accounting or auditing functions. The firm(s) with whom the member is associated should have received a pass report on either its system or engagement review. The reviewer should obtain at least 48 hours of continuing professional education in subjects related to accounting and auditing every 3 years, with a minimum of 8 hours in any one year. A reviewer of an engagement in a high-risk industry should possess not only current knowledge of professional standards but also current knowledge of the accounting practices specific to that industry. In addition, the reviewer of an engagement in a high-risk industry should have current practice experience in that industry. If a reviewer does not have such experience, the reviewer may be called upon to justify why he or she should be permitted to review engagements in that industry. The AE has the authority to decide whether a reviewer s or review team s experience is sufficient to perform a particular review. Ensuring that reviewers resumes are updated annually and are accurate is a critical element in determining if the reviewer or review team has the appropriate knowledge and experience to 11

16 perform a specific peer review. The AE must verify information within a sample of reviewers resumes on an annual basis. All reviewer resumes should be verified over a 3-year period, as long as, at a minimum, one-third are verified in 1 one, a total of two-thirds have been verified by year 2, and 100 percent have been verified by year 3. Verification must include the reviewers qualifications and experience related to engagements performed under GAGAS, audits of employee benefit plans subject to ERISA, audits of insured depository institutions subject to the FDICIA, audits of carrying broker-dealers, and examinations of SOC 1 engagements and SOC 2 engagements, as applicable. Verification procedures may include requesting copies of their license to practice as a CPA; continuing professional education (CPE) certificate from a qualified reviewer training course; CPE certificates that document the required 48 CPE credits related to accounting and auditing to be obtained every 3 years with at least 8 hours in one year; and CPE certificates that document qualifications to perform audits under Government Auditing Standards, if applicable. The AE should also verify whether the reviewer is a partner or manager in a firm enrolled in a practice-monitoring program and whether the reviewer s firm received a pass report on its most recently completed peer review. Results Each AE submitted a copy of its oversight policies and procedures indicating compliance with this oversight requirement, along with a list of reviewers whose resume information was verified during See exhibit

17 Exhibit 1 Administering Entities Approved to Administer the 2016 AICPA PRP Administering Entity Licensing Jurisdiction(s) Alabama Society of CPAs Alabama Arkansas Society of CPAs Arkansas California Society of CPAs California, Arizona, Alaska Colorado Society of CPAs Colorado, New Mexico 2 Connecticut Society of CPAs Connecticut Florida Institute of CPAs Florida Georgia Society of CPAs Georgia Hawaii Society of CPAs Hawaii Idaho Society of CPAs Idaho Illinois CPA Society Illinois, Iowa, and South Carolina 3 Indiana CPA Society Indiana Kansas Society of CPAs Kansas Kentucky Society of CPAs Kentucky Society of Louisiana CPAs Louisiana Maryland Association of CPAs Maryland Massachusetts Society of CPAs Massachusetts, New Hampshire 4 Michigan Association of CPAs Michigan Minnesota Society of CPAs Minnesota Mississippi Society of CPAs Mississippi Missouri Society of CPAs Missouri Montana Society of CPAs Montana National Peer Review Committee N/A Nevada Society of CPAs Nevada, Wyoming, Nebraska, Utah New England Peer Review, Inc. Maine, Rhode Island, Vermont New Jersey Society of CPAs New Jersey New York State Society of CPAs New York North Carolina Association of CPAs North Carolina North Dakota Society of CPAs North Dakota The Ohio Society of CPAs Ohio Oklahoma Society of CPAs Oklahoma, South Dakota Oregon Society of CPAs Oregon, Guam, Northern Mariana Islands Pennsylvania Institute of CPAs Pennsylvania, Delaware, Virgin Islands Puerto Rico Society of CPAs Puerto Rico Tennessee Society of CPAs Tennessee 2 Effective August Effective August Effective May

18 Exhibit 1 Administering Entities Approved to Administer the 2016 AICPA PRP Texas Society of CPAs Texas Virginia Society of CPAs Virginia, District of Columbia Washington Society of CPAs Washington West Virginia Society of CPAs West Virginia Wisconsin Institute of CPAs Wisconsin 14

19 Exhibit 2 On-Site Oversights of Administering Entities Performed by AICPA Oversight Task Force For the years 2015 and 2016, a member of the OTF performed an on-site oversight visit to each of the following 39 AEs. As part of the oversight procedures, each AE is visited by a member of the OTF whenever deemed necessary, ordinarily, at least once every other year. The oversight results can be found on the AICPA s website Connecticut Georgia Hawaii Idaho Illinois Indiana Kentucky Louisiana Maryland Massachusetts Minnesota New York North Carolina Oklahoma South Carolina Texas Virginia Washington Alabama Arkansas California Colorado Florida Kansas Michigan Mississippi Missouri Montana Nevada New England New Jersey North Dakota Ohio Oregon Pennsylvania Puerto Rico Tennessee West Virginia Wisconsin 15

20 Exhibit 3 Observations From On-Site Oversights of Administering Entities Performed by the AICPA Oversight Task Force As discussed in more detail in the Oversight Visits of the AEs section, each AE is visited at least every other year by an OTF member who performs various oversight procedures. At the conclusion of the visit, the OTF member issues an AICPA oversight visit report as well as an AICPA Oversight Visit Letter of Procedures and Observations, which details the oversight procedures performed, observations noted by the OTF member, and includes recommendations that may enhance the entity s administration of the AICPA PRP. The AE is required to respond to the chair of the OTF, in writing, to any findings reported in the Oversight Visit Report and Letter or, at a minimum, when there are no findings reported, an acknowledgement of the visit. The two oversight documents and the AE s response are presented by the AICPA OTF PRB members at the next AICPA PRB meeting for acceptance. A copy of the acceptance letter, the two oversight visit letters, and the response are posted to the following AICPA PRP website: ( results.html) The following represents a summary of common observations made by the OTF resulting from the on-site oversight visits performed during The following observations are not indicative of every AE and may have been a single occurrence that has since been corrected upon notification. Administrative Procedures The appropriate letters for overdue information and documents, reviewer performance, and other reminders not generated according to the time requirements in the administrative manual Inadequate monitoring of open reviews, open corrective actions, and implementation plans by staff and committee members Annual POA not submitted timely Acceptance letters not sent timely Documents not uploaded timely to the FSBA website Formal communications not sent to reviewed firms and peer reviewers when the RAB has either delayed or deferred acceptance of the review Confidentiality letters not obtained from technical reviewers Confidentiality letters not obtained from committee members All required materials not provided to the RAB Resume verification not completed timely and not performed by an appropriate individual Noncompliance with confidentiality requirements Ineligible reviewer not suspended Website and Other Media Information The data maintained on the website as it relates to peer review not current Working Paper Retention Working papers not retained and destroyed 120 days after acceptance by the peer review committee in accordance with the working paper retention policy of the administrative manual Committee Procedures Reviewer feedback not issued when necessary. Also, reviewer feedback not signed by a peer review committee member 16

21 Exhibit 3, continued Observations from On-Site Oversights of Administering Entities Performed by the AICPA Oversight Task Force Technical reviewers failed to address all significant issues before reviews presented to the RAB The status of open reviews and follow-up status not periodically monitored and discussed by the committee and related documentation of such presentations and discussions recorded in the committee minutes RAB composition failed to comply with requirements of the RAB Handbook Technical reviewers not evaluated annually Internal oversight of the administration of the AICPA PRP not performed timely Required oversights not performed timely Oversights not monitored to ensure at least two required on-site oversights are selected and completed before the end of each year 17

22 Exhibit 4 Comments From RAB Observations Performed by AICPA Peer Review Program Staff and OTF Members Throughout each year, a sample of RABs is selected for observation. At least one RAB observation is performed for each AE per year. The documents provided to the RAB are reviewed (by PRP staff, OTF members, or both) to increase the probability that the RAB process is operating properly and to ensure the results of reviews are being evaluated on a consistent basis within an AE in all jurisdictions. The following is a summary of recurring comments generated from the RAB observations performed by the AICPA PRP staff and OTF members for The comments are intended to provide the AEs, their committees, RABs, peer reviewers, and technical reviewers with information and constructive recommendations that will increase consistency and improve the peer review process in the future. The comments vary in degree of significance and are not applicable to all the respective parties. Potential issues regarding auditor compliance with independence requirements of Government Auditing Standards (Yellow Book) Reviewers risk assessments not comprehensive; items not addressed include firm mergers, firm industry concentrations, staff CPE, and unique risks associated with employee benefit plan audits when the firm had multiple types Findings for further consideration (FFC) form did not contain all required information to be provided in the reviewed firm s response The systemic cause missing or did not appropriately address the underlying cause of the finding on the FFC forms Matters for Further Consideration (MFC) forms included specific reviewer, firm, or client names Firm representation letters not consistent with the illustration in appendix B of the Standards Peer review report language not consistent with current professional standards Peer reviewer did not expand scope in accordance with Standards and guidance Reviewer performance feedback not initially recommended when peer reviewers did not appropriately aggregate and evaluate matters; peer reviewers failed to identify nonconforming engagements; oversight resulted in issues not previously detected by the reviewer; and peer review reports and letters of response were not in compliance with Standards Reviews are not consistently presented to the RAB free from open technical issues RAB members did not meet training requirements as established in the RAB Handbook Acceptance and deferral letters not sent timely 18

23 Exhibit 5 Material Departures From Professional Standards Identified by SMEs As discussed in more detail in the Enhanced Oversights section, the SMEs identified a large number of material departures from professional standards that were not identified by the peer reviewers. The following is a list of departures from professional standards identified in the 2016 sample. The SMEs identified these departures from professional standards, individually or in the aggregate, as material departures from professional standards that caused the engagement to be considered nonconforming. Employee Benefit Plan Lack of documentation of design and implementation of internal controls, including documentation of reliance on system and organization controls report Failure to obtain sufficient appropriate audit evidence to provide reasonable assurance that fair value measurements (including appropriate leveling) and disclosures in the financial statements are in conformity with generally accepted accounting principles (GAAP) No testing of participant data and participant elections No testing or inadequate testing of benefit payments or distributions, including inadequate sample sizes No testing of vesting and forfeitures for distributions No testing for contributions received or receivable No documentation of consideration of material passed adjustments Inadequate testing of timeliness of participant contributions No documentation of how sample sizes were determined No testing of payroll data and plan-defined compensation used in contribution calculations No explanation of variances identified during employee contribution testing Material receivables not recorded in the financial statements Auditor did not identify that the plan used the incorrect definition of compensation Material GAAP departures in revenue recognition, benefit payments, and benefit obligations in a defined benefit health and welfare plan No documentation of testing of employer contributions at the participant level Errors in the auditor s report and financial statement preparation, including the following: Financial statements prepared on the modified cash basis of accounting and the auditor s report failed to include language regarding management s responsibility for determining that the modified cash basis of accounting is an acceptable basis for presentation of the financial statements No indication that the prior year s financial statements were compiled No indication that the prior year s financial statements were prepared under the modified cash basis of accounting Auditor s report referred to the incorrect custodian certifying the investments and investment income Financial statements should have been prepared using the liquidation basis of accounting 19

24 Exhibit 5, continued Material Departures From Professional Standards Identified by SMEs Single Audit/Uniform Guidance and Government Auditing Standards No testing of internal controls over direct and material compliance requirements, including testing internal controls over some, but not all, direct and material compliance requirements No documentation or limited documentation of internal controls over compliance No documentation of auditor analysis and judgment of which compliance requirements were determined not direct and material No documentation of specific procedures performed to test direct and material compliance requirements, including use of generic tick marks, such as vouched or tested No documentation of how sample sizes were determined, including selecting a single sample for multiple grants Yellow Book independence documentation, including the following: No documentation of skills, knowledge, or experience (SKE) No documentation of the evaluation of non-audit services for threats to independence No documentation of a self-review threat for a sole practitioner No documentation of safeguards other than SKE No documentation of why financial statement preparation was not considered a threat to independence Auditor incorrectly identified applicable compliance requirements as not direct and material No documentation of controls over the preparation of the schedule of expenditures of federal awards (SEFA) No reconciliation of SEFA amounts included in the working papers No audit programs for single audit procedures included in the working papers No documentation of consideration of performing a single audit when a single audit was not required No documentation of materiality for each major program Representation letter did not include single-audit-specific information No documentation of determination of Type A versus Type B programs No documentation of risk assessment for Type B programs Auditor used incorrect compliance supplement Auditor identified the auditee as low risk when the data collection form was not submitted by the deadline in the prior year SOC 1 Engagement Documentation did not include sufficient detail to support the testing performed 20

25 Exhibit 6 Overall Non-Conforming Identified During 2016 Enhanced Oversights by Firm Size The 2016 enhanced oversight sample was divided into two samples: a random sample and a targeted sample. One hundred ninety must-select engagements were selected for oversight (85 random, 81 single audit, and 24 targeted selections). The following tables detail the number of nonconforming engagements identified in relation to the number of must-select engagements performed by the firm in that category for all 190 must-select engagements selected for oversight. Number of Must-Select Performed by Each Firm Selected* Number of Nonconforming Overall Sample Must-Select Audit Reviewed Percentage of Reviewed Identified as Nonconforming % % % 11 or more % Total % Number of Must-Select Performed by Each Firm Selected* Employee Benefit Plan Number of Nonconforming Must-Select Audit Reviewed Percentage of Reviewed Identified as Nonconforming % % % 11 or more % Total % Number of Must-Select Performed by Each Firm Selected* GAS/Uniform Guidance Number of Nonconforming Must-Select Audit Reviewed Percentage of Reviewed Identified as Nonconforming % % % 11 or more % Total % *Column represents the number of must-select engagements performed by the firm in the must-select category selected for oversight. Note: One system and organization controls engagement and one FDICIA engagement was selected for oversight. 21

26 Exhibit 7 Non-Conforming Identified During 2016 Enhanced Oversights Random Sample by Firm Size The 2016 enhanced oversight sample was divided into two samples: a random sample and a targeted sample. One hundred ninety must-select engagements were selected for oversight (85 random, 81 single audit, and 24 targeted selections). The following tables detail the number of nonconforming engagements identified in relation to the number of must-select engagements performed by the firm in that category for the 85 must-select engagements randomly selected for oversight. Random Selections Number of Must-Select Performed by Each Firm Selected* Number of Nonconforming Must-Select Audit Reviewed Percentage of Reviewed Identified as Nonconforming % % % 11 or more % Total % Number of Must-Select Performed by Each Firm Selected* Employee Benefit Plan Number of Nonconforming Must-Select Audit Reviewed Percentage of Reviewed Identified as Nonconforming % % % 11 or more % Total % Number of Must-Select Performed by Each Firm Selected* GAS/Uniform Guidance Number of Nonconforming 22 Must-Select Audit Reviewed Percentage of Reviewed Identified as Nonconforming % % % 11 or more % Total % *Column represents the number of must-select engagements performed by the firm in the must-select category selected for oversight.