FTA s Office of Program Oversight: Updates and Trends May 15, 2016 John Bodnar Office of Program Oversight
Agenda FTA Program Oversight Overview Program Oversight Lifecycle Initiatives to Improve Compliance Oversight Data Trends Most Common Findings 2
Agenda FTA Program Oversight Overview Program Oversight Lifecycle Initiatives to Improve Compliance Oversight Data Trends Most Common Findings 3
FTA Oversight in Context Oversight is necessary and beneficial for grantees, the public, and FTA Strengthens the capacity of our grantees to improve public transit for America s communities Required in legislation Fulfills FTA s fiduciary responsibility to prevent and identify improper payments and to ensure every dollar counts toward improving public transit. Confirms and promotes grantee compliance with FTA requirements Helps address Congressional and public questions about the use of federal funds 4
FTA Oversight in Context FTA s regulatory environment is constantly changing Federal laws, Congressional mandates for new regulations, Presidential Executive Orders, and findings and recommendations from agencies that review and audit FTA can affect program oversight and present challenges to maintaining review consistency FTA must follow, and help grantees adhere to, these changes through program material updates and circulars 5
What is Program Oversight? Comprehensive Reviews (Triennial & State Management) Occur every three years Forward-looking instead of reactive Assess management practices and program implementation Specialized Reviews (Financial, Procurement, & Civil Rights) Occur at FTA s discretion Focus on specific areas of Federal compliance 6
OIG Program Oversight Audit August 2012: OIG report Improvements Needed in FTA s Grant Oversight Program Six recommendations for FTA have two major themes: Consistency Quality Assurance Additional staff required to develop and implement: Quality Assurance enhancements Update of internal Findings Management process Revision of internal Performance 7
Agenda FTA Program Oversight Overview Program Oversight Lifecycle Initiatives to Improve Compliance Oversight Data Trends Most Common Findings 8
Annual Program Oversight Lifecycle The Program Oversight Lifecycle is a framework and schedule for program oversight reviews that helps: Provide grantees with better consistency in timing/scheduling Allow FTA to better coordinate reviews to avoid overlap, conflicts, and overburdening grantees Guide FTA workflow and provide holistic view of dependent processes Promote efficient use of contractor and FTA staff time and help balance work throughout the year 9
Annual Program Oversight Lifecycle Review Material Update w/ FTA Subject Ma8er Experts InformaAon Requests Sent to Grantees Review of Grantees InformaAon Request Responses Schedule De- ConflicAon On-Site Reviews Report Development & Issuance 10
De-Confliction Results Grantees Receiving Multiple Reviews by Year 60 # of Grantees 50 40 30 48 20 10 0 25 15 8 8 FY12 FY13 FY14 FY15 FY16* *Drug and Alcohol and State Safety Oversight reviews added to the de-confliction process as of FY 2016. 11
Agenda FTA Program Oversight Overview Program Oversight Lifecycle Initiatives to Improve Compliance Oversight Data Trends Most Common Findings 12
Dynamic Review Guide Purpose: Prepare individualized review guides and information requests based on funding program and grantee characteristics Process: Analyze and sort existing review questions Standardize review questions across Triennial and State Management Review programs Determine applicability of each question to each grantee by: Funding program received (5307, 5310, 5311, etc.) Organization type (state/non-state) Whether the grantee directly operates service Modes operated (bus, rail, ferry, etc.) Whether or not the grantee has sub-recipients Generate grantee-specific review guides and information requests Intended outcomes: Reduced information request response burden on grantees More flexible review program 13
Grantee Oversight Workshops Purpose: Improve consistency across review programs and communicate FTA's oversight responsibilities, processes, and expectations in a unified forum Process: Merge program workshops into a single Oversight Workshop Identify and consolidate common content for plenary sessions Separate tracks/breakouts for topic areas More interactive exercises and real world examples Intended outcomes: Increased grantee attendance and participation Clearer, more consistent information about programs and processes More proactive approach with grantees Greater convenience and flexibility 14
Online Training Program Purpose: Provide additional opportunities for grantees to learn about and understand FTA's oversight review process Process: Develop short webinars and online training modules covering key compliance and oversight review topics Webinar topics and online modules to be developed in consultation with grantees, regions, program offices, and review contractors Intended outcomes: Reinforce information provided at Oversight Workshops Refresher for workshop participants in advance of review Capacity building for grantee staff 15
Enhanced Technical Assistance Purpose: Provide more targeted and effective technical assistance to grantees in order to reduce overall findings, resolve open ones, and prevent repeat findings. Process: Identify and understand current and anticipated assistance Identify gaps in assistance to determine areas of need Initiate proactive assistance for grantees Focus on capacity building in addition to resolution of specific issues Create a feedback mechanism and evaluation process Intended outcomes: Higher compliance rates (fewer findings) Better relationships between FTA and transit industry 16
Contractor Summit Purpose: Bring review contractors together to foster greater consistency in the review process Process: Pilot summit will focus on Triennial Review and SMR contractors Presentations to reinforce FTA priorities, program updates and changes, trends, and experiences Interactive scenarios/exercises involving actual review findings Discussion/brainstorming about improving consistency Intended outcomes: Understand inconsistencies in findings and corrective actions Identify where and why those differences originate Determine need for updates, education, and better communication Emphasize the need for documenting the rationale for findings 17
Oversight Data Trends Purpose: Enhance analysis of program oversight data to identify and communicate compliance trends Process: Analyze data from FTA s largest review program (Triennial Review) 596 Triennial Reviews with 4,700 findings from 2013 to 2015 50 states, 2 territories, and 1 federal district Determine area(s) of interest (First up: Number of findings per review) Identify compliance trends based on grantee characteristics (location, size, organization type, modes operated, etc.) Communicate analyses with transit industry stakeholders Intended outcomes: A broader understanding of compliance trends Effective and helpful compliance program improvements developed utilizing program oversight data 18
Agenda FTA Program Oversight Overview Program Oversight Lifecycle Initiatives to Improve Compliance Oversight Data Trends Most Common Findings 19
60 2013-2015 Triennial Review Performance Frequency of Review Finding Counts # of Reviews 55 50 45 40 35 30 25 20 15 Average = 7.89: Distribution is skewed right. Average is raised by outliers on the high side. 1st QuarAle (apx) 0-2 Findings 142 Grantees 2nd QuarAle (apx) 3-6 Findings 161 Grantees 3rd QuarAle (apx) 7-11 Findings 142 Grantees 75th-95th PercenAle (apx) 12-19 Findings 121 Grantees 95th+ PercenAle (apx) 20-42 Findings 30 Grantees 10 5 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 # of Findings 20
2013-2015 Triennial Review Performance Average Number of Findings for Reviews Within Each State 5.00 12.29 17.00 7.00 6.00 10.20 10.00 4.33 6.33 7.00 6.43 9.00 37.00 19.33 13.50 6.50 8.25 8.64 7.00 5.50 6.00 7.25 8.00 8.95 5.40 5.88 8.00 6.75 6.62 11.33 14.29 3.88 6.08 5.14 5.64 Not Shown: Puerto Rico (9.33) Virgin Islands (20.00) 10.00 7.75 7.00 3.17 9.27 8.40 2.30 4.00 3.10 DC: 3.00 DE: 9.00 MA: 10.00 MD: 12.00 RI: 6.00 VT: 6.00 USA: 7.89 21
2013-2015 Triennial Review Performance Average Number of Findings by Operating Expenses and Fleet Size FY13 to FY15, Average # of TR Findings per Grantee Over $1 Billion Operating Expenses 7.14 Over $100 Million Operating Expenses & Over 500 Vehicles 9.19 All Other Grantees 7.83 22
2013-2015 Triennial Reviews by Organization Type 18 3% 16 3% 13 2% 32 5% 63 11% 237 40% City / City Agency Transit Authority County Agency Other MulA State Group State Agency Council of Govt. 217 36% 23
Average # of Findings 14 12 10 8 6 4 2013-2015 Triennial Review Performance Average Number of Findings by Organization Type * Findings related to 5307 funding only. 5311 not included in this analysis. NaAonal Average = 7.89 2 0 12.00 10.00 9.50 9.35 9.29 8.50 8.50 8.50 8.00 7.23 5.94 5.75 5.17 4.54 4.33 OrganizaAon Type 24
# of Reviews 60 55 50 45 40 35 30 25 20 15 2013-2015 Triennial Review Performance Frequency of Review Finding Counts (City Data Overlay) NaAonal Average = 7.89 [City Average = 9.35] 1st QuarAle (apx) 0-2 Findings 142 Grantees 41 CiAes 2nd QuarAle (apx) 3-6 Findings 161 Grantees 55 CiAes 3rd QuarAle (apx) 7-11 Findings 142 Grantees 58 CiAes 75th-95th PercenAle (apx) 12-19 Findings 121 Grantees 65 CiAes 95th+ PercenAle (apx) 20-42 Findings 30 Grantees 18 CiAes CiAes 10 5 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 # of Findings 25
2013-2015 Triennial Review Performance Frequency of Review Finding Counts (Transit Authority Data Overlay) 60 # of Reviews 55 50 45 40 35 30 25 20 15 10 NaAonal Average = 7.89 [Transit Authority Average = 7.23] 1st QuarAle (apx) 0-2 Findings 142 Grantees 65 Transit AuthoriAes 2nd QuarAle (apx) 3-6 Findings 161 Grantees 54 Transit AuthoriAes 3rd QuarAle (apx) 7-11 Findings 142 Grantees 54 Transit AuthoriAes 75th-95th PercenAle (apx) 12-19 Findings 121 Grantees 36 Transit AuthoriAes 95th+ PercenAle (apx) 20-42 Findings 30 Grantees 8 Transit AuthoriAes Transit AuthoriAes 5 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 # of Findings 26
2013-2015 Triennial Review Performance Largest Organization Types Cities (237 Reviews) Transit Authorities (217 Reviews) 1st Quartile 0-2 Findings 2nd Quartile 3-6 Findings 3rd Quartile 7-11 Findings 75-95th Percentile 12-19 Findings 95th+ Percentile 20-42 Findings 65 27% 18 8% 58 25% 41 17% 55 23% 54 25% 36 16% 8 4% 54 25% 65 30% 41% of city grantees are in the first 2 quartiles, 59% in the last 2 quartiles Approximately 1 in 6 city grantees are in the 1 st quartile and 1 in 3 in the 4 th quartile 55% of transit authority grantees are in the first 2 quartiles, 45% in the last 2 quartiles Approximately 1 in 3 transit authority grantees are in the 1st quartile, compared to 1 in 5 in the 4 th quartile 27
2013-2015 Triennial Review Performance 1st and 4 th Quartile (Number of Findings) by Organization Type City Transit Authority County Agency Other Multi State Group State Agency Council of Govt. 1st Quartile (0-2 Findings) 19 13% 1 1% 8 6% 3 2% 5 3% 65 46% 41 29% Transit agencies are 28% more likely to achieve 1 st quartile results 4th Quartile (12-42 Findings) 44 29% 4 3% 11 7% 5 3% 2 2% 2 1% 83 55% Cities are 38% more likely to achieve 4 th quartile results Transit authorities make up 36% of FTA 5307 grantees, cities make up 40% 28
2013-2015 Triennial Reviews by Number of Modes Operated 103 17% 15 3% 8 1% 31 5% 4 1% 1 0% 32 5% 336 57% 66 11% 0 1 2 3 4 5 6 7 8 Source: 2013 National Transit Database 29
2013-2015 Triennial Review Performance Average Number of Findings by Number of Modes Operated 20.00 18.00 Average Number of Findings 16.00 14.00 12.00 10.00 8.00 6.00 4.00 NaAonal Average = 7.89 2.00 0.00 6.44 7.70 8.17 8.60 5.35 6.60 3.75 10.50 18.00 0 1 2 3 4 5 6 7 8 Number of Modes Operated Source: 2013 National Transit Database 30
Agenda FTA Program Oversight Overview Program Oversight Lifecycle Initiatives to Improve Compliance Oversight Data Trends Most Common Findings 31
Number of Grantees with at Least One Triennial Review Area Finding by Federal Fiscal Year of Review - Procurement # of Grantees 0 50 100 150 200 250 FY13 115 of 189 (60.9%) FY14 127 of 198 (64.1%) FY15 141 of 209 (67.5%) 32
Number of Grantees with at Least One Triennial Review Area Finding by Federal Fiscal Year of Review - ADA # of Grantees 0 50 100 150 200 250 FY13 96 of 189 (50.8%) FY14 84 of 198 (42.4%) FY15 105 of 209 (50.2%) 33
Number of Grantees with at Least One Triennial Review Area Finding by Federal Fiscal Year of Review - DBE # of Grantees 0 50 100 150 200 250 FY13 65 of 189 (34.4%) FY14 80 of 198 (40.4%) FY15 96 of 209 (46.0%) 34
Number of Grantees with at Least One Triennial Review Area Finding by Federal Fiscal Year of Review Technical Capacity # of Grantees 0 50 100 150 200 250 FY13 71 of 189 (37.6%) FY14 80 of 198 (40.4%) FY15 94 of 209 (45.0%) 35
30% 2013-2015 Triennial Review Top 6 Finding Areas by Percentage of All Findings 25% 20% 15% 10% 5% 0% 26.9% 12.3% 10.9% 10.6% 7.4% 7.0% Procurement ADA DBE Technical Capacity Maintenance Financial Management / Capacity 36
2013-2015 Triennial Review Top 5 Findings Top 5 Findings Nationwide Area Count % of Findings Lacking independent cost estimate Procurement 160 3.4% No verification that excluded parties are not participating Procurement 142 3.0% Lacking required cost/price analysis Procurement 142 3.0% Insufficient no-show policy ADA 135 2.9% No FTA clauses Procurement 132 2.8% 37