Monitoring 1. Describe how and the frequency with which you monitored your activities. 2. Describe the results of your monitoring including any improvements. 3. Self Evaluation a. Describe the effect programs have in solving neighborhood and community problems. b. Describe progress in meeting priority needs and specific objectives and help make community s vision of the future a reality. c. Describe how you provided decent housing and a suitable living environment and expanded economic opportunity principally for low and moderate-income persons d. Indicate any activities falling behind schedule. e. Describe how activities and strategies made an impact on identified needs. f. Identify indicators that would best describe the results. g. Identify barriers that had a negative impact on fulfilling the strategies and overall vision. h. Identify whether major goals are on target and discuss reasons for those that are not on target. i. Identify any adjustments or improvements to strategies and activities that might meet your needs more effectively. 1) Both CDD and LAHD, as the primary City departments implementing the Consolidated Plan grants, conduct thorough financial and programmatic monitoring annually in addition to the annual OMB A-133 (Single Audit) as well as periodic Controller audits. The monitoring process used by the two departments varies based on the type of program administered; each of the Consolidated Plan grant have a separate and distinct regulations and requirements. However, both systems are designed to incorporate a variety of monitoring techniques and approaches in a coordinated effort to assure that all funded activities receive an appropriate level of review, and that regulations specific to the Entitlement grants received are followed. The vast majority of the City s Consolidated Plan-funded programs and project activities involve direct services to low- and moderate-income clients. This made it imperative that City departments and related City agencies develop monitoring procedures that will ensure that activities meet HUD regulatory requirements. This is particularly true of the CDBG program. Recipients of CDBG funds must demonstrate that they are continuing to meet both a National Objective and CDBG eligibility requirements. Recipients must also ensure that subrecipients (or subcontractors) are monitored in a consistent and regular manner and that contract goals and terms are adhered to. The following narratives illustrate the approaches used by several City departments in monitoring Consolidated Plan-funded programs and project activities. The CDD Administrative Services Division (ASD) grants unit reviews all applications for CDBG funding for adherence to CDBG regulatory requirements. Staff frequently consults with City departments and agencies on eligibility and national objective issues, and provides technical assistance in assuring that approved project activities continue to meet HUD CDBG statutory and regulatory requirements. Funding and eligibility recommendations are provided by CDD, Grants Section to the Mayor upon review of all proposed projects. Other Consolidated Plan-funded entities (such as LAHSA) that administer grants provide a similarly strict initial review of applications for funding. A standing committee of the City Council (Housing and Economic Development) also reviews CDBG funding requests. CDD grants staff also reviews proposed CDBG programs and projects for readiness, to further assure that activities are begun and completed in a timely manner thereby assuring timely expenditure of CDBG funds. 31st Program Year (2005-2006) CAPER 37 FINAL
In addition, the City Controller s Office and the City Administrative Officer (CAO) conduct indepth audits and performance reviews of federal grant programs. Since the City has two departments responsible for the four grants, each department s approach will be discussed separately. CDD is responsible for monitoring part of the CDBG grant, and those actions of four key operating divisions - ICD, HSFD, NDD, and WDD, which administer third party contracts for CDBG. All four divisions use a Request for Proposal process to identify contractors. Monitoring of Human Services Programs The Human Services and Family Development (HSFD) Division of the Community Development Department (CDD) administers several key CDBG-funded programs, including the Human Services Delivery System, the Domestic Violence Shelter Operations program, and the Rita Walters Learning Complex, among others. In most instances, nonprofit contractors operating these programs are selected through a competitive Request for Proposal process to ensure that contractors have the capacity to properly administer federal grant funds and the organizational capacity to deliver effective services. HSFD staff monitors contracts by conducting a minimum of one comprehensive compliance review each year as well as quarterly site visits. In addition, program analysts are available on an as-needed basis to provide technical assistance to contractors regarding issues such as federal grant requirements, City contract requirements and reporting results of program accomplishments. The HSFD continues to move toward performance-based contract monitoring procedures and is working to redesign the compliance review documents used to better track contractor performance in key areas. Over the past two years, the Financial Management Division (FMD) of the CDD implemented a standard risk assessment (rating) tool to document fiscal performance of all Department contractors. Monitoring by the Department of Aging The Department of Aging receives, in addition to CDBG funds, Federal Older Americans Act funds, State Older Californians Act funds, and City General Fund dollars. The Department is accountable to both the Mayor and City Council, and must comply not only with California Department of Aging regulations, state regulations, and federal regulations as a City entity. The Department must also monitor the limited clientele status of its participants in the Aging Services Delivery System to assure compliance with CDBG regulations, as well as the contract goals and standards for administrative and financial management systems jointly established with its agencies. Through the Management Information System, program activity is measured on a monthly basis, and reported to management. Quarterly, performance measurement reports are prepared and evaluated for each contracted agency in respect to the contracted services. Project activities are carried out through multiple nonprofit organizations located in the City. Project progress is monitored through routine extensive desk review, which consists of reviewing monthly expenditure and service level reports and quarterly program performance reports. These measurements are in place to ensure that contractors are in compliance with the scope of work and units of service as contracted. On-site field visits are conducted quarterly to ensure quality of service provided and mitigation of problems. When problems occur, corrective action plans are initiated and monitoring staff work with service providers to resolve issues. Monitors also provide technical assistance when the need arises. The Department of Aging conducts monthly training for both the Activity coordinator and the ADSC directors. It also has established best practices benchmarks. 31st Program Year (2005-2006) CAPER 38 FINAL
An Equal Access and Safety Protocol (EASP) program has been developed by the Department which includes a Standards of Practice Manuals, program protocol, and protocol tools to assist and facilitate performance by the contracted agencies. Annual training s for program monitoring staff have been instituted to continue development of these tools and ensure regular monitoring. Monitoring of Homeless Programs LAHSA s monitoring of ESG- and CDBG-funded projects consists of both desk and on-site reviews. The desk review is performed monthly through invoice processing and quarterly through progress reports submitted to LAHSA by the service provider. Service providers are required to report to LAHSA on a monthly basis (for Winter Shelter), quarterly basis (for ESG, SHP and CDBG) and to HUD on an annual basis (for SHP). The report tracks the progress made by the service provider in meeting its goals as specified in the contract. The report is submitted to the contract specialist who reviews it and forwards a copy to the planning analyst in charge of reporting. If the contract specialist needs any clarification regarding the report, he/she contacts the program director of the project. In the event that a service provider has not met its contract goal(s), the contract specialist must follow through and inquire why the service provider is not achieving its goal(s). If the implementation of the program demonstrates that the service provider set unrealistic goals, then the service provider may request to modify the goals. However, the service provider must demonstrate that there is a need to modify the program goals. If the service provider does not submit its invoices and/or reports in a timely manner as required by LAHSA, the contract specialist may put the service provider on probation for a specified period of time. The on-site monitoring review requires the contract specialist to conduct a visit to the service provider s facility or premises. The purpose of the on-site review is to verify the information reported to LAHSA during the program and determine overall compliance with the terms of the contract. LAHSA has developed two monitoring tools (services and housing projects checklists) that enable the contract specialist to perform the monitoring visit in a consistent manner. The contract specialists review includes, but is not limited to: Documentation of homelessness; Individual Client Files; Program outcomes as reported in the quarterly reports submitted to LAHSA; Systems in place for quality assurance; and Compliance procedures. The on-site monitoring review is based on the risk level each agency. LAHSA performs a risk analysis to determine the risk level. The on-site visit entails: Issuance of a notification letter to the agency stating the proposed date of the site visit; An entrance conference conducted on the day of the on-site visit prior to the review of documentation. This conference basically summarizes the scope of the review; Documentation of all work papers produced during the on-site visit; An exit conference conducted at the end of the on-site visit. The purpose of the exit conference is to present preliminary conclusions resulting from the review and provide the agency with an opportunity to clarify any issues raised or provide additional documentation; 31st Program Year (2005-2006) CAPER 39 FINAL
LAHSA staff is required to provide the agency with a written monitoring report within 25 days of the monitoring visit. The report is typically positive. It should recognizing areas where the agency has done a good job or shown significant improvement, as well as pointing out areas where corrective action or improvement is required. The monitoring letter should also: Fully and correctly identify the finding; Cite program requirements or applicable regulations that have been violated; Specify corrective actions that must be taken; and Include a deadline for responding to the monitoring letter and also for correcting each finding identified in the monitoring letter. Monitoring by the Department on Disability, AIDS Coordinator (ACO) For CDBG-funded programs administered by the ACO, Department on Disability, program effectiveness and efficiency are continuously monitored through periodic reporting, site visits, and staff participation in funded education and awareness events. For all non-needle exchange contractors, monthly reports are received, including numbers of people served (maintained on Excel worksheets) and a narrative explaining what services were provided and where and when. For needle exchange contractors detailed quarterly computer reports are received showing all activities. Additionally, a site visit is made to each contractor to check on documentation of activity reports and invoices, compliance with all City, state and federal regulations, and to interview all persons being paid under the contract, with a written report prepared for each contractor. Site visits are made to all of the needle exchange sites with the monitor staying for the full duration of the exchange services. The AIDS Coordinator's Office strives to improve its evaluation capacity by working with and coordinating with the Los Angeles County Office of AIDS Programs and Policy; the Los Angeles County Health Department HIV/AIDS Epidemiology Program; local AIDS services agencies; and the Centers for Disease Control and Prevention (CDC) regarding current trends in HIV infection rates and segments of the population most at risk. This includes improvement and refinements to prevention, treatment, and social services programs provided to clients. In addition to monitoring contract compliance, the ACO is actively involved in creating evaluation tools that meet evolving local, state and federal standards to measure the effectiveness and efficiency of City funded programs. The tools are being developed by the office in conjunction with UCLA and other research institutions using universal standards. These tools measure the effectiveness and impact of the intervention being implemented by the provider. Development and implementation of these tools will ensure that City funded programs are held to the same or greater standards as county and state programs. The tools will enable this office to provide better, more targeted technical assistance to providers and ensure that obstacles are overcome before they become chronic problems. It will also ensure that funds are better allocated to programs that reach populations most disproportionately impacted by HIV/AIDS in the City of Los Angeles. Monitoring of Code Enforcement Programs The Pro-Active Code Enforcement (PACE) program inspection teams of the Department of Building and Safety (DBS) are reviewed and closely monitored at several levels of management to ensure that the program s goals are met. 31st Program Year (2005-2006) CAPER 40 FINAL
Similarly, the Nuisance Property Abatement and Demolition program is continually monitored by both the Department s Code Enforcement Bureau and Financial Services Division to ensure funding is accurately and appropriately expended. Monitoring of Housing Programs Monitoring Procedures for Affordable Housing Units - LAHD Affordable housing developments assisted with Consolidated Plan grant funds are monitored in two phases: during construction and afterwards, when the property is occupied by low income residents. Before any funds can be released, loan agreements and related documents must be signed by the Borrower and approved by the City. Besides repayment terms, the promissory note, and loan agreement, the City contract may include additional terms agreed to by the borrower, including requirements related to habitability standards, owner residency, tenant eligibility, and/or rent affordability guidelines. Low-income tenancy and affordable rents are effectuated through a covenant, which is signed by the owner, recorded against the property title, so it runs with the land. Covenants have provisions that require annual tenant re-certification and periodic physical inspections when required by the grant. These additional provisions are no less important than the repayment terms, and a material breach thereof may result in acceleration of the loan and/or foreclosure action against the collateral property. The development cycle begins with the LAHD commitment, and ultimately culminates into a finished housing development. Typically, the development cycle is 30 to 36 months assuming all goes as planned. LAHD utilizes certain milestones as indicators to determine if the project is on track, within the development cycle described above. Those milestones include, but are not limited to the following: Award of leveraged financing sources (tax credit allocation, MHP award or 202/811 HUD Award). As a requirement of the LAHD commitment, each prospective borrower must immediately apply for an award of subsidy from one of the three aforementioned sources and be in receipt of an award not greater than one year after the LAHD commitment is obtained. Project Readiness within 90 120 days of receipt of leveraged financing source as demonstrated by receipt of a building permit, an executed construction contract, complete set of plans and building specifications. Start of Construction as demonstrated by the developer s issuance of a Notice to Proceed. Construction Schedule (generally from 12 18 months depending upon the extent of construction, size of building and other building criteria.) Construction Completion as evidenced by a Notice of Completion, Final Building Sign off by the City s Department of Building and Safety, or a Certificate of Occupancy. Marketing and Lease-Up (generally a three- to six-month process) These milestones are defined in LAHD s loan agreements, and are monitored by both the Finance Officers and Rehabilitation Construction Specialist from application intake, throughout predevelopment and during the course of construction. The construction phase is monitored by LAHD Construction Specialists. They perform inspections at each milestone of the process; their approval is required before funds can be released. The City s Building and Safety Department also plays a role in this process, as it will not issue building permits unless all code requirements have been met. 31st Program Year (2005-2006) CAPER 41 FINAL
After construction, the Occupancy Monitoring section of LAHD takes over the monitoring responsibility as it checks for the required occupancy ratios, and also determines if clients meet income requirements. Compliance monitoring is accomplished through the regular monitoring of a borrower-provided management plan. Owners must provide LAHD with reports on the current tenants and the rent schedule. These are reviewed for compliance by LAHD staff. Housing developments of five or more units receive annual site visits to confirm written reports and to verify the continued maintenance of the building. In addition, LAHD may randomly audit the property owner's financial, tenant and maintenance records to verify compliance with the management plan. When HUD issues revised income or maximum rent levels, all property owners are notified of the changes by the Occupancy Monitoring section. LAHD maintains a listing of all assisted properties with covenant restrictions HOME Grant Monitoring In addition to adhering to CDBG regulations, and administering the HOPWA and ESG programs, LAHD is the administrator of the HOME program. As described above, LAHD follows monitoring standards and procedures to review and fund affordable housing developments to ensure compliance with HUD s program regulations. Here are a few procedures that ensure specific HOME-program requirements: Monitoring Community Housing Development Organizations (CHDOs) CHDOs are evaluated and re-certified by LAHD every year to ensure that organizations continue to meet all of the CHDO requirements. The minimum CHDO set-aside of 15% is calculated annually by LAHD, and allocated to CHDO-sponsored housing development projects. CHDO oversight by LAHD includes an evaluation of compliance with the HOME maximum purchase price/after-rehab value limits, the FHA 203(b) limits, for owner-occupied and homebuyer properties. Other HOME Program Monitoring Activities LAHD reviews the status of the HOME grant to ensure that the 24-month deadline to commit and 5-year deadline to expend funds are complied with. LAHD monitors and reports back on the HOME match requirements to ensure that the 12.5% match requirement is met. An annual report is sent to HUD along with the Consolidated Annual Performance and Evaluation Report (CAPER). LAHD also ensures compliance with the minimum HOME subsidy amount of $1,000 per rental unit as well as the maximum 221(d)(3) per-unit limit subsidy amounts. LAHD ensures that HOME-assisted rental units are inspected at the required frequency of inspections, as stated in the HOME regulations, and ensure compliance with Housing Quality Standards. LAHD also has the responsibility to ensure that homebuyer and rental properties adhere to the applicable period of affordability. During the affordability period, the City will review tenant household incomes and rents for compliance with HOME and other program requirements. Monitoring Procedures for Third Party Contractors Implementing Housing Programs Affordable Housing The process of implementing a housing program with third party contractors begins with a Request for Proposals (RFPs). The RFP specifies program goals, objectives, a schedule, and grant 31st Program Year (2005-2006) CAPER 42 FINAL
restrictions. The latter includes a description of the eligible client population. One section of the RFP will include the standards for the service or product to be provided. On a competitive basis, LAHD will select the program design that has the highest potential of meeting the identified need at a reasonable cost. Once a contractor is selected, an agreement is executed. The agreement will include the scope of work, measurable products, and the standards that must be met to receive compensation and the City's remedies when a contractor fails to perform. All RFPs, Requests for Qualifications (RFQs), and resulting contracts are reviewed and approved by the Mayor and City Council, following analysis by the City Attorney and the City Administrative Officer. All are discussed as agenda items at public meetings, with multiple opportunities for public comment. LAHD staff is assigned to monitor progress toward goals and compliance of the completed product with the contract standard. Monitoring is accomplished through a combination of reports and invoices from the contractor coupled with on-site reviews by LAHD staff. Several Consolidated Plan programs utilize third party contractors for program implementation, including LAHD s array of rehabilitation loan programs which use both HOME and CDBG funds, the Urgent Repair Program which utilizes CDBG funds, HOPWA which uses both HOPWA and HOME funds, and LAHSA, which uses ESG and CDBG funds. Monitoring of Code Enforcement Programs - PACE The Department of Building and Safety generates weekly performance reports which are useful in monitoring the progress of various projects funded through CDBG. Reports are reviewed by all levels of management. In addition, the Community Development Department Environmental Unit approves all work orders submitted by Building and Safety prior to execution and payment through the PEP process. Monitoring Procedures to Comply with CDBG Regulations The Project and Expenditure Plan (PEP), developed by CDD, is designed to address constraints on efficient and effective program implementation. The PEP is an important innovation in strengthening the city s grants management and overall institutional structure, as it replaces informal agreements between CDD and implementing departments, and Memoranda of Understanding previously used for administration of contracted Consolidated Plan-funded activities. City departments must prepare a schedule of projected reimbursement through the four Consolidated Plan grants, and identify all sources of leveraging. The PEP also requires eligibility, labor standards, and environmental review signoffs, and evaluation of progress in meeting its performance goals as the program year progresses; this requirement directly links to the CAPER, and provides a baseline for verification of information contained therein. The City has also adopted a procedure to assure prompt and accurate reconciliation of reprogrammed Consolidated Plan funds with City grant accounting records. Each month, CDD Administrative Services receives an Expenditure Report from the CDD Financial Management Division and project and program data is carefully monitored for timeliness. All City departments are required to submit a PEP form for their approved project(s). No funds are released until the PEP is reviewed and approved by CDD. PEPs are used primarily for CDBG funded programs and project activities, but are also used for UDAG program income-funded projects and Section 108 loan guarantee projects as well. The PEP provides CDD an additional review of approved CDBG projects from the time of application and City Council approval of the Action Plan program. All PEP forms identify the 31st Program Year (2005-2006) CAPER 43 FINAL
following: project budget, project description, eligible activity, national objective, service area census tracts, low/mod percentage, maps, photographs (before pictures for physical improvements), income verification forms for low/mod limited clientele projects, number of jobs created for LMJ projects, and project milestones. Any discrepancies with the approved project description are noted. The PEP also includes an Environmental Checklist for the Environmental Unit to review. PEP training is provided annually to City departments. The training has evolved to present a form that is user friendly, easy to understand, attach documents, and submit for review and approval. In addition, the submitting City Departments demonstrate their understanding of CDBG eligibility guidelines, national objectives, and the required outcome as it relates to their project. For the 31 st Program Year (2005-2006), PEP training was conducted in smaller groups for individual departments, or for departments with similar projects. Training also emphasized record keeping guidelines, per HUD s requirements and expectations, for future monitoring visits. The PEP form not only serves as an alternative to the traditional MOU/MOA (memorandum of understanding/memorandum of agreement) used in the past as a contract document to assure that has all of the fundamental information and attachments required to serve as a remote monitoring tool for Consolidated Plan funded programs and project activities. Over time, the PEP form has developed into a vital tool for reviewing and determining eligibility. The PEP form is a growing and ever changing document, and each year its importance increases. Other City Departmental Audit and Monitoring Responsibilities The City Controller s Office and the City Administrative Officer (CAO) conduct in-depth audits and performance reviews of grant programs. Since the City has two departments responsible for the four grants, each department s approach will be discussed separately. CDD is responsible for monitoring part of the CDBG grant, and those of four key operating divisions - ICD, HSFD, NDD, and WDD, which administer third party contracts for the Community Development Block Grant. All three divisions use a Request for Proposal process to identify contractors. Monitoring of Economic Development Programs The Economic Development Division (EDD) administers the Los Angeles Business Assistance Program (LABAP), which has three components (Micro-enterprise/Entrepreneur, Targeted Retail, and Growth Business), funded entirely with CDBG and each with specific criteria for participation. The LABAP service providers use these standards to select those business or individuals that are seeking services, and they gather the appropriate documentation and keep it on file. The Economic Development Division (EDD) conducts semi-annual monitoring site visits to its LABAP service providers to ensure compliance with local contract requirements as well as HUD and Federal rules and regulations that govern CDBG funded projects. Additionally, EDD ensures that contractual performance goals are met and fiscal procedures are in place to safeguard the CDBG funds. The objective of the LABAP Micro-Enterprise program is to help existing small business to stabilize and increase revenues and profitability. For entrepreneurs, the objective is the establishment of a business, which leads to job creation. For the Targeted Retail program, the outcome is measured by the number of businesses establishing operations in the targeted area. And last, the Growth Business program s objective is to provide assistance to growing companies in the City, with a primary goal of enhancing their economic viability and increasing revenues and operational performance that will lead to the creation of new jobs. 31st Program Year (2005-2006) CAPER 44 FINAL
Monitoring of Public Service and Neighborhood Facility Projects Each City Human Services Program subcontract contains measurable objectives that permit assessment of accomplishments, e.g., standards of effectiveness. Such objectives are, with few exceptions, expressed in terms of both planned monthly performance rates and cumulative contract objectives. These measures permit the City to assess both the short-term (monthly) and long-term (cumulative) performance rates of all projects at any time following project implementation. Data gathered through such assessments are the basis for required corrective action(s) and also serve as support information for future funding recommendations. For 1995 and subsequent years, the City will measure client/family outcomes as well. Besides measuring quantitative performance, City staff also monitors project performance through analysis of monthly subcontractor performance reports. The subcontractor is required to report monthly on specific program performance benchmarks such as number of clients served, client demographics, services provided, and overall subcontractor activities relating to the subcontract. Subcontractor compliance with these program outcome measurements is one of several factors receiving specific monitoring review. As part of this regular monitoring process, the City evaluates service providers compliance with record keeping, financial management, and reporting requirements of their contracts. When problems arise, corrective action plans are initiated. Should improvements not materialize, the City exercises its contractual options to safeguard the expenditure of funds. Project activities are also monitored through review and audit of cash requests from agencies. Regular City monitoring activities are augmented during the program operating cycle by special monitoring activities. Examples of two special City monitoring activities are fiscal procedures review and client eligibility review. 2) The Project Expenditure Plan (or PEP) was developed by CDD Grants staff, and has proven to be a valuable monitoring tool in providing a thorough second level of review for CDBG programs and project activities that were initially found as eligible and meeting a National Objective through the annual Action Plan application and approval process. However, in many cases, review of the PEP has revealed that the program or project scope has substantially changed since its approval in the annual Action Plan, planned accomplishments have changed, or there has been a change in location or beneficiaries, or programs are no longer eligible (in the case of programs and projects approved under Area Benefit criteria) because census tract or block group data was erroneous, or the service area was incorrectly defined. Contracts for the expenditure of CDBG funds cannot be consummated without a reviewed and signed approved PEP form. CDD requires both before and after pictures for many construction projects, and often will conduct on-site reviews to assure that work is completed as described in the submitted PEP. The use of PEPs has been expanded over the past year to include commitments under the Section 108 Loan Guarantee Program and projects involving the first use of UDAG program income, and therefore subject to HUD program regulations. Automation of the PEP form to facilitate its completion on line was substantially complete by May, 2006. The PEP is integrated with the original on-line application for project funding, so that key data is brought forward into the PEP (such as the Scope of Work/Scope of Services). This makes preparation of the PEP by departments and agencies - and subsequent CDD review of the PEP - far more efficient and effective. 31st Program Year (2005-2006) CAPER 45 FINAL
Monitoring has benefited both LAHSA and its agencies in that it provides an opportunity for both parties to assess program performance and make adjustments where necessary. However, with the implementation of the Homeless Management Information System (HMIS) mandated by Congress for all McKinney-Vento funds, LAHSA s monitoring procedures will change significantly. This is because the HMIS is a web-based system that allows users to receive real time information throughout the implementation the program. In addition, LAHSA is implementing its HMIS using an outcome-based approach. This approach emphasizes results rather than activities. Results are measured against performance targets that specify the outcomes of the program. Under this approach, the targets remain constant but the activities performed to reach them may vary. LAHSA plans to use the HMIS to generate monthly reports that are shared with the agencies. The latter may also generate reports from the system at any time during the program year. Data collected from the report will be analyzed by the contracts staff and concerns and recommendations will be communicated to the agencies. This will enable the agencies to review their programs and determine areas that need to be addressed immediately in order to achieve the outcomes set for the program. Therefore, instead of waiting for a monitoring visit to evaluate the agency s performance, both LAHSA and the agency will be proactively monitoring the program. 31st Program Year (2005-2006) CAPER 46 FINAL