Exploratory Study of the Accuracy of Home Mortgage Disclosure Act (HMDA) Data Final Report Cambridge, MA Lexington, MA Hadley, MA Bethesda, MD Washington, DC Chicago, IL Cairo, Egypt Johannesburg, South Africa Contract #C-OPC-5978 Task Order No. 5 April 1999 Prepared for Mr. John M. Goering U.S. Department of Housing and Urban Development 451 Seventh Street, SW, Room 8144 Washington, DC 20410-3000 Abt Associates Inc. 55 Wheeler Street Cambridge, MA 02138 Prepared by Judith D. Feins, Ph.D. Christopher E. Herbert, Ph.D. Christine Connolly
This is the final report of an exploratory study of the accuracy of Home Mortgage Disclosure Act (HMDA) data, carried out for the Program Evaluation Division and the Housing Finance Analysis Division of HUD's Office of Policy Development and Research (PD&R). Under Task Order #5 of Contract C-OPC-5978, Abt Associates has assisted HUD to address various questions related to HMDA reporting. The assistance has taken two forms: audits to compare the data reported under HMDA with the information in lenders' files, and analytic support to examine additional issues related to HMDA reporting. Background The Home Mortgage Disclosure Act became law in 1975, in response to concerns about mortgage credit shortages in older urban neighborhoods, particularly those with a predominantly minority population. This lack of mortgage credit was believed to contribute to the decline of these urban neighborhoods. HMDA was intended to provide public information that would help show whether financial institutions were serving the housing credit needs of the communities in which they operated. In 1990, non-depository mortgage lenders were brought under HMDA s jurisdiction, ending a key omission from the data, since mortgage companies were an increasingly important part of the industry. HUD was named as the agency responsible for the collection of HMDA data from these entities, in line with its existing responsibility for the oversight of FHA-insured lending by the mortgage banking industry. This study focuses on non-depository lenders that submit HMDA data to HUD. Information on the race and income of the applicant was required through the HMDA changes made in 1990, as was information on the characteristics and disposition of each loan application (whether it was approved, denied, withdrawn, or not completed). These changes made the HMDA data useful for examining numbers and volumes of mortgage applications by race and income, as well as proportions of loan originations by these same factors. One of the most valuable aspects of the HMDA data is the information reported on property location, which includes the state, county, MSA, and census tract. HMDA data have been used extensively over the years to examine the volume and composition of home loans in various communities. As the uses of HMDA data have diversified, the data and the statistical methodology have come under more careful examination. The Federal Reserve Board (along with the other supervisory agencies that receive HMDA data from lenders) has made significant attempts to control for reporting errors, by screening statistically for outliers and by requiring lenders to correct obviously incorrect or inconsistent reports. In essence, the test has been, Is it reasonable? Abt Associates Inc. Final Report i
Through this task order, HUD is extending the inquiry by asking the questions: Is it correct? and Is it complete? To answer these questions, an exploratory analysis for a small number of lenders was conducted that compared the HMDA reports with the paper loan folder kept by the lender. The object of these lender audits was to verify that the HMDA reports from the lender were accurate and complete, based on the documentation in the loan folder. If they were not, the audits probed why, by gathering information on the process of reporting and by discussing the audit findings with officials of the lending institution. The results of this analysis may assist HUD in guidance to reporting institutions and in its use of HMDA data. Scope and Applicability of Study Findings This study was intended to be exploratory in nature, rather than generalizable to part or all of the lender population reporting under HMDA. With a very small number of field audits of nondepository lenders under HUD s jurisdiction for HMDA reporting (three in total) and a limited sampling window for the files examined in the audits, the quantitative analyses are meant at most to serve as a prototype for future research. While the analyses may be suggestive of patterns in HMDA reporting, we cannot be sure that any such patterns hold in the larger lender population or in the full HMDA data set. We cannot identify any specific bias likely to result from the selection of these three particular lenders, but the lenders were not scientifically sampled. The study s value lies far more in the experience of lender field auditing and in the questions raised about the HMDA reporting requirements and the instructions that guide lenders in complying with them. All three of the lenders audited for this study were non-depository institutions with responsibility for reporting HMDA data to HUD. Therefore, the study s findings are at best of limited applicability to HMDA reporting by depository institutions. Depository institutions are under the jurisdiction of financial institution regulators, including the Federal Reserve, Office of the Comptroller of the Currency (OCC), the Federal Deposit Insurance Corporation (FDIC), the Office of Thrift Supervision (OTS), and the National Credit Union Administration (NCUA). Their compliance with HMDA reporting requirements is included in the regulatory audits performed by these oversight agencies, although little information is available on the results of regulatory audits in this area. On the other hand, although HUD is responsible for collecting the HMDA data from nondepository lenders, HUD has no supervisory review authority for non FHA-approved lenders that includes on-site comparison of data from the HMDA Loan/Application Register (LAR) with loan files. As a result, compliance of these lenders with HMDA reporting is only monitored to assure that the data HUD receives are free of validity errors, meaning they contain only valid values for each field. Any such errors flagged during HUD processing are reported to the lender, which must provide a corrected LAR back to HUD. But there is no guarantee that valid entries are a valid representation of the loan application. This study represents the first effort to conduct onsite comparisons between a non-depository lender s paper loan files and the data reported to HMDA corresponding to those files. It thus offers an opportunity to look beyond HUD s current monitoring of HMDA compliance. ii Final Report Abt Associates Inc.
Findings From the Audit Data Based on field audits of some 420 loan files for three non-depository lenders, comparisons between the loan file data and reported HMDA data showed the following: 1) The field audit of three non-depository lenders revealed substantial gaps and irregularities in HMDA reporting coverage, relative to the loan files produced for the audit. One lender's files revealed sizeable under-reporting of applications. Another lender's files showed that a notable share of reported loans were not for housing purposes (and should not have been reported). The third lender's files revealed a large share of loans reported as purchased (from a broker) rather than originated, even though this lender had clearly done the underwriting. This reporting error resulted in loss of data in a number of important fields. 2) Substantial numbers of cases showed incorrect non-reporting of data for various fields in the lenders' HMDA submissions. The great majority of these cases were due to entire applications being incorrectly not reported or being erroneously reported as a purchased loan instead of an origination. Apart from the nonreporting for entire applications, property location and borrower characteristics variables showed the highest proportions of incorrect omissions from the reported data (5 to 15 percent for property location, 17 to 25 percent for borrower characteristics). 3) Estimated error rates for HMDA variables, based on the audits of lender files, were predominantly under the 5 percent threshold set by the Federal Reserve for these data. However, error rates for certain fields were significantly above the threshold in statistical terms. For all three sampled lenders and the pooled sample, error rates were significantly above the 5 percent threshold for these fields: income (26.3 percent), census tract (14.2 percent), application date (42.3 percent), loan purpose (17.2 percent), lender action (23.1 percent), and action date (12.8 percent). Error rates were generally low with regard to the protected class (race and sex) variables. 4) Comparison of reported with corrected property location information did not reveal significant patterns of bias in the racial or economic characteristics of census tracts. There was a slight tendency for tract errors to result in reporting tract income levels lower than the true levels and tract minority population shares higher than the true levels, but there was great variability and these differences were not statistically significant. Errors were almost as likely to raise tract income as to lower it, and almost as likely to lower minority share as to increase it. Abt Associates Inc. Final Report iii
5) Comparison between the reported and the corrected HMDA data (for these lenders and sampled files) revealed substantial impacts on the volume of loan applications and the dollar value of loan activity represented, with much smaller effects on other variables. The HMDA data for the audited files were corrected for the combination of omitted files, incorrect variable non-reporting, and errors in reported fields. The major effects of correcting the errors were to increase by about 9 percent the total number of loan applications and to increase even more (by 28 percent) the total number of loan originations. The average loan amount rose by 12 percent, the total value of loan applications rose by 26.3 percent, and the total value of originations rose by 61.7 percent. 6) Although the data on borrower characteristics were not greatly affected by the errors and omissions, the corrected data reveal sharp reductions in the denial rates for minority applicants, compared to the data originally reported. The minority applicants in the audit sample were primarily African-American (73 percent of applicants, as reported) or Hispanic (17 percent of applicants, as reported). Once the race data are corrected, these figures change to 65 percent and 21 percent, respectively. As originally reported, the loan denial rate for minority applicants was 19.5 percent. After corrections to both the race and lender action variables (based on the loan file contents), the minority denial rate fell to 9.7 percent. This was substantially below the corrected white denial rate of 16.8 percent. 7) Errors were also detected in the reporting of loan sales to the Government-Sponsored Enterprises (GSEs), which could lead to substantial errors measuring the secondary market coverage of HMDA data. In this small sample, one lender s coding errors led to over-reporting of loan sales to Fannie Mae. The corrected data showed that GSE purchases totaled only $609,000 instead of $1,625,000, reducing the dollar volume by nearly two-thirds. It is unclear how widespread this type of error may be. But since HMDA is one of the most important data sources HUD uses for measurement and analysis of GSE activity, the consequences of this type of error--if widespread--could be significant. Findings Concerning the Methodology of Lender Field Audits This study was also a test of the field audit approach to assessing the accuracy of reported HMDA data. Regarding methodology, the research found the following: iv Final Report Abt Associates Inc.
1) It is feasible to use the field audit technique to compare HMDA-reported data with the contents of lenders paper loan files. The contents of paper loan files appear generally sufficient to allow evaluation of the accuracy of the full range of HMDA variables. However, files for originated loans are clearly more complete than those for loans with other dispositions; in particular, the latter may be missing documentation about the way the transaction ended. The feasibility of conducting a field audit of any particular lender depends on whether that lender s paper files are gathered and kept in a centralized storage location. Many non-depository lenders have large numbers of branches across the country; if loan files are not centralized, the costs of locating and obtaining a copy of the file may be heavy both for the lender and for the auditor. Feasibility is also dependent on stored files being organized by an identifying number or by date. 2) HUD would need to expand its supervisory authority to carry out a wider and more representative HMDA field audit of non-depository lenders. With regard to the non-depository institutions whose HMDA reporting is supervised by HUD, it appears unlikely that voluntary cooperation with a field audit would be sufficient to enable a fuller audit for research purposes. Recruitment of just three lenders for this study proved very challenging. In order to achieve a representative sample of lenders and a robust sample of their HMDA-covered files, HUD would need to require lender participation. However, HUD only has authority to require on-site inspections for FHA-approved lenders. For non FHA-lenders, voluntary cooperation would still be needed, absent increased enforcement authority for HUD. Recommendations for HUD on HMDA Data Reporting As noted by the General Accounting Office (in their 1996 report Fair Lending: Federal Oversight and Enforcement Improved but Some Challenges Remain), enforcement of HMDA reporting requirements by the responsible federal agencies has largely been limited to requiring institutions to correct and resubmit data. In fact, all the HMDA data used in this study (to which the loan file contents were compared) passed the validity tests for HMDA reports. But the procedures currently in use for processing HMDA submissions cannot detect the kinds of errors identified here. The errors detected in this study include substantial proportions of mis-coded borrower characteristics, lender actions, and property locations. These inaccuracies cannot become the subject of enforcement actions, such as the imposition of civil money penalties, unless and until there is a means of detecting them. For the non-depository lenders (and particularly those that are not FHA-approved), no such means exists. Abt Associates Inc. Final Report v
In light of this consideration, and based on this study s analysis of the factors shaping HMDA data (both factors internal to the sample lenders and more general industry-wide factors), we make the following recommendations to HUD concerning HMDA data reporting: Consider seeking enforcement authority over all non-depository institutions that may be required to submit HMDA reports to HUD. While HUD is responsible for gathering HMDA data from all non-depository institutions, HUD can neither require on-site inspections nor impose penalties for HMDA reporting violations by non FHAapproved lenders. Absent this sanction, lenders have little incentive to prepare accurate and complete HMDA reports. Consider collecting data on corporate structure and relationships to understand the links between or among various reporting entities. The audits revealed that corporate relationships can strongly influence HMDA reporting. For example, loans originated by a company and sold to its sister subsidiary may or may not be reported on both the origination and the purchase sides, depending on how the HMDA reporting entities correspond to the corporate entities. Collecting these data would require lenders to report (for federal agency use only) all corporate links among different HMDA reporters, regardless of the supervisory agency to which the entity submits HMDA data. Suggest to the Board of Governors of the Federal Reserve System that HMDA be expanded to add indicators or fields for subprime loan grade and/or the terms and conditions of the loan. Because many lenders offer a range of loan products, including prime and subprime loans, with a variety of terms, this information will better characterize the credit being offered. Suggest to the Board of Governors of the Federal Reserve System that the rules concerning home equity loans be clarified; make them consistent between closed-end and credit lines and include or exclude them, rather than keeping them optional as under the present rule. Emphasize the exclusion of home equity loans for debt consolidation only. Suggest to the Board of Governors of the Federal Reserve System that clear and comprehensive instructions be issued on the definition and handling of mortgage prequalifications and pre-approvals for HMDA reporting purposes. As the Fed indicated in recent rule-making, neither pre-qualifications (involving a cursory review) nor denials of applications for pre-approvals (involving full underwriting) are subject to reporting. However, these practices are growing, and it is important that the HMDA data accurately reflect the full set of decisions of lenders about access to mortgage credit. vi Final Report Abt Associates Inc.
We have also identified a number of ways to help lenders improve their HMDA reporting accuracy: Provide clearer instructions and examples with regard to reporting responsibility for wholesale loans. There appears to be widespread confusion and probable error as to which entity should report originations and which should report purchases. The lenders are largely unaware of the degree to which HMDA reporting of wholesale loans is inconsistent across the industry. Consider conducting annual audits of a sample of lenders to gather information on data quality and to provide suggestions for improvements in reporting practices. Assure sufficient funding and staff to enable regular contact with HMDA reporters. HMDA data are regularly used by the Office of Fair Housing at HUD and by the Justice Department in monitoring fair lending compliance. They are also heavily used by HUD s research staff as a critical source of information on trends in mortgage markets and for oversight of the GSEs. The importance of the HMDA data needs to be reflected in the effort HUD invests to ensure that the data are comprehensive, complete, and accurate. The exploratory nature and limited scope of this study must be recalled in reviewing the findings and recommendations made here. But even without assuming that the findings are fully generalizable, the study raises important questions about HMDA reporting and the quality and interpretability of the data. These questions deserve to be addressed by HUD and the other supervisory agencies, through a more comprehensive assessment of reporting practices and data accuracy. Abt Associates Inc. Final Report vii