MEMORANDUM FOR THE SECRETARY FROM: John C. Layton Inspector General SUBJECT: INFORMATION: Report on "Audit of Environmental Restoration at the Los Alamos National Laboratory" BACKGROUND: The Los Alamos National Laboratory (Los Alamos) Environmental Restoration Program was primarily monitored by the U.S. Environmental Protection Agency (EPA) and the New Mexico Environment Department (NMED). However, in 1995, the Department of Energy (DOE) and Los Alamos adopted a performance-based management approach to contracting to hold Los Alamos more accountable for its Environmental Restoration Program. At Los Alamos, the Environmental Restoration Program's mission is to remediate sites contaminated with various forms of hazardous, radioactive, and mixed waste produced by past research and development projects. From Fiscal Year (FY) 1991 through FY 1996, Los Alamos spent approximately $386 million on environmental restoration. Of the $386 million, 79 percent, or $305 million, was spent on assessment and program management and 21 percent, or approximately $81 million, was spent on remediation and decommissioning activities. Of the $81 million spent on such activities, $45 million was spent in FY 1996. DISCUSSION: One goal of Los Alamos' Environmental Restoration Program is to expeditiously and cost effectively remediate contaminated sites. To monitor progress toward this goal, the University of California (University) and DOE negotiated eight performance measures. The objective of the audit was to determine whether the contract performance criteria were reasonable, measurable, and complete, thereby allowing DOE to determine if Los Alamos had effectively remediated contaminated sites. 2 The audit determined that Los Alamos did not generate the information needed to assess the cost effectiveness of remediation on a site-by-site basis. This situation occurred because the performance criteria used to evaluate cost effectiveness were not always reasonable, measurable, and complete. As a result, neither Los Alamos nor DOE could evaluate the cost effectiveness or progress of the remediation program or accurately budget for upcoming remediation activities. The audit also determined that Los Alamos' sample validation procedures were too costly because Los Alamos validated more samples than called for by Federal and New Mexico standard practices. While the Office of Inspector General recognizes the importance of prudent sample validation, Los Alamos paid $540,000 more than necessary to validate sample results. These funds could have been better used to remediate contaminated sites. We recommended that the Manager, Albuquerque Operations Office (1) require Los Alamos to track detailed cost data; (2) incorporate a performance measure that would evaluate how well actual costs compare to planned baseline cost data; and (3) require Los Alamos to follow the guidance established by the EPA and the NMED for data validation. Management concurred with the recommendations and agreed to implement substantive changes to Los Alamos' Environmental Restoration Program. Attachment cc: Deputy Secretary Under Secretary U.S. DEPARTMENT OF ENERGY OFFICE OF INSPECTOR GENERAL AUDIT OF ENVIRONMENTAL RESTORATION AT THE LOS ALAMOS NATIONAL LABORATORY The Office of Inspector General wants to make the distribution of its reports as customer friendly and cost effective as possible. Therefore, this report will be available electronically through the Internet five to seven days after publication at the following alternative addresses: Department of Energy Headquarters Gopher gopher.hr.doe.gov Department of Energy Headquarters Anonymous FTP vm1.hqadmin.doe.gov U.S. Department of Energy Human Resources and Administration Home Page http://www.hr.doe.gov/ig Your comments would be appreciated and can be provided on the Customer Response Form attached to the report. This report can be obtained from the U.S. Department of Energy Office of Scientific and Technical Information P.O. Box 62 Oak Ridge, Tennessee 37831 Report Number: DOE/IG-0410 Western Regional Audit Office Date of Issue: July 15, 1997 Albuquerque, New Mexico 87185 AUDIT OF ENVIRONMENTAL RESTORATION AT THE LOS ALAMOS NATIONAL LABORATORY TABLE OF CONTENTS Page SUMMARY........................................... 1 PART I - APPROACH AND OVERVIEW.................. 2 Introduction............................ 2 Scope and Methodology................... 2 Background.............................. 3 Observations and Conclusions............ 3 PART II - FINDING AND RECOMMENDATIONS........... 4 Cost Effectiveness of the Environmental Restoration Program at the Los Alamos National Laboratory............................ 4 PART III - MANAGEMENT AND AUDITOR COMMENTS......... 9 PART IV - APPENDIX................................. 11 U.S. DEPARTMENT OF ENERGY OFFICE OF INSPECTOR GENERAL OFFICE OF AUDIT SERVICES AUDIT OF ENVIRONMENTAL RESTORATION AT THE LOS ALAMOS NATIONAL LABORATORY Audit Report Number: DOE/IG-0410 SUMMARY Los Alamos' Environmental Restoration Program is charged with cost effectively remediating contaminated sites. To monitor progress toward this goal, the University of California, the contractor operating Los Alamos, and the Department negotiated eight performance measures. The objective of this audit was to determine whether the contract performance criteria were reasonable, measurable, and complete, thereby allowing the Department to determine if Los Alamos had expeditiously and cost effectively remediated contaminated sites. The audit determined that Los Alamos did not generate the information needed to assess the cost effectiveness of remediation on a site-by-site basis. This situation occurred because the performance criteria used to evaluate cost effectiveness were not always reasonable, measurable, and complete. As a result, neither Los Alamos nor the Department could evaluate the cost effectiveness or progress of the remediation program or accurately budget for upcoming remediation activities. The audit also determined that Los Alamosm sample validation procedures were too costly because Los Alamos validated more samples than called for by Federal and New Mexico standard practices. While the Office of Inspector General recognizes the importance of prudent sample validation, Los Alamos paid $540,000 more than necessary to validate sample results. These funds could have been used to remediate contaminated sites. We recommended that the Manager, Albuquerque Operations Office (Albuquerque) (1) require Los Alamos to track detailed cost data, (2) incorporate a performance measure that would evaluate how well actual costs compare to planned baseline cost data, (3) require Los Alamos to follow the guidance established by the Environmental Protection Agency (EPA) and the New Mexico Environment Department (NMED) for data validation. Albuquerque concurred with the recommendations. _______/signed/__________________ Office of Inspector General PART I APPROACH AND OVERVIEW INTRODUCTION The University and the Department negotiated a contract which included performance measures. These measures would function as standards against which the University's performance of administrative and/or managerial obligations would be assessed. In addition, the measures would be used to evaluate whether Los Alamos' Environmental Restoration Program expeditiously and cost effectively remediated contaminated sites. The purpose of this audit was to determine whether the performance measures included in the contract were reasonable, measurable, and complete, and; therefore, would make Los Alamos accountable for expeditiously and cost effectively remediating contaminated sites. SCOPE AND METHODOLOGY The audit was conducted at Los Alamos from July 29, 1996, through December 3, 1996. To accomplish the audit objective, we: o reviewed the prime contract between the Department and the University relating to the terms and conditions of the contract; o reviewed laws and regulations as they applied to environmental restoration; o interviewed EPA and NMED officials responsible for monitoring the Los Alamos Environmental Restoration Program; o reviewed reports and general guidance pertinent to performance-based contracts and performance measures; and, o interviewed Los Alamos Area Office and Albuquerque personnel responsible for management of Los Alamos' Environmental Restoration Program. The audit was performed in accordance with generally accepted Government Auditing Standards for performance audits and included tests of internal controls and compliance with laws and regulations to the extent necessary to satisfy the audit objective. We did not rely on computer-generated data in developing this audit because such data did not appear to reflect actual occurrences and transactions. Because our review was limited, it would not necessarily have disclosed all internal control deficiencies that may have existed at the time of the audit. An exit conference was held with Albuquerque officials on December 19, 1996. Followup meetings with Albuquerque were held on April 1 and April 18, 1997. BACKGROUND The Los Alamos Environmental Restoration Program was primarily monitored by the EPA and the NMED. In 1995, however, with the adoption of the performance-based management approach to contracting, the Department began to hold Los Alamos accountable for the achievement of program objectives. At Los Alamos, the Environmental Restoration Program focuses on remediating sites that were contaminated with items like hazardous, radioactive, and mixed waste produced by past research and development projects. From Fiscal Year (FY) 1991 through FY 1996, Los Alamos spent approximately $386 million on environmental restoration. Of the $386 million, 79 percent, or $305 million, was spent on assessment and program management and 21 percent, or approximately $81 million, was spent on remediation ($48 million) and decommissioning activities ($33 million). OBSERVATIONS AND CONCLUSIONS In numerous discussions throughout the audit, officials from Albuquerque's Environmental Restoration Division expressed their desire to work with Los Alamos toward improving performance measures. These officials stated that while the measures agreed to in Fiscal Years 1995 and 1996 were a good starting point, future refinements were needed to assure that environmental restoration goals were cost effectively and expeditiously met. In his response to an earlier draft of this report, Albuquerque's Assistant Manager for Environment/Project Management also suggested additional techniques for improving financial accountability, including the implementation of a work authorization procedure for environmental restoration work. While we did not specifically examine the merits of a work authorization system during the audit, we support mechanisms to ensure the authorization of specific remediation tasks and related internal controls that help assure accountability. We were appreciative of management's comments regarding its commitment to enhancing cost effective cleanup at Los Alamos. PART II FINDING AND RECOMMENDATIONS Cost Effectiveness of the Environmental Restoration Program at the Los Alamos National Laboratory FINDING The contract between the University and the Department requires Los Alamos to expeditiously and cost effectively remediate contaminated sites. To monitor progress toward this goal, the parties negotiated eight performance measures. The audit determined, however, that Los Alamos did not generate the information needed to assess whether specific sites were remediated in a cost effective manner. This situation occurred because the performance criteria used to evaluate cost effectiveness were not always reasonable, measurable, and complete. As a result, neither Los Alamos nor the Department could evaluate the cost effectiveness or progress of the remediation program or accurately budget for upcoming remediation activities. The audit also showed that Los Alamosm sample validation procedures were not cost effective because Los Alamos validated more samples than called for by Federal and New Mexico standard practices. Los Alamos paid up to $540,000 more than necessary to validate results. These funds could have been used to remediate contaminated sites. RECOMMENDATIONS We recommend that the Manager, Albuquerque Operations Office: Require Los Alamos to provide actual detailed remediation cost data on a site-by-site basis. Incorporate a performance measure that would evaluate how well actual costs compare to planned baseline cost data. Require Los Alamos to follow the guidance or tolerances established by the EPA and the NMED for data validation. MANAGEMENT REACTION Albuquerque concurred with the recommendations. Management comments are summarized in Part III. DETAILS OF FINDING In 1993, the Secretary of Energy formed a Contract Reform Team to evaluate the Department's contracting practices and to formulate specific proposals for improving those practices. Their results were published in Making Contracting Work Better and Cost Less, which noted several weaknesses in the Department's contracting system and encouraged the use of performance-based contracts. In discussing performance-based contracts, the report identified key elements including the necessity to have: o clearly stated, results-oriented performance criteria and measures; and, o improved financial accountability. The need to measure performance is also addressed in the Government Performance and Results Act of 1993. In addition, the Information Technology Management Reform Act of 1996 requires each Federal agency to benchmark, where possible, its performance in terms of cost, speed, productivity, and quality of outputs and outcomes. Also, the Chief Financial Officers Act of 1990 requires that each Federal agency develop and maintain an integrated agency accounting and financial management system including financial control which provided for the systematic measurement of performance. In negotiating the contract for Los Alamos, the Department and the University agreed to include performance objectives and measurements. One overall objective was that Los Alamos' Environmental Restoration Program would expeditiously and cost effectively remediate contaminated sites. In addition, the University was required to maintain "current cost information to reflect the cost of performing the work . . . ." Finally, the contract stipulated that if the University did not meet specified baseline objectives, the Department could modify the contract. This modification would allow the Department to seek alternative management mechanisms for the Environmental Restoration Program. DETAILED COST DATA Although Los Alamos budgeted for remediation on a site-by- site basis, it did not track detailed cost data needed to assess the cost effectiveness of specific remediated sites or activities. In addition, sample validation, the one related activity with traceable cost data, was not cost effective. Los Alamos used a baseline document, which included detailed cost estimates by remediation site, to request funds for its Environmental Restoration Program. After receiving the funds, however, Los Alamos tracked and reported costs in the aggregate rather than breaking the costs down by site. Consequently, site- by-site evaluation of cost effectiveness was impossible. A review of ten remediated sites, for example, showed that Los Alamos did not have detailed data to support the costs of four common remediation activities (1) planning, (2) assessment, (3) characterization, and (4) actual clean up. As a result, Los Alamos did not have the detailed data necessary to conduct meaningful comparisons between estimated baseline costs and actual costs. The audit confirmed the results of a 1995 Independent Technical Review of the Environmental Restoration Program. This review also showed that the lack of historical cost data prevented comparisons of the estimated cost of remediation with actual costs. Without such comparisons, the cost effectiveness of specific remediation activities cannot be determined. We discussed the need for tracking and reporting detailed costs with Los Alamos officials. They stated that achieving this level of accountability would cost approximately $1 million and 10 full time equivalents. Los Alamos could not fully document the expenditures it claimed would be necessary to enhance accountability for remediation costs. However, even if the Los Alamos estimate is accurate, the benefits may be well worth the costs given the magnitude of future environmental remediation expenditures. Difficulties in determining actual site specific costs were further compounded because Los Alamos replaced 26 of the 52 individual sites that it had originally agreed to remediate during FY 1996. While these substitutions may have been necessary for technical or programmatic reasons, they were made without the Department's prior approval. Further, the Department could not document whether the substituted sites compared favorably to the original sites either in cost or priority ranking. In fact, a Los Alamos official stated that many of the 26 substitutions were made based on the need to meet a numeric quota for specific site remediations, not because of scientific or technical reasons. These substitutions, combined with the absence of detailed cost data, render impossible a judgment on whether the $29 million spent on remediation activities during FY 1996 represented optimal use of the Department's environmental restoration funds. PERFORMANCE MEASURES The inability to assure cost effective site redemption occurred because the performance criteria used by the Department and Los Alamos were not always reasonable, measurable, and complete. Reasonable and objective performance measures would have included benchmarks based on historical data or estimates. However, the Department negotiated the performance measures for the number of sites to be remediated prior to the completion of a baseline document which included such estimates. The baseline ultimately provided the number of sites (52) to be remediated in FY 1996, as well as a detailed cost estimate for each site. However, the baseline document was not available until approximately one month after the performance measures had been negotiated As a result, the performance measures did not appear to use the information available in the baseline document and were not reasonable as to the number of sites to be remediated for the year. For performance measures to be effective, both the Department and Los Alamos needed accurate and timely information on actual costs. However, the Los Alamos performance measures for financial accountability did not produce results-oriented data. Performance measure number 6, for instance, required Los Alamos to compare planned to actual cost. However, the Department did not require Los Alamos to collect the detailed cost data necessary to perform the required comparison. Performance measure number 6, therefore, could not produce any results-oriented measure- ments that would allow the Department to determine whether specific remediation activities were cost effective. Finally, the performance measures were not complete. For completeness, one or more of Los Alamos' performance measures should have contained a rating factor on financial accountability. However, no performance measure included any cost effectiveness rating factors even though cost effective remediation was an overall goal. FINANCIAL ACCOUNTABILITY Since Los Alamos did not provide detailed cost data, neither the Department nor Los Alamos could determine whether specific remediation activities were performed in a cost effective manner. The Department also needs detailed cost data in order to compare the cost of remediation projects at Los Alamos to the costs incurred for remediation projects at other Department facilities and the private sector. Such comparisons would enable the Department to determine if its remediation funds are spent cost effectively and whether its environmental restoration expenditures are being optimized. Finally, without this cost data, the Department cannot realistically budget for future remediation projects. SAMPLE VALIDATION In contrast to its practice of not tracking site-specific cost data, Los Alamos maintained detailed cost information related to sample validation. This data showed, however, that sample validation was not cost effective. Los Alamos' validation practices were, in fact, excessive and duplicative. For example, Los Alamos routinely validated 90 to 100 percent of sample data returned from analytical laboratories. Further, in order to meet reporting deadlines, Los Alamos routinely performed in-house data validation while simultaneously paying to have the same data validated by a local subcontractor. Since both validations produced the same results, the duplication of effort did not provide added value. Los Alamos did not use guidance available from the EPA or NMED to establish its validation program or procedures. An EPA official stated that the EPA validates about 10 percent of its sample data from remediation sites and generally considers this level acceptable. The EPA's "Quality Assurance/Quality Control Guidance" most stringent requirement for validation efforts was to validate at least 10 percent of the data. Moreover, a Headquarters Department official stated that Departmental policy was to comply with guidelines established by the NMED. New Mexico's guidelines incorporated EPA's standards. Although an NMED official noted that an acceptable level of validation was a gray area, he stated that the EPA notified his department that at least 10 percent of sample data should be validated. Los Alamos spent more than necessary for sample validation. For instance, although Los Alamos planned to spend only $300,000 for data validation, it actually spent $600,000 in FY 1996. Validating 10 percent of the samples, criteria acceptable to EPA and NMED, would have saved Los Alamos approximately 90 percent of the $600,000 actual costs paid to a local subcontractor for data validation, or $540,000. We recognize that data validation is an important aspect of a successful cleanup program. Nevertheless, unnecessary expenditures on validation take away funds that might otherwise be available for direct remediation of contaminated sites. PART III MANAGEMENT AND AUDITOR COMMENTS Albuquerque's Assistant Manager for Office of Environment/Project Management commented on the report and concurred with the recommendations. The Assistant Manager's comments are summarized below. Recommendation 1 Require Los Alamos to provide actual detailed remediation cost data on a site-by-site basis. Management Comments. Albuquerque concurred, indicating that Los Alamos will be required to provide detailed cost data on a site- by-site basis for remediation sites. According to the Assistant Manager, collecting all costs at a lower level may not be a cost effective effort. Therefore, the Department will proceed first with the limited approach of collecting costs for higher value remediations, and then evaluating the results to determine cost effectiveness. The sites that will be targeted for this effort will be the sites that Los Alamos contracts out under a fixed price or fixed unit price subcontract. Costs for these sites will be collected at the next Work Breakdown Structure level below the site level. Contracts for remediation will be modified to require the Los Alamos subcontractors to collect and report these costs at this lower level. This information will be reported by Los Alamos quarterly. Auditor Comments. Albuquerque's comments and proposed actions are responsive to the recommendation. Recommendation 2 Incorporate a performance measure that would evaluate how well actual costs compare to planned baseline cost data. Management Comments. Albuquerque concurred, stating that Los Alamos and the Department plan to develop an appropriate performance measure for FY 1998. According to the Assistant Manager, the performance measure will focus on the goal of determining whether Los Alamos is expending funds in accordance with the Department-approved baseline. Auditor Comments. Albuquerque's comments and proposed actions are responsive to the recommendation. Recommendation 3 Require Los Alamos to follow the guidance established by the EPA and the NMED for data validation. Management Comments. Albuquerque concurred. According to the Assistant Manager, the Department has a benchmarking study underway on the sample management operations of Los Alamos, Sandia National Laboratories, and Pantex. The objective of the study is to determine if the data validation efforts by the three facilities are appropriate. The study will determine if Los Alamos is performing the appropriate level of data validation while following relevant guidance. Auditor Comments. Albuquerque's comments and proposed actions are responsive to the recommendation. Cost Savings Albuquerque concurred with the OIG's estimated monetary impact. PART IV APPENDIX Related Office Of Inspector General Audit Reports DOE/IG-0293 - Audit of Testing Laboratory Support to the Environmental Survey Program DOE/IG-0294 - Environmental Training at the Department of Energy DOE/IG-0298 - Department of Energy's Waste Minimization Program DOE/IG-0401 - Inspection of the Performance Based Incentive Program at the Richland Operations Office WR-B-94-06 - Subcontracting for Environmental Services at Los Alamos National Laboratory Report No. DOE/IG - 0410 CUSTOMER RESPONSE FORM The Office of Inspector General has a continuing interest in improving the usefulness of its products. We wish to make our reports as responsive as possible to our customers' requirements, and therefore ask that you consider sharing your thoughts with us. On the back of this form, you may suggest improvements to enhance the effectiveness of future reports. Please include answers to the following questions if they are applicable to you: 1. What additional background information about the selection, scheduling, scope, or procedures of the audit or inspection would have been helpful to the reader in understanding this report? 2. What additional information related to findings and recommendations could have been included in this report to assist management in implementing corrective actions? 3. What format, stylistic, or organizational changes might have made this report's overall message more clear to the reader? 4. What additional actions could the Office of Inspector General have taken on the issues discussed in this report which would have been helpful? Please include your name and telephone number so that we may contact you should we have any questions about your comments. Name ______________________________ Date______________________ Telephone _______________________ Organization__________________________ When you have completed this form, you may telefax it to the Office of Inspector General at (202) 586-0948, or you may mail it to: Office of Inspector General (IG-1) U.S. Department of Energy Washington, D.C. 20585 ATTN: Customer Relations If you wish to discuss this report or your comments with a staff member of the Office of Inspector General, please contact Wilma Slaughter at (202) 586-1924.