Electronic Records Archive: National Archives Needs to Strengthen Its Capacity to Use Earned Value Techniques to Manage and Oversee Development
Highlights
Since 2001, the National Archives and Records Administration (NARA) has been working to develop an Electronic Records Archive (ERA) to preserve and provide access to massive volumes and all types of electronic records. However, in acquiring this system, NARA has repeatedly revised the program schedule and increased the estimated costs for completion from $317 million to $567 million. NARA is to manage this acquisition using, among other things, earned value management (EVM). EVM is a project management approach that, if implemented appropriately, provides objective reports of project status and unbiased estimates of anticipated costs at completion. GAO was asked to (1) assess whether NARA is adequately using EVM techniques to manage the acquisition and (2) evaluate the earned value data to determine ERA's cost and schedule performance. To do so, GAO compared agency and contractor documentation with best practices, evaluated earned value data to determine performance trends, and interviewed cognizant officials.
NARA has, to varying degrees, established selected best practices needed to manage the ERA acquisition through EVM, but weaknesses exist in most areas. For example, the scope of effort in ERA's work breakdown structure is not adequately defined, thus impeding the ability to measure progress made on contractor deliverables. These weaknesses exist in part because NARA lacks a comprehensive EVM policy, training, and specialized resources and also frequently replans the program. As a result, NARA has not been positioned to identify potential cost and schedule problems early and thus has not been able to take timely actions to correct problems and avoid program schedule delays and cost increases. ERA's earned value data trends do not accurately portray program status due to the program's weaknesses in implementing EVM; however, historical program trends indicate that future cost overruns will likely be between $195 million and $433 million to fully develop ERA as planned and between $205 and $405 million at program end. In contrast, the contractor's estimated cost overrun is $2.7 million. Without more useful earned value data, NARA will remain unprepared to effectively oversee contractor performance and make realistic projections of program costs. GAO recommends, among other things, that NARA establish a comprehensive plan for all remaining work; improve the accuracy of earned value performance reports; and engage executive leadership in correcting negative trends. NARA generally concurred with GAO's recommendations.
Recommendations
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| National Archives and Records Administration | To improve NARA's ability to effectively implement EVM on its ERA system acquisition program, and while the current system development contract is active, the Archivist of the United States should direct the NARA CIO to direct the ERA program to establish a comprehensive baseline (through an integrated master schedule) for all remaining work on contract. |
Closed – Implemented
NARA agreed with our recommendation. In February 2011, NARA conducted a review of the ERA program's cost and schedule baseline. Shortly thereafter, NARA finalized its baseline integrated master schedule, which reflects the agency's plan to complete remaining ERA development work on contract by the end of fiscal year 2011.
|
| National Archives and Records Administration | To improve NARA's ability to effectively implement EVM on its ERA system acquisition program, and while the current system development contract is active, the Archivist of the United States should, direct the NARA CIO to ensure that the ERA program obtains reliable EVM performance reports, taking into consideration the data anomalies and weaknesses identified in this report. |
Closed – Implemented
NARA agreed with our recommendation. The agency began conducting more detailed reviews of contractor-provided performance reports. These reviews resulted in NARA identifying a number of data anomalies, including instances where more earned value had been achieved than what had been spent on certain items. In addition, NARA increased outside surveillance support by the Defense Contract Management Agency in reviewing contractor-provided data.
|
| National Archives and Records Administration | To improve NARA's ability to effectively implement EVM on its ERA system acquisition program, and while the current system development contract is active, the Archivist of the United States should direct the NARA CIO to engage senior NARA and contractor leadership/oversight officials to direct attention to reversing current negative performance trends, as shown in the earned value data, and take action to mitigate the potential cost and schedule overruns. |
Closed – Implemented
In response to our recommendation, NARA developed a corrective action plan which outlined its intention to expand presentations and reports to senior NARA executives to include information on cost and schedule. NARA subsequently provided additional information on cost, schedule, and performance in briefings to senior agency officials and conducted monthly presentations to senior NARA and contractor leadership and oversight officials. As a result, NARA management had a more reliable factual basis for subsequent decisions about the ERA program's direction.
|
| National Archives and Records Administration | To improve NARA's ability to effectively implement EVM on its ERA system acquisition program, and while the current system development contract is active, the Archivist of the United States should direct the NARA CIO to include as part of its acquisition policy governing EVM requirements for (1) EVM training for senior executives and program staff responsible for ERA investment oversight and (2) ongoing surveillance of the ERA program's EVM system to ensure its compliance with industry standards. |
Closed – Implemented
As a result of our reporting the problems related to the cost, schedule, and performance of the ERA system, the Office of Management and Budget also took action directing NARA to halt any further development of the ERA system by the end of fiscal year 2011. (See approved GAO accomplishment report GAO-12-93A.) Accordingly, while our specific recommendation is no longer applicable, the action taken to halt the program is consistent with our intent to expend resources wisely.
|
| National Archives and Records Administration | To improve NARA's ability to effectively implement EVM on its ERA system acquisition program, and while the current system development contract is active, the Archivist of the United States should direct the NARA CIO to ensure that the ERA program has the appropriate level of specialized staff in place to perform EVM analysis and oversight activities. |
Closed – Implemented
As a result of our reporting the problems related to the cost, schedule, and performance of the ERA system, the Office of Management and Budget also took action directing NARA to halt any further development of the ERA system by the end of fiscal year 2011. (See approved GAO accomplishment report GAO-12-93A.) Accordingly, while our specific recommendation is no longer applicable, the action taken to halt the program is consistent with our intent to expend resources wisely.
|
| National Archives and Records Administration | Taking into consideration the new ERA program direction, the Archivist of the United States should direct the CIO), using a gap analysis of the work completed through fiscal year 2011, to determine and clearly define the remaining work that will be pursued in the future ERA system development phase (Phase 2 and the original ERA requirements set. |
Closed – Implemented
As a result of our reporting the problems related to the cost, schedule, and performance of the ERA system, the Office of Management and Budget also took action directing NARA to halt any further development of the ERA system by the end of fiscal year 2011. (See approved GAO accomplishment report GAO-12-93A.) Accordingly, while our specific recommendation is no longer applicable, the action taken to halt the program is consistent with our intent to expend resources wisely.
|
| National Archives and Records Administration | Taking into consideration the new ERA program direction, the Archivist of the United States should direct the CIO to direct the ERA program to develop new cost and schedule estimates for a comprehensive Phase 2 baseline, as well as for the total program life cycle. In combination with the above action, this should provide the program with enough information to disclose to the Congress the exact work that will be accomplished and the cost of that work. |
Closed – Implemented
As a result of our reporting the problems related to the cost, schedule, and performance of the ERA system, the Office of Management and Budget also took action directing NARA to halt any further development of the ERA system by the end of fiscal year 2011. (See approved GAO accomplishment report GAO-12-93A.) Accordingly, while our specific recommendation is no longer applicable, the action taken to halt the program is consistent with our intent to expend resources wisely.
|
| National Archives and Records Administration | Taking into consideration the new ERA program direction, the Archivist of the United States should direct the CIO to direct the ERA program to implement the EVM practices that address the detailed weaknesses that we identified in this report, taking into consideration the criteria used, including (1) establishing a comprehensive Phase 2 baseline (through an integrated master schedule) that has been validated through an integrated baseline review and limits the use of nonobjective metrics; (2) ensuring that reliable reports of EVM performance are being produced, including records of work completed, forecasts of estimates at completion, and explanations/corrective actions for variances and data anomalies; and (3) engaging senior NARA leadership/oversight officials to ensure that earned value data are being used for decision-making purposes, including holding and documenting executive meetings to ensure that cost and schedule risks/issues have been tracked to closure, negative performance trends are mitigated, and major updates made to the baseline have been validated through an integrated baseline review. |
Closed – Implemented
As a result of our reporting the problems related to the cost, schedule, and performance of the ERA system, the Office of Management and Budget also took action directing NARA to halt any further development of the ERA system by the end of fiscal year 2011. (See approved GAO accomplishment report GAO-12-93A.) Accordingly, while our specific recommendation is no longer applicable, the action taken to halt the program is consistent with our intent to expend resources wisely.
|