2000 Census: Coverage Evaluation Interviewing Overcame Challenges, but Further Research Needed
Highlights
As part of its Accuracy and Coverage Evaluation (ACE), the U.S. Census Bureau interviewed people across the country to develop an estimate of the number of persons missed, counted more than once, or otherwise improperly counted in the 2000 census. In conducting the interviews, which took place in person or over the phone, Census faced several challenges, including (1) completing the operation on schedule, (2) ensuring data quality, (3) overcoming unexpected computer problems, (4) obtaining a quality address list, and (5) keeping the interviews independent of census follow-up operations to ensure unbiased estimates of census errors. The Bureau completed the interviews largely ahead of schedule. On the basis of the results of its quality assurance program, the Bureau assumes that about one-tenth of one percent of all cases nationally would have failed the program because they were believed to have been falsified. Early on, the Bureau dealt with an unexpected problem with its automated work management system, which allows supervisors to selectively reassign work among interviewers. According to the Bureau officials, the Bureau addressed the underlying programming error within two weeks, and the operations proceeded on schedule. The address list used for interviews had fewer nonexistent listings than did the lists used by the major census questionnaire delivery operations. An accurate address list is important to prevent unnecessary and costly efforts to locate nonexistent addresses. Although the Bureau implemented controls to keep the nonresponse operation separate from the interviews, the assumed independence of the census and ACE was put at risk because another follow-up operation intended to improve census coverage overlapped with the interviews.
Recommendations
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Commerce | As the Bureau documents its lessons learned from the 2000 Decennial Census and as part of its planning efforts for 2010, the Secretary of Commerce should conduct research that determines the relationship, if any, between operational measures of person interviewing, such as proxy rates, and the accuracy of A.C.E. estimates of census undercounts as planned. |
Closed – Implemented
In March 2003, the Bureau reported that proxy response was related to unresolved or less complete answers to important coverage measurement questions during the A.C.E. in 2000. The findings of the March 2003 report were echoed in subsequent Bureau research used to inform the planning for the Bureau's coverage measurement program for the 2010 Census. For example, in a February 2004 report, the Bureau noted that "The 2010 coverage measurement program should minimize proxy interviews and only accept interviews from knowledgeable respondents."
|
| Department of Commerce | As the Bureau documents its lessons learned from the 2000 Decennial Census and as part of its planning efforts for 2010, the Secretary of Commerce should conduct research that determines how best to define, measure, and report interview quality failure rates that include interviews rejected for all reasons, and not just for a subset of reasons such as falsification. |
Closed – Implemented
The Bureau concurred with GAO's recommendation and developed an action plan for implementation. The action plan noted that for future coverage evaluation operations, the Bureau is to develop a comprehensive glossary of terms, definitions, and formulae to be used by all Bureau divisions working on the project. This could prevent the use of different terminology across the Bureau and help the Bureau in evaluating results and procedures.
|
| Department of Commerce | As the Bureau documents its lessons learned from the 2000 Decennial Census and as part of its planning efforts for 2010, the Secretary of Commerce should conduct research that determines and documents the extent, if any, of the actual overlap between census follow-up operations and A.C.E. person interviewing in 2000. |
Closed – Implemented
In May 2006, the Bureau issued an action plan for the 2010 Research and Development Planning Group on Coverage Measurement. In the plan, the Bureau indicated that, during the 2006 test, person interviewing will be scheduled such that it may occur before or around the same time as the census coverage follow-up interview. In its August 2005 evaluation study plan, the Bureau indicated it would determine whether person interviewing is adversely affecting census results. Part of this analysis will determine the extent to which these operations overlap during the 2006 test. This analysis will provide information about the likely level of overlap in Census 2000.
|
| Department of Commerce | As the Bureau documents its lessons learned from the 2000 Decennial Census and as part of its planning efforts for 2010, the Secretary of Commerce should conduct research that determines whether sufficient overlap may have occurred to violate the independence assumption. |
Closed – Implemented
In August 2002, the Bureau issued an evaluation, Contamination of Census 2000 Data Collected in Accuracy and Coverage Evaluation Block Clusters, which examined whether the A.C.E. contaminated Census data collected in A.C.E. blocks. The Bureau stated that contamination is another way of referring to instances when the independence assumption fails. This evaluation found that contamination bias was not a problem during Census 2000.
|
| Department of Commerce | As the Bureau documents its lessons learned from the 2000 Decennial Census and as part of its planning efforts for 2010, the Secretary of Commerce should conduct research that determines whether increasing the flow of status data on specific decennial follow-up operations to the managers of independent surveys can help ensure the independence of such surveys, particularly when such operations are scheduled to overlap in the field. |
Closed – Implemented
In its May 2006 Action Plan for the 2010 Research and Development Planning Group on Coverage Measurement, the Bureau indicated that its Survey Design Team will develop a high-level design for 2010 that shows functional and chronological relationships among the operations needed by the coverage measurement survey. This design will be assessed based on the results of the 2006 test. The Operations Team will also research issues surrounding the timing of A.C.E. field operations and independence from census field operations.
|
| Department of Commerce | As the Bureau documents its lessons learned from the 2000 Decennial Census and as part of its planning efforts for 2010, the Secretary of Commerce should conduct research that determines what additional steps or controls to preserve the independence of census follow-up and person interviewing, if any, could be implemented for other census follow-up operations that collect enumeration data and are scheduled contemporaneously with person interviewing. |
Closed – Implemented
In its May 2006 Action Plan for the 2010 Research and Development Planning Group on Coverage Measurement, the Bureau indicated that its Survey Design Team will develop a high-level design for 2010 that shows functional and chronological relationships among the operations needed by the coverage measurement survey. This design will be assessed based on the results of the 2006 test.
|