—Unknown

In This Section

u Action Checklist

uTips

uProcess in Action: Examples from the Field

Potential Health Measures

Setting Targets for Objectives

Measuring Progress

Evaluating Data

Explaining Data Changes

Existing Data Systems

µ  Hot Picks: Resources

Data are the foundation of any effective objectives-setting or benchmarking initiative. As shown in the previous section, the collection and analysis of both quantitative and qualitative data are critical for setting health priorities. Once a state identifies the priority health areas and potential indicators, a baseline must be set (may require collecting new data) to determine where the state or community currently is on a given problem or indicator and set the stage for determining where it wants to be by the turn of the next decade (target). Setting targets (determining the desired amount of change over a given time interval) is the next critical step. Finally, monitoring progress toward meeting objectives, through collection and analysis of tracking data, should be done on a scheduled basis. Regular reporting and analysis of progress can help state planning groups and leaders refocus resources where they are needed most.

Action Checklist:
Obtaining Baseline Measures,
Setting Targets, and 
Measuring Progress
(See a complete planning and development checklist.) 

• Consult with state experts on census, age-adjustment, ICD-10, and other data changes
• Set criteria for evaluating existing public and private data sources
• Inventory relevant public and private data sources to measure objectives
• Review progress in achieving state Healthy People 2000 objectives

• Develop targets with appropriate baselines and measures and finalize objectives
• Develop methods for measuring objectives without existing data sources
• Gather and evaluate other data and information to include in state plan
• Plan regular intervals to measure and track achievement of targets

Process in Action: Examples from the Field

Below are examples of how the nation and states addressed data issues.

From the National Initiative

Monitoring data

In 1991, the Health Promotion Statistics Division was established at CDC/National Center for Health Statistics (NCHS) to monitor Healthy People 2000. Staff in this unit coordinate with the HHS lead agencies in collecting and reporting on the national Healthy People objectives. This division produces the Healthy People 2000 Review, available at: http://www.cdc.gov/nchs/products/pubs/pubd/hp2k/hp2k.htm.

Developing new data

Healthy People 2000 spearheaded the development of new data throughout the past decade. In 1991, nearly one-third of the national objectives had no baselines when they were initially set. By 1998, 82 of these 91 objectives had measures. These include areas such as school health, health provider activities, and worksite health.

Promoting continuity between plans

The initial draft of Healthy People 2010 disseminated for public comment continued many of the objectives from Healthy People 2000. In fact, 138 objectives were maintained from Healthy People 2000, while 96 objectives were revised and 297 new objectives were introduced. The continuity between the plans helps to confirm trends and promotes long-term analysis of the same subjects.

Setting targets that are challenging, but necessary

The Healthy People 2010 draft proposed a goal of eliminating health disparities resulting in one target for all population groups to achieve. In fact, for behaviors, risk factors, and services objectives, the target is better than the best population group. For most outcomes, national averages were used with explicit recognition that all groups should improve.

Healthy People 2000 Newsletters: Statistical Notes and Statistics and Surveillance

Two NCHS newsletters address technical issues related to assessing progress toward the year 2000 objectives. Visit: http://www.cdc.gov/nchs/products/pubs/pubd/hp2k/hp2k.htm.

From State Initiatives

Assessing data and data needs in order to set objectives

The Connecticut Department of Public Health responded to the year 2000 national initiative with a coordinated, internal data-oriented review of Healthy People 2000 and development of state objectives. In 1992, the Department of Public Health produced Healthy Connecticut 2000 Baseline Assessment Report, as a framework for program planning, evaluation, policy development, and assurance activities. The report originally contained 112 objectives in 18 priority areas that focused on health status and risk reduction. The Department of Public Health updated the Baseline Assessment Report in 1997 with 42 service and protection objectives. The objectives set targets for the services needed to address the health status and risk reduction objectives.

In the District of Columbia, the State Center for Health Statistics was given the task of working with Program Administrators and staff to produce a comprehensive review of progress from 1993 to 1998 toward meeting Healthy Residents Year 2000 Objectives. In January of 1999, the Progress Review was completed and released. Following the evaluation and documentation of progress, program administrators and staff working with their Advisory Board members, community-based contacts, and collaborating federal agencies developed the draft year 2010 objectives for both internal review and public comment.

In Ohio, as a part of Ohio’s Public Health Plan, the Data System Work Group assisted the Healthy People Ohio (HP Ohio) Work Group by preparing a Data Inventory. The inventory specifies the data source and whether data are available for each HP Ohio objective. The Data System Work Group also identified baseline data for some of the HP Ohio objectives, and made recommendations for data collection for objectives with no data source. The HP Ohio objectives are included in the Ohio Department of Health’s data warehouse.

The Great Lakes Inter-Tribal Council of Wisconsin and the Inter-Tribal Council of Michigan serve Tribes in both states through a Cooperative Agreement Epidemiology Project (The EpiCenter). The EpiCenter developed Tribal-specific community health profiles based on health indicators by making use of Indian Health Service’s Base Line Measures, a needs assessment, and Healthy People 2000. Data in the community health profiles serve as baseline measures and descriptions of changing health status for the Tribes in the project service area.

In South Dakota, data activities begin at the program level with programs following the grant proposal/reporting process for developing baseline measures, setting targets, and determining methods for progress measurement. Many grants, such as the Maternal and Child Health Block Grant, use Healthy People performance measures, grant-specific performance measures, and state-specific performance measures.

In 1995 Minnesota developed objectives to improve its data systems' ability to measure progress toward the year 2000 objectives. Among these objectives, Minnesota sought to collect and disseminate data from state agencies, local agencies, health plan companies, and other health care providers. The state planned to identify significant gaps in disease prevention and health promotion data, as well as establish methods to collect and analyze health status indicators.

Identifying and communicating data sources and data needs specific to the measurement of each objective in the plan

Healthy New Jersey 2000 details state data needs for each goal and corresponding objectives. New Jersey expanded its list of relevant data needs beyond health status objectives. As examples, the plan calls for better patient socioeconomic and clinical outcome data, standardized definitions of certain conditions, evaluation data on prevention interventions, and economic impact data.

In Texas, through a grant from the CDC, the department received staff assistance to develop a series of on-going reports tracking state progress according to the 18 Health Status Indicators recommended by CDC in conjunction with the Healthy People 2000 initiative. The preparation of this series of reports was institutionalized within the department and is continued as an important component of its ongoing assessment of the state’s health status.

The Illinois Project for Local Assessment of Needs (IPLAN) was developed to assist local health departments to complete community health needs assessments. The system utilizes Healthy People 2000 and Healthy People 2010 objectives as reference points, where applicable, and provides over 100 state- and county-level population-based health indicators. For some indicators, community-level data are available. Originally designed as a stand-alone PC-based data system, the current IPLAN system is available through the Internet and can be viewed at: http://163.191.194.35.

Selecting indicators based upon previously identified performance measures or benchmarks

The Colorado Statewide Outcomes/Indicators Task Force established a defined set of measures to rate the performance of the Colorado Department of Public Health and Environment (CDPHE). Performance was measured in terms of outcomes (e.g., heart disease death rates), rather than processes (e.g., number of adults who have had their blood pressure checked). Task Force members represented public health agencies, managed care organizations, academia, and philanthropic organizations. Population-based objectives were developed to reflect the Healthy People 2000 national plan and the CDPHE budget requests.

Rhode Island’s Minority Health Information Improvement Project aimed to strengthen the state’s ability to assess and respond to the health needs of its diverse population. The project developed methods to use existing data sources to measure progress toward year 2000 objectives for racial and ethnic minority populations. Through a collaboration between the Rhode Island Health Department and the Minority Health Advisory Committee, the project published a minority health status sourcebook that established baselines and identified data gaps for minority populations.

To provide continuity with earlier statewide health improvement plans, Washington based its primary health indicators on existing "performance measures" in six public health areas. Each indicator has a primary measure (e.g., the mortality rate) followed by other measures of impact and burden (e.g., hospitalization, years of potential life lost). To assist a wide range of audiences engaged in local planning and implementation, Washington compiled for each health area existing data on population risk factors, protective factors, and intervention effectiveness from research and practice.

The Oregon Legislature directed all state agencies to develop performance measures with ties to the state’s indicators of well being, called Oregon Benchmarks. From 1992 through 1997, Oregon used funding from a CDC grant (Assessment Initiative) to compile valid existing data and measure their benchmarks. These results were submitted to the legislature in an annual progress report.

Tracking and communicating progress toward objectives

For its 1996 and 1999 updates to the state’s year 2000 objectives, New Jersey’s statistical and program staff assessed progress and analyzed trends. Based on their trend analysis, staff categorized each objective and sub-objective as "likely to be achieved," "unlikely to be achieved," or "uncertain."

Washington analyzed data from local, county, state, and national sources in its 1996 statewide assessment of health status, health risks, and health systems. The state used a standard format to present data on its progress in each priority area, including analyses of time trends, geographical variation (including numerous objectives tracked at the county level), variation by age, gender, race, ethnicity, income, and education (where available).

California created individual county health status profile tables, containing 26 Healthy People health status indicators. Data for the profiles are provided by the state Center for Health Statistics, the Division of Communicable Disease Control, and the Office of AIDS of the Department of Health Services. The Demographic Research Unit and the Census Data Center of the Department of Finance provided the 1990 census data and the 1996 race/ethnic population estimates, by county, with age and sex detail.

In addition, as part of its strategy for addressing data needs, California has the Health Information and Strategic Planning Division (HISP) of the California Department of Health Services (DHS). This division takes the lead in making the DHS health data systems more integrated, accessible, and useful for policy development and program management. It also develops uniform health data systems to promote the collection of information on health status outcomes, provides technical assistance and support to local health agencies, organizes strategic planning and special initiatives in support of DHS priorities, and builds strong relationships with public health organizations and schools of public health.

In 1992 the Michigan Department of Community Health (MDCH) established a strategic planning process, which linked state-level health assessment to the identification of priorities, goals, objectives, and strategies to improve health. Healthy Michigan 2000, issued in 1993, provided a guide for improving health by the year 2000. The foundation of the plan was an assessment of not only health status and health system trends, but also the economic, demographic, public perception, and management trends likely to influence the public’s health. In 1996 Healthy Michigan 2000, Second Edition, re-affirmed the goals documented in the first edition and streamlined the objectives to reflect the areas most in need of significant emphasis or change in order to reach the goals.

Between 1992 and 1996, MDCH created an agency-wide Surveillance and Data Strategic Work Group to promote greater use of surveillance data in policy and program decision-making and to determine the data needed to monitor progress toward reaching Healthy Michigan 2000 objectives. The work group drafted a "Health Surveillance Plan" to enhance the capacity for the collection, analysis, interpretation and dissemination of information on health status, health risks, and health systems. The "Health Surveillance Plan" established variables for monitoring objectives contained in Healthy Michigan 2000, identified gaps in data and potential problems, and suggested possibility for new and enhanced data sources. The plan also recommended a set of critical health indicators as a means of communicating the overall health of the state’s population. Based on the "Health Surveillance Plan," the state initiated annual reporting on selected critical health indicators in 1996. Michigan Critical Health Indicators are linked to both key Healthy Michigan 2000 objectives and related interventions.

In Utah the governor’s Office of Planning and Budget coordinates data collection and monitoring of performance measures for all state agencies, as specified in the Utah Tomorrow strategic plan. The governor’s office maintains performance measurement data in their information base. In 1995, with funding from the CDC, the Utah Department of Health, Office of Public Health Data published data to track the 18 Healthy People 2000 health status indicators by local health department district.

On July 1, 1993, the North Dakota Department of Health began to assess the state’s progress toward meeting the year 2000 objectives. A point-in-time study was conducted from July 1, 1993, through March 11, 1994. The report was published in June 1994 and helped the state health department and local communities to identify high priority needs. Some of the findings included: 22 percent of the Healthy People 2000 objectives had been met, 23 percent were unmet, 5 percent were moving away from the HP 2000 target, and 49 percent had no data available.

Alaska completed two Healthy Alaskans 2000 data reports. One report was released in March 1997 which updated the health status objectives for Healthy People 2000 objective 22.1. The second report was released in December 1998 which was a complete update on all Healthy Alaskans 2000 objectives. The December 1998 report also emphasizes that data collection is the first step in public health planning and an interim step in developing a comprehensive review of Healthy Alaskans 2000.

Total Objectives/Sub-Objectives Using "Top Five" Sources = 3,130
(N=36)

Note: Each state was asked to identify its "top five" data sources used for tracking objectives and to identify the number of objectives/sub-objectives tracked by those five data sources. Data sources not appearing in a state's "top five" list were captured in the "other" category.

Source: Public Health Foundation. Measuring Health Objectives and Indicators: 1997 State and Local Capacity Survey. March 1998.

Extent that Availability of Baseline Data Influenced the
Selection of States' Year 2000 Objectives/Sub-Objectives

Source: Public Health Foundation. Measuring Health Objectives and Indicators: 1997 State and Local Capacity Survey. March 1998.

Source: Public Health Foundation. Measuring Health Objectives and Indicators: 1997 State and Local Capacity Survey. March 1998.

Potential Health Measures

The following is intended to assist you in identifying different types of measures for your state's health plan. It is not meant to be an exhaustive list, but provides types of measures many communities have found beneficial in developing and monitoring health objectives.

COMMUNITY MANAGEMENT
Interagency networks
Open city council meetings
Planning - economic development, social planning  
    council
Policy environment
Readiness - fire escape plans, CPR training, 
    retirement preparation
Representation in community groups
Responsiveness - emergencies
Volunteerism level
Voter turnout

DEMOGRAPHICS
Age distribution
Education levels
Median income
Occupations
Population stability
Poverty levels
Unemployment rates

GROWTH AND NUTRITION
Breastfeeding prevalence
Developmentally delayed children
Fruit and vegetable consumption
Disability prevalence
Enrollment in entitlement programs
Elders who participate in fitness programs
Life expectancy
Self-reported health status
Women, Infants and Children (WIC)

HEALTH BEHAVIORS
Exercise levels
Overweight prevalence
Tobacco use
Alcohol use/abuse prevalence
Substance abuse treatment need

HEALTH CARE RESOURCES
Insurance status
Medicaid/Medicare providers
Managed care penetration

HEALTH CARE UTILIZATION
Hospital use rate
Preventable hospitalizations rate

HEALTH OF EMPLOYEES
Sick days used
Workmen’s compensation claims
Worksite injuries and deaths

HEALTH OF MOTHERS AND CHILDREN
Contraceptive services and need
Low birth weight babies percent
Prematurity prevalence
Prenatal care percent
Teen parenting prevalence

MORBIDITY
Dental caries among children
Communicable diseases rates
Vaccine preventable disease/deaths
Mental illness prevalence

MORTALITY
Infant mortality - neonatal, post-neonatal
Major killers - CHD, cancer, stroke, homicide,    suicide, motor vehicle injuries, unintentional
   injuries, diabetes, COPD, AIDS
Overall and age-level

PHYSICAL ENVIRONMENT
Environmental conditions - air, water, recreational 
     water site quality
Environmental hazards
Epidemics
Household smoke detectors
Households on water and sewage treatment systems,
    septic systems
Household fuel efficiency
Household recycling
Industrial waste recycling
Lead paint housing vulnerability, soil
Natural disasters
Nuisance index - noise, dirt, odors

PREVENTIVE MEASURES
Blood pressure checks
Childhood immunization rates
Cholesterol checks
Colon cancer screening prevalence
Diabetic eye and foot exams
Flu vaccine use among the elderly
Mammography screening prevalence
Pap test prevalence

SOCIAL SUPPORT MEASURES
Bike path mileage
Recreation center use
Child abuse investigations
Domestic violence services
Family and friend support networks
Religious center use
Law enforcement
Neighborhood Watch Programs
Self help group participation
Suicide prevention services
Transportation services

Setting Targets for Objectives

One of the central issues many states struggle with when developing objectives is how to set achievable, realistic targets for outcome, performance, and process objectives. The guidance below focuses primarily on setting targets for health outcomes and performance.

• Using an absolute percent decline

Some Healthy People 2000 objectives used an absolute percent decline based on "best guesses"/expert opinion. Calculations can be made based on the percent of the target population reached and change expected. For example, a decline in mortality of 30 percent expected in two-thirds of the women with breast cancer.

[Start Amount x (1 – .30) x 2/3] + [Start Amount x 1/3] = End Amount

Example: Breast cancer rate of 33/100,000

[33 x (1 - .30) x 2/3] + [33 x 1/3] = 15.4 + 11 = 26.4/100,000

• Using peer communities

You can set targets by comparing your community to others like it. Age and poverty distribution and population size and diversity may define peer communities. The following may be used to describe one’s peers: typical values for a specific objective, means or medians, or the variation among peers. Visit the Public Health Foundation web site for more information on the Community Health Status Indicators Project, which is utilizing this strategy:

• Using the pared-mean method to set data driven benchmarks

The pared-mean method determines "top performance." This is defined as the best outcome accomplished for at least 10 percent of the population.

Steps to Compute the Pared-Mean (The article cited below uses an example of mammography screenings)

1. Rank order providers or other units of analysis (e.g., health departments, jurisdictions) in descending order of adherence. In this example, metropolitan statistical areas were ranked according to average mammography rates.
2. Order providers in descending sequence until you have a subset that equals or exceeds 10 percent of all patients in the survey. In this example it was 10 percent of women over the age of 50 in the survey.
3. Calculate the benchmark based on the subset of units analyzed, dividing the total number of patients in the subset receiving the recommended intervention (e.g., mammography screenings by the population).

In the example of the mammography screenings, a benchmark rate of 71 percent was found, exceeding the Healthy People 2000 target of 60 percent.

Data sources to use for the pared-mean method include vital statistics and the Behavioral Risk Factor Surveillance System.

This method is not feasible for all Healthy People objectives. Data may not be available for some objectives, or the nature of the objective may not lend itself to using the pared-mean method. For example, access to preventive care should be available for 100 percent of the population, regardless of what the data show.

Source: Allison J., Kiefe C.I., Weissman N.W. "Can Data-Driven Benchmarks be Used to Set the Goals of Healthy People 2010?" American Journal of Public Health, 89(1):61-5, 1999.

• What if areas in the state have already achieved or surpassed the national Healthy People target for an objective?

You can calculate a new, higher state target that will be challenging for local areas that have achieved or surpassed the national target. You also may wish to note in your plan the jurisdictions that have not achieved your previous targets and redouble your efforts in these areas as well as set equally ambitious targets for year 2010.

• Setting targets for process objectives

Many process objectives, particularly those that pertain to infrastructure (data systems, workforce, and research), are new this year in Healthy People 2010. These should be examined carefully by states to determine their applicability to the state plan. Setting measurable targets for process objectives requires judgment and is not an exact science. To set process targets, planners should consider the current status (baseline) of the state's public health infrastructure, seek stakeholder input on the desired level of improvement, and make a realistic assessment of what can be accomplished given the state's experience, resources, political opportunities, and partner commitment.

• Using performance measures

"Performance measurement responds to the need to ensure efficient and effective use of resources, particularly financial resources. It links the use of resources with health improvements and the accountability of individual partners."  (Prevention Report, Winter 1997) This is of particular importance since the inception of the Government Performance and Results Act of 1993, which aims at holding Federal agencies accountable for spending public dollars. This extends to states, local jurisdictions, and other organizations that receive Federal funding. Performance measures can be incorporated into or based upon Healthy People objectives. Please see the following pages for a detailed description of setting performance measures.

---

Setting Performance Measures Step by Step

These examples are based on the State of Maryland’s Healthy Maryland 2000 document

Source: U.S. Department of Health and Human Services. "Improving the Nation’s Health with Performance Measurement." Prevention Report, 12(1):1-5, 1997. http://odphp.osophs.dhhs.gov/pubs/prevrpt/97winfoc.HTM.

Measuring Progress

Annual Percent Change

This measure can be used to track whether progress is on course and determine if the 2010 objectives will be reached. It provides the amount of decline each year that is needed to reach the target.

Formula:

{(Target rate ¸ Baseline rate) ^{[1/ (Target year)]} - 1} x 100 = Annual Percent Change

Calculations:

(Target rate ¸Baseline rate) = 7/1,000 ¸ 10/1,000 = 0.700
^{[1/ (Target year – Baseline year)]} =1/ (2010 – 2000) = 1/10 = 0.100
(Target rate ¸ Baseline rate) ^{[1/ (Target year – Baseline year)]} = 0.70 ^0.10 = 0.965
{(Target rate ¸ Baseline rate) ^{[1/ (Target year – Baseline year)]} – 1} = 0.965 – 1 = -0.035

{(Target rate ¸ Baseline rate) ^{[1/ (Target year – Baseline year)]} – 1} x 100 = 0.035 x100 = -3.5%

A decline of 3.5% per year between year 2000 and 2010 is needed to reach the target.

Measuring Progress

This equation is used in measuring progress for each objective, adapted from Healthy People 2000 Midcourse Review and 1995 Revisions:

(Current Status – Baseline) (Year 2000 Target – Baseline)

x 100

= Percentage of Target Achieved

Note: You will get a negative percentage when the baseline has gotten worse.

Evaluating Data:
Data Issues and Uses

What are some general data issues that you may want to address?

• Data Quality – When using new data collection systems, be sure to check for standardization of data collection and recording, data management and analysis, and structure and content of questions.
• Limitations of Self-Reported Data – When relying on self-reported data such as income level, exercise frequency, or health screening behaviors, be aware of self-reporting bias. Measures will vary based on the type of data collection alone (written survey, telephone interview, direct observation, etc.).
• Data Validity and Reliability – Revision of survey questions and the development of new data collection systems will require careful validity and reliability testing. In monitoring efforts, the validity of responses over time may also become an issue.
• Periodicity of Data Availability – Data collection efforts are not always performed on a regular basis. Take this into consideration when planning your dissemination and communication efforts.
• Timeliness of Data Availability – As previously stated, this is not always possible, but still important. It helps to be able to regularly identify progress and areas that may need additional efforts.
• Representativeness of Data – Special considerations need to be made when collecting data for specific population groups or local communities. Do responses collected represent those individuals of interest?
• Small-Area Analysis – Take into account the limitations of applying national data to the state, local and community levels. This pertains to using small numbers in one’s statistics. Poisson distribution, non-parametric statistics, and standardized mortality rates/ratios (SMRs) may be appropriate methodologies.

Source: Committee on Leading Health Indicators for Healthy People 2010. Leading Health Indicators for Healthy People 2010: Final Report. Division of Health Promotion and Disease Prevention, Institute of Medicine, 1999.

Evaluate your existing data collection methods using these guidelines:

Source: Klaucke D.N., Buehler J.W., Thacker S.B., Parrish R.G., Trowbridge F.L., Berkelman R.L., and the Surveillance Coordination Group. "Guidelines for Evaluating Surveillance Systems." Morbidity and Mortality Weekly Report. 37(S-5):1-18, 1988.

Characteristics of High-Quality and Effective Data for Policy Making

Source: Feldman P., Gold M., Chu K. "Enhancing Information for State Health Policy." Health Affairs, 13(3):238, 1994.

Explaining Data Changes

Age-Adjusting to Year 2000:
State and Local Age-Adjusted Rates Will Increase

Explanation of Age-Adjustment: Age-standardization is a practice of adjusting for differing age composition of populations. Age-standardization is useful when comparing disease outcomes across time, place, or populations. Prior to 1998, the conventional standard population used for adjustment was the U.S.1940 population. As of 1998, the National Center for Health Statistics (NCHS) will use and has recommended that others use the year 2000 standard. [The year 2000 standard resembles the current population structure and for many geographic areas is close to the crude (unadjusted) disease rate.] Healthy People 2010 uses the year 2000 adjusted rate for baselines and target rates.

Impact of Changes: For most disease categories, especially where disease rates increase with increasing age, year 2000 adjusted rates will increase substantially. Diseases that occur among young people, such as homicide, will decline while others which affect the age extremes will stay the same using the new age standard. Users of the year 2000 standardized rates will not be able to readily compare them to prior years’ statistics that were calculated using the 1940 standard.

Existing Data Systems

Data are the foundation of Healthy People objectives. While the national Healthy People initiative has stimulated the development of new data systems, there are still areas where information is missing. These areas constitute the developmental objectives, where efforts are being made over the next decade to measure these indicators. In the meantime, how do you manage the data presently available? There are approximately 200 data sources used to track the national Healthy People objectives.

Source: Leading Indicators for Healthy People 2010. A Report from the HHS Working Group on Sentinel Objectives. U.S. Department of Health and Human Services, March 1998. http://odphp.osophs.dhhs.gov/pubs/LeadingIndicators/ldgindtoc.html

µ ICD-10 – The following sites provide information on the ICD-10.

http://www.healthmkt.com
http://www.cdc.gov/nchswww/about/otheract/icd9/abticd10.htm

µ National Association of Health Data Organizations. http://www.nahdo.org