Program Evaluation Design – The WISEWOMAN Public Health Program in Pueblo,Colorado

2. Introduction

2.1. Summary of the Program
The WISEWOMAN Public Health Program is a CDC-funded program that assists
women between the ages of 40 and 64 to reduce their risk of heart disease and stroke (The
Centers for Disease Control and Prevention [CDC, 2022 a). WISEWOMAN is a mnemonic for
Well-Integrated, Screening and Evaluation for Women Across the Nation. The initiative is
funded and supported by the CDC and is implemented in all 50 states and tribal organizations
participating in the NBCCEDP (National Breast and Cervical Cancer Early Detection Program).
Since its inception, the program has served more than 1.3 million women and has been shown to
reduce health disparities by providing access to care for underserved populations. The
WISEWOMAN is a comprehensive and evidence-based approach to improving heart disease and
stroke prevention among low-income women.
This project evaluation specifically focuses on Pueblo’s WISEWOMAN program. In
Pueblo, Colorado, the WISEWOMAN program is specifically designed for low-income,
uninsured, and under-insured women enrolled in the Colorado Women’s Health Check program,
which provides heart disease and stroke prevention services. The program offers various
services, including health coaching, group education sessions, blood pressure, and cholesterol
screenings, and referrals to lifestyle programs and community resources (CDC, 2022a). The
screenings are free and conducted by trained health professionals at community health clinics,
community centers, and other outreach sites. Furthermore, women with high-risk factors for
heart disease and stroke are referred to appropriate health services, such as primary care
physicians, nutritionists, and other specialists. The program works with local healthcare
providers to ensure women receive timely and appropriate care.

4

2.2. The Public Health Issue Addressed by the WISEWOMAN Program
Like other regions, the WISEWOMAN Program in Colorado addresses the risk of stroke
and heart disease by providing free screenings for critical markers like glucose, cholesterol, and
blood pressure. Pueblo is a predominantly Hispanic/Latino low-income community with a higher
prevalence of heart disease and stroke than the state average (US Census Bureau, 2022). The
program also provides lifestyle interventions for treating and preventing stroke and heart disease,
including nutrition and physical activity education, plus provides follow-up care and referral
medical services for patients.
Heart disease is a general term that refers to multiple conditions that affect the heart. The
commonest category of heart disease is CAD (cardiovascular artery disease), which affects blood
flow to the heart. Heart attack, arrhythmia, and heart failure are adverse events associated with
CAD. Heart disease risk factors include excessive alcohol use, physical inactivity, unhealthy
diets, obesity and overweight, diabetes, genetics, and other social determinants of health
(SDOH), such as household income and education or literacy levels.
On the other hand, stroke or “brain attack” occurs when the blood supply to the brain is
blocked when a blood vessel ruptures in the brain, causing parts of the brain to die and,
ultimately, leading to long-term disability damage or death. Heart disease and stroke are
prevalent in the US and Colorado and are the leading causes of death. Approximately 697,000
and 137,000 Americans die from heart disease and stroke annually (CDC, 2022b). About 7,060
and 1,988 people die in Colorado annually from heart disease and stroke (CDC, 2018).
2.3. The Program’s Stage of Development
Colorado’s WISEWOMAN program is currently in the “monitoring and evaluation
phase.” This means that the program’s developers are presently tracking and assessing the

5
program’s success and progress, measuring the impact and outcomes, and collecting feedback to
make informed decisions about potential modifications and improvements. Colorado’s
WISEWOMAN initiative implemented two unique, innovative interventions during the
implementation phase. For the first strategy or intervention, Colorado’s Department of Health
partnered with multiple organizations to perform outreach to women in Pueblo and other
counties and connect them to WISEWOMEN services. Implementation started in 2020 at the
start of the COVID-19 public health emergency. Specifically, the state partnered with the Ute
Mountain Ute Tribe and the Somali American Community Center) to increase their outreach
initiatives headed by CHWs (community health workers) to include WISEWOMEN services. In
particular, Colorado provided outreach funding, virtual meetings, and technical support webinars
to the two partners, focusing on supporting cardiovascular health educational activities hosted by
CHWs (CDC, 2020c).
For the second intervention, Colorado engineered and supported bidirectional referrals
between healthcare systems and community programs by assisting WISEWOMAN clinics to roll
out the CRISeR (Community Resource Inventory Service for Patient e-Referral) system and
making contributions toward the state’s inventory of referral resources. The state’s department of
health curates and uploads cardiovascular health-linked data to the state’s referral database (2-1-
1), which is principally funded via the CDC’s 1817 grant program and operated by a faction of
not-for-profit organizations. This database is available to the public and healthcare systems,
including WISEWOMAN medical provider facilities and sites (Colorado Department of Public
Health & Environment, n.d.).
2.4. Community Stakeholders

6
Community stakeholders for the WISEWOMAN program evaluation may include
women enrolled in the program, community-based organizations, healthcare providers, local
government officials, local health departments, and community leaders. These stakeholders can
provide valuable insights into the program’s impact, implementation, and sustainability. The
stakeholders will be crucial in the support and effectiveness of this program

3. Evaluation Design

This program evaluation will use a mixed method to evaluate the outcomes and the
implementation/processes. A mixed method means quantitative and qualitative techniques will
be used to assess whether or not the program has met the set objectives/goals, including
effectiveness and areas of improvement. Effectiveness means reviewing the extent to which the
WISEWOMAN initiative has improved health behaviors and lowered the risk factors associated
with cardiovascular disease among low-income, underinsured, or uninsured women in Pueblo,
Colorado (Prosek, 2020). The evaluation will help identify areas of the program that need
improvement or modification to meet the needs of the target population better. Quantitative
methods will specifically assist in identifying whether the program has been effective.
Qualitative methods will be used to identify potential areas of improvement.

4. Evaluation Questions

i. To whom, what, when, and where did the Colorado WISEWOMAN program
activities occur?
ii. What were the facilitators and barriers to implementation
iii. Did implementing the WISEWOMAN program improve the health behaviors of low-
income, underinsured, or uninsured women in Pueblo, Colorado?

7

iv. Did the WISEWOMAN program reduce the risk factors associated with
cardiovascular disease among low-income, underinsured, or uninsured women in
Pueblo, Colorado?
4.1. Program Elements that each Question will examine
For the mixed methods, the first two questions apply to the qualitative aspect of the
evaluation plan. They assess whether program activities and strategies were implemented as
desired or intended and identify the potential speedbumps and barriers that derail or impact
implementation. Question one tracks the program’s information about where, when, what, and
who questions. Question two identifies potential barriers and speedbumps to implementation. On
the other hand, questions three and four apply to the quantitative aspect of the program
evaluation. They quantitatively measure whether the WISEWOMAN program has achieved the
outcome effectiveness it was designed to do, including improving health behaviors and reducing
the risk factors associated with cardiovascular disease among low-income, underinsured, or
uninsured women in Pueblo, Colorado.
4.2. How the Questions Address the CDC Standards of Utility, Feasibility, Propriety,
and Accuracy
The four questions presented above can be assessed based on the CDC’s standards of
utility (who needs the information and how will they utilize it), feasibility (how much effort,
skill, time, and money can be pumped into the evaluation), propriety (who needs to be engaged
in the assessment for it to be ethical), and accuracy (what design would lead to accurate data) as
follows (Program Performance and Evaluation Office, 2018). Based on “utility,” Question one
details the effectiveness of the program’s implementation and can assist stakeholders in
understanding what services were provided, who the program served, and where and when it was

8
implemented. Question two focuses on why understanding the barriers and facilitators that
affected the program’s implementation. For future project implementation, this can assist in
replicating the success stories and addressing barriers. Questions three and four underscore the
WISEWOMAN program’s role in improving women’s heart disease and stroke health outcomes
in Pueblo; this can assist stakeholders in understanding the program’s effect on women’s health in
the community.
Question one does not relate to feasibility. However, Question two is relevant to the
program’s implementation feasibility in that knowledge of the barriers and facilitators can assist
program implementors and stakeholders in estimating staff, funds, partnerships, and other
resources for the successful implementation of the WISEWOMAN program. Questions three and
four are also relevant to the program’s feasibility because knowledge of the program’s health
outcomes can assist stakeholders in gauging the program’s success or impact and the resources
needed to keep it going.
Regarding “propriety,” all four questions raise the ethical concern about informed
consent. Before enrolling, did the project implementers inform the participants about the
benefits, procedures, risks, and their rights? Did they consent willingly? Questions one, three,
and four also raise ethical issues related to justice. Did the interventions or activities undertaken
consider equity in resource distribution? Did every participant receive equal services
notwithstanding their race, sexual orientation, nationality, economic status, or age?
Finally, all the Questions are relevant to “data accuracy” in multiple ways. For example,
for questions one and two, understanding the precise details of the program planning, design, and
implementation, including barriers and facilitators, can guarantee correct information gathering
and reporting. For questions three and four, establishing the program’s efficiency and

9
effectiveness needs reliable and accurate information on pre-and post-implementation screening
rates and health outcomes.

5. Ensure the Use of the Findings

The evaluation approach (mixed method) selected will ensure the use of evidence and
findings by incorporating both quantitative and qualitative methods, from planning to
implementation to evaluation. For example, quantitative methods (statistical analyses,
standardized tests, and surveys) will assist in quantifying program outcomes and measuring its
effectiveness, including whether it has improved health behaviors and lowered the risk factors
associated with cardiovascular disease among low-income, underinsured, or uninsured women in
Pueblo, Colorado. On the other hand, qualitative methods – collecting and analyzing
nonnumerical data gathered from interviews, focus groups, and observations – can provide
insights into the program outcomes and identify barriers to implementation and areas of
improvement. These barriers can include financial constraints, cultural and language barriers,
and resistance from the target population (Hamilton & Finley, 2019).
One takeaway from the WISEWOMAN program is the importance of designing
community-based, culturally-sensitive interventions to address issues affecting underserved
populations. The WISEWOMAN program leverages community-based approaches to reach out
to low-income, underinsured, or uninsured Latino women in Pueblo, Colorado, at the greatest
risk of heart disease and stroke. Evidence has shown that this approach is promising to improve
healthcare outcomes. The program has also revealed the important of community partnerships
and collaborations in the success of programs targeting local populations.

10

References
CDC. (2018). Stats of the state of Colorado.
https://www.cdc.gov/nchs/pressroom/states/colorado/colorado.htm
CDC. (2022a). Wisewoman Overview. https://www.cdc.gov/wisewoman/about.htm
CDC. (2022b). About heart disease. https://www.cdc.gov/heartdisease/about.htm
CDC. (2022c). WISEWOMAN innovation spotlight. Colorado Department of Public Health and
Environment.
https://www.cdc.gov/dhdsp/evaluation_resources/wisewoman_innovation/colorado.htm
Colorado Department of Public Health & Environment. (n.d.). WISEWOMAN.
https://cdphe.colorado.gov/wisewoman
Hamilton, A. B., & Finley, E. P. (2019). Qualitative methods in implementation research: An
introduction. Psychiatry Research, 280. doi: 10.1016/j.psychres.2019.112516
Program Performance and Evaluation Office. (2018). Program evaluation framework checklist
for step 3. CDC. https://www.cdc.gov/evaluation/steps/step3/index.htm
Prosek, E. A. (2020). An introduction to mixed-methods design in program evaluation.
https://doi.org/10.1080/21501378.2019.1590688 Counseling Outcome Research and
Evaluation, 11(2), 60-70.
US Census Bureau. (2022). QuickFacts: Pueblo city, Colorado.
https://www.census.gov/quickfacts/fact/table/pueblocitycolorado#