Impact & Evidence
We measure the work, study the results, and use what we learn.
Across multiple years of federally supported work, Life that Counts has built, delivered, adapted, and evaluated programming with schools. The record includes positive participant experience, encouraging pre/post findings, results that were null or unfavorable, and practical lessons that now shape stronger program and evaluation practice.
This page presents findings from two external evaluation sources. Each study is reported with its population, period, measure, denominator, method, and limitation kept together. Historical findings demonstrate experience, participant response, and lessons for design. They are not carried forward as automatic evidence for a different or current program.
Middle-school programming across Alabama and Montana
WPRE | Final report compiled September 2022 | Descriptive evaluation
Washington Park Research and Evaluation compiled a final report on three levels of historical programming delivered primarily to middle-school students in Alabama and Montana. The report combined completed survey records with staff interviews about implementation and contextual conditions.
Washington Park Research and Evaluation compiled the historical report summarized in this section. The findings and recommendations are attributed to WPRE as the report’s evaluator.
Survey records—not a participant-retention count
The report lists 2,541 entry surveys and 2,204 exit surveys across the three levels. Dividing those totals produces an aggregate exit-to-entry survey-record ratio of 86.7%. Because entry and exit records were not linked to the same individuals, that ratio is not a participant completion rate, retention rate, or longitudinal follow-up rate.
Historical level | Survey window | Entry surveys | Exit surveys — Level 1 | September 3, 2020–May 19, 2022 | 1,910 | 1,681 — Level 2 | January 25, 2021–April 13, 2022 | 526 | 414 — Level 3 | May 17, 2021–February 7, 2022 | 105 | 109
Table note: These are completed survey records, not verified unique participants or matched entry/exit pairs. Level 3 recorded more exit surveys than entry surveys; the report does not identify the precise source of that difference.
What respondents said about the program experience
Across the three levels, most respondents said the material was clear, activities helped them learn, and they felt respected all or most of the time. The graph reports each level separately and uses percentages calculated from the exact category counts in the report.
Participant experience — Level 1
Percent selecting “all of the time” or “most of the time.”
Interested in sessions — 70.8% (1,113 of 1,573)
Material was clear — 85.3% (1,341 of 1,572)
Activities helped learning — 82.9% (1,301 of 1,569)
Had a chance to ask questions — 80.5% (1,260 of 1,566)
Felt respected — 83.5% (1,305 of 1,562)
Participant experience — Level 2
Percent selecting “all of the time” or “most of the time.”
Interested in sessions — 76.2% (301 of 395)
Material was clear — 88.0% (346 of 393)
Activities helped learning — 86.5% (341 of 394)
Had a chance to ask questions — 85.2% (333 of 391)
Felt respected — 88.0% (346 of 393)
Participant experience — Level 3
Percent selecting “all of the time” or “most of the time.”
Interested in sessions — 74.8% (80 of 107)
Material was clear — 86.9% (93 of 107)
Activities helped learning — 87.7% (93 of 106)
Had a chance to ask questions — 85.0% (91 of 107)
Felt respected — 85.0% (91 of 107)
Respondent experience across three historical curriculum levels. Percentages are calculated from the report’s category counts; item denominators vary.
What respondents said participation made them more likely to do
The report also asked post-program respondents whether participation made them more likely to resist peer pressure, manage emotions, plan toward goals, and understand healthy relationships. These were retrospective self-assessments collected after programming. They were not observed behaviors or matched changes from baseline.
Post-program retrospective self-assessment — Level 1
Percent selecting “much more likely” or “somewhat more likely.”
Resist or say no to peer pressure — 72.2% (1,142 of 1,581)
Manage emotions in healthy ways — 68.1% (1,074 of 1,578)
Make plans to reach goals — 79.8% (1,259 of 1,578)
Better understand healthy relationships — 81.7% (1,326 of 1,623)
Post-program retrospective self-assessment — Level 2
Percent selecting “much more likely” or “somewhat more likely.”
Resist or say no to peer pressure — 75.4% (230 of 305)
Manage emotions in healthy ways — 71.5% (218 of 305)
Make plans to reach goals — 86.8% (264 of 304)
Better understand healthy relationships — 84.5% (257 of 304)
Post-program retrospective self-assessment — Level 3
Percent selecting “much more likely” or “somewhat more likely.”
Resist or say no to peer pressure — 78.5% (84 of 107)
Manage emotions in healthy ways — 73.8% (79 of 107)
Make plans to reach goals — 73.6% (78 of 106)
Better understand healthy relationships — 79.2% (84 of 106)
Source: Washington Park Research and Evaluation, final three-year report, September 2022. Historical programming in Alabama and Montana. Percentages were calculated from reported category counts. Item denominators vary. Entry and exit surveys were not linked at the individual level. The four retrospective items were measured only after programming; 96 early Level 2 exit respondents were not asked them. No comparison group, confidence intervals, effect sizes, or statistical-significance tests were reported for these items.
What WPRE recommended
The evaluation documented schedule interruptions, differences in facilitation arrangements, and discrepancies between classroom counts and submitted surveys. WPRE recommended stronger roster reconciliation, make-up-survey documentation, cohort assignment, session-exposure tracking, and entry/exit matching. The report also recommended additional data collection for the two smaller curriculum levels.
What this evidence supports
The WPRE report supports statements about historical implementation, participant experience, perceived likelihood at post-program, and organizational learning. It does not establish individual change, long-term behavior change, or causal effectiveness.
High-school peer-mentoring evidence
ITMESA | Reporting materials assembled September 2021 | Pre/post and post-program evidence
Life that Counts participated in a multi-state high-school mentoring initiative examined through participant entry and exit surveys, post-program feedback, and open-ended responses. Life that Counts led implementation in Alabama. ITMESA reporting materials assembled in September 2021 summarize evidence from 2019 and 2020–2021; those materials do not provide Year 2 outcome data.
The materials do not document an untreated comparison group, an assignment process, or a causal counterfactual. They should therefore be described as historical participant-survey evidence—not as a controlled impact evaluation.
2019 Alabama findings
The 2019 Alabama records include 434 pre-surveys and 343 post-surveys, producing a 79.0% exit-to-entry survey-record ratio. The report describes multilevel repeated-measures regression adjusted for gender, race, Hispanic ethnicity, age, mentor/mentee role, and state. It does not report the exact analytic sample for each outcome, confidence intervals, standardized effect sizes, complete missing-data procedures, or a no-program comparison. The reported Alabama model estimates were favorable for Abstinence Risk Avoidance Attitude, Risk Reduction Knowledge, and Risk Reduction Skill. The estimate for Abstinence: Risk Avoidance Behavior Intention was not statistically significant. These are point estimates on the study’s 0–100 scales, not percentages of participants who improved.
Reported 2019 Alabama adjusted difference estimates
Risk-reduction knowledge — +10.9 points — p=.0001
Risk-avoidance attitude — +6.6 points — p=.0003
Risk-reduction skill — +4.8 points — p=.0002
Risk-avoidance behavioral intention — +3.2 points — Not statistically significant
Alabama and Indiana: concurrent peer-mentoring results
ITMESA reported the same four principal constructs for concurrent implementations in Alabama and Indiana. See the construct-by-construct comparison, exact estimates and p-values, study context, and public evidence brief.
Explore the Alabama–Indiana comparison
Source-reported, covariate-adjusted pre/post estimates for the 2019 Alabama survey records. These are model estimates, not percentages of participants who improved.
Source provenance: Life that Counts identifies the supplied September 2021 materials as an ITMESA analysis of 2019 high-school participant surveys. Alabama records include 434 pre-surveys and 343 post-surveys. The source describes multilevel repeated-measures regression with participant and state covariates. It does not report the exact outcome-specific analytic sample, confidence intervals, a contemporaneous comparison group, or complete missing-data procedures. Findings describe adjusted pre/post associations and do not establish causation.
What participants reported about the experience in 2020–2021
In the 2020–2021 post-program file, 79.8% reported that the material was clear all or most of the time, 73.2% said discussions or activities helped them learn all or most of the time, and 84.5% said they felt respected all or most of the time. The same file reports that 63.9% felt interested all or most of the time and 69.4% had opportunities to ask questions all or most of the time. The source lists 247 post surveys across both states but does not provide the item-specific denominator for each percentage. These participant-experience findings are displayed without assigning 247 as the denominator for every item.
2020–2021 participant experience
Interested in sessions — All 25.5% | Most 38.4% | Some 29.2% | None 6.8%
Material was clear — All 42.7% | Most 37.1% | Some 18.0% | None 2.1%
Activities helped learning — All 34.5% | Most 38.7% | Some 21.1% | None 5.7%
Had a chance to ask questions — All 46.6% | Most 22.8% | Some 22.3% | None 8.3%
Felt respected — All 57.7% | Most 26.8% | Some 12.4% | None 3.1%
Post-program participant experience in the 2020–2021 combined survey file. The file reports 247 post surveys overall; item-specific denominators are not supplied.
Results were not uniform across years
The 2020–2021 pre/post analysis used unmatched groups and reported mixed, predominantly nonsignificant results. Alabama’s Positive Future Attitude estimate was unfavorable at −4.6 points (p=.033). In the combined two-state analysis, the report also listed unfavorable changes in a risk-reduction behavior-intention measure and the school-success score. Delivery conditions, participating cohorts, and learning modes changed during this period. These differences do not prove why results changed, but they demonstrate why mode, implementation consistency, session exposure, baseline context, measurement, and participant linkage must be documented. Life that Counts does not treat the 2019 findings as guaranteed results for later cohorts or current programs.
Participant voice
Open-ended responses discussed school effort and future goals, confidence and self-respect, kindness toward others, relationship decisions, emotional regulation, and healthier choices. Some respondents reported no change or uncertainty. The response file does not provide a formal coding protocol, unique-respondent denominator, or publication review. These themes are presented only as illustrative context; no participant quotations, word cloud, or percentages are published.
What this evidence supports
The ITMESA materials support statements about historical implementation, participant experience, and reported pre/post estimates within the surveyed cohorts. They do not establish long-term behavior change, a program-versus-no-program difference, or effectiveness of a current Life that Counts program.
Positive findings matter. So do null findings and limitations.
Taken together—but never statistically combined—the historical evaluations show that many participants described clear instruction, respectful treatment, useful activities, and greater readiness to make thoughtful decisions. They also show that results were not uniform across measures, cohorts, years, or delivery conditions. That full record strengthened five priorities:
Keep participant records linkable and private. Use privacy-protected identifiers that allow approved evaluators to connect measurement points without exposing identities to program staff.
Document what was actually delivered. Record the planned sequence, sessions offered, sessions attended, make-up work, delivery mode, adaptations, and major disruptions.
Separate implementation, experience, and impact. A well-received program may still require a controlled study before effectiveness can be claimed.
Protect evaluator independence. Research instruments, identifiable outcome data, analysis, and scientific reporting should remain under independent evaluator authority.
Report the full result. Favorable, null, and unfavorable findings should be interpreted with the same attention to method, context, and limitation.
