How does dosage affect school readiness?
For this use case, Utah Community Action (UCA) wanted to measure the impact of their program on children’s school readiness and to understand how dosage impacts school readiness. In order to measure dosage, they compared children in half-day classes and children in full-day classes. They also compared children in groups based on how long they had been in the program to understand the differentiated impact by length of time.
In comparing these groups, UCA was able to document that children in full-day classes scored higher on the school readiness measure than those in half-day classes, and that the longer a child was in the program, the higher they scored on school readiness.
ANALYSIS
UCA used their end-of-year foundational academic skills assessment and created an overall score computed by summing each subscale of the assessment.
The data was split two different ways. First, by children in full-day and part-day programming, and then based on the length of time children had been enrolled in the program. Children who were enrolled at the beginning of the program year (September) and were still enrolled at the end of the year were in the 9+ month group. A child enrolled in October through the end of academic year (May) was grouped in the 8 month group (etc.)
UCA created a bar graph showing this summed score for each of the different subgroups, to understand how both time-in-program and dosage affect children’s development.
RESULT
UCA was able to document that children in full-day classes scored higher on the school readiness measure than those in half-day classes, and that the longer a child was in the program, the higher they scored on school readiness.
ADDITIONAL DETAILS
AUDIENCE: Management, leadership, staff, parents, board/policy council
LEVEL OF ANALYSIS: Program, sub-groups
DIFFICULTY: Moderate
CONTENT AREA: School readiness
DATA SOURCES: Assessment tool, Excel
CONSIDERATIONS AND CAVEATS: As a follow-up analysis, UCA also conducted a predictive analysis (regression) to estimate the expected increase in end-of-year assessment scores were expected for each month a child is enrolled in the program (each month predicts an 8.4-point increase for single-session and a 9 point increase in scores for full-day classroom children.
There was a slight age difference in children in single-session and dosage delivery options with children in dosage classrooms being slightly older (4.25 compared to 4.12) than children in single session classrooms. Older children tend to score higher than younger children. Statistical analyses controlling for age differences still showed a delivery model effect with children in full-day classrooms showing a larger program effect on foundational academic skills assessment scores.
TECHNICAL APPENDIX
To conduct the analysis, UCA created comparison groups using the program entry date given in the ChildPlus Participant Snapshot report, grouping children by the date they started in the program. Because they used the end-of-year assessment as the outcome variable, only children who were enrolled in the program at the end of the year were included in this analysis.
Children who were enrolled at the beginning of the program year (September) and were still enrolled at the end of the year or were in the year-round classrooms for at least 9 months were in the 9+ month group. A child enrolled in October through the end of academic year (May) was grouped in the 8 month group (etc.) To simplify the analysis and to increase group sizes for statistical analysis, they combined groups to make four groups for comparison (9+ months, 7-8 months, 4-6 months, 1-3 months.)
This analysis was performed in Microsoft Excel by grouping assessment skills scores by group and computing the mean academic skills assessment score for each group. The Analysis ToolPak for Excel can be used to assess the statistical significance of mean differences if this is of interest, but at a basic level, using the mean scores for the different groups can help show differences across groups. The analysis was conducted for the single-session and full-day groups separately.
Questions about this use case? Email them to analytics@nhsa.org and we’ll get in touch with the author for you!