What patterns are there in our self-assessment?
In the spring of 2019, ABC Head Start* began using data visualization to better understand their self-assessment data. Before then, they collected self-assessment data annually in both manual and digital forms, which was time-consuming and not as informative as it could be. By using data visualization, they were able to enhance their analysis and highlight trends or patterns that previously went unnoticed. ABC Head Start was able to identify what centers had the highest rates of findings, the areas of greatest concern, and the number and status of these findings.
(*ABC Head Start is a generic name we use in the Playbook when a program does not wish to be identified.)
ANALYSIS
In order to do this analysis, ABC Head Start recorded all of their self-assessment data in the DataSay software. They had previously conducted their self-assessment largely on paper, without the benefit of data aggregation or visualizations to guide their analysis or follow-up actions.
To create their self-assessment in DataSay, ABC Head Start created a self-assessment activity then identified the centers, classrooms, and resources they wanted to review in the process and what tools/checklists they wanted to use to conduct those reviews, e.g. bus inspections on 25% of their bus pool using the bus safety inspection checklist. After identifying the scope of their annual self-assessment, ABC Head Start conducted the self-assessment entirely inside of DataSay.
After conducting the self-assessment and data collection, ABC Head Start began to analyze the data they collected. Basic data visualizations are available in real-time as they conduct their self-assessment. Additionally, ABC Head Start wanted to normalize their data for reporting purposes so they cleaned and formatted their data, selected all the reviews conducted for their annual self-assessment, and uploaded a csv file into a DataSay data dashboard. The dashboard converted the raw, flat data into visualized results in real-time. The visualized results were displayed and ready for analysis.
In the pie charts below, ABC Head Start was able to look at findings, strengths, and reviews conducted across all their centers. “Findings” indicate situations that require follow-up actions, “strengths” highlight practices that are contributing to positive outcomes and quality improvement, and “reviews” indicate how often a center is conducting monitoring events.
RESULT
By looking at the “Findings by Category” pie chart, ABC Head Start found that School Readiness and Child Development (abbreviated as ECD) was the biggest category of findings in their self-assessment. They used the drilldown feature of the dashboard to highlight ECD findings specifically by clicking on that piece of the pie.
The dashboard then specifically highlighted the number of all total findings for each center (labeled “Area” in the dashboard) that were in the ECD category. This deeper red part of the bar showed ABC Head Start that 11 of their 13 centers had ECD findings.
In addition to the high-level dashboard, ABC Head Start used the drill down feature of the dashboards to access more granular data and review specific findings to determine corrective actions as displayed in the graphic below. (Details have been blurred below to protect privacy).
Using this drill down feature in DataSay and reviewing these specific findings, ABC Head Start identified that there were two main issues that accounted for the majority of the ECD findings in their classrooms: lagging language development skills and delayed social and emotional development indicators. Further analysis indicated that the deficiencies were persistent across all classrooms.
To address these issues, the program reviewed their current curriculum tools and placed an emphasis on early literacy and communication, including listening, speaking, and vocabulary development. They also decided to increase their school readiness monitoring assessments to a quarterly cycle and to actively measure student achievement against established baselines and annual achievement goals. They will monitor the results over the next several program years to see if their new approach is working to narrow the school readiness gap and make changes to the curriculum and staff training as indicated by their outcomes.
ADDITIONAL DETAILS
AUDIENCE: Management, Leadership
LEVEL OF ANALYSIS: Center
DIFFICULTY: Basic
CONTENT AREA: Child development, program management
DATA SOURCES: DataSay
CONSIDERATIONS AND CAVEATS: To have this level of data available, the program conducted all its reviews for self-assessment within the DataSay platform. Reviews conducted outside the system or not entered into the system would not have data represented.
TECHNICAL APPENDIX
As long as self-assessment reviews are conducted and reported within the DataSay platform, this information is available directly in the program.
Questions about this use case? Email them to analytics@nhsa.org and we’ll get in touch with the author for you!