By Nicole Lovecchio, Chief Program Officer at WINGS for kids.
The Afterschool Alliance is pleased to present the third installment of our "Evaluating afterschool" blog series, which turns to program providers in the field to answer some of the common questions asked about program evaluation. Be sure to take a look at the first and second posts of the series from Dallas Afterschool and After-School All-Stars.
As afterschool providers, we know that the hours from 3 to 6 p.m. offer an incredible opportunity to engage students who need it most and help them feel more connected to their peers, the school day, and to their community—and the key to maximizing that potential lies in the skills and abilities of the afterschool staff.
For many years we worked on codifying and documenting every element and detail of our program. We created manuals explaining how we ran our social and emotional learning (SEL) afterschool program in an effort to replicate our program throughout several sites in the Southeast. We then focused heavily on fidelity and the implementation of these program elements.
Along the way, we realized that a checklist of items, however exact, couldn’t guarantee a high-quality program. By gathering data on our staff and kids, we were able to see the shift that was needed: clearer focus on building up the skills of our staff on the ground.
3 ways data collected impacted program quality
Improving adult skills and practices. Quarterly program assessments (observations) of our sites uncovered a trend indicating that our staff (12 college-aged mentors per site who work in 1:12 ratio with students) were primarily focused on reacting to the negative behaviors of our kids. Because our staff was reactive instead of proactive, there was little room for engaging activity. Therefore, we redesigned staff trainings and redefined the WINGS approach to SEL, all with a focus on building adult skills and practices first and foremost.
The S-E-T Approach to SEL is multi-pronged; at the core lies adult skills and practices. This approach provides the building blocks that allow adults to cultivate a culture of SEL through positive student support and engagements, and implicit and explicit teachings. Once adults comprehend and internalize these skills on an individual level, they are then able to transfer these skills and knowledge as educators, mentors, and youth workers.
We have used S-E-T to shape our week-long 40-hour training for incoming and returning staff as well supplemental trainings that occur over the course of the year. The S-E-T Approach has refocused staff on three main points by encouraging them to reflect on: Am I supporting? Am I engaging? Am I teaching? Now, through our program assessments and observations, we see a staff that is engaged and involved and we see a culture where kids feel supported, listened to, and safe.
Assessing performance. When we instituted the S-E-T Approach to SEL at WINGS, we then made sure our staff was assessed on their performance regularly. Staff now undergo monthly performance assessments on how S-E-T they are, where each category of S-E-T is scored on a scale of 1 to 5. Any staff member scoring below a 3 in any part of S-E-T requires a Performance Improvement Plan addressing ways to improve that skill set. These Performance Improvement Plans document staff’s skill development and competencies as well as the monitoring and follow-up that will be required. Staff now have clear and cohesive expectations and regular performance evaluations to ensure their skill development is on display -- driving up program quality and a positive program climate.
Connecting to the school day. With the data we’ve collected and applied, WINGS has improved program quality through staff trainings, assessments, and observations. Improvements in staff skills and SEL are vital steps to creating an environment where kids can be kids, a place where they both celebrate and work through struggles, all while feeling understood by caring adults.
WINGS for kids is an education organization that believes strong social and emotional skills transforms the lives of kids. Our mission is to help as many low-income kids as possible experience high-quality and effective social and emotional learning. We work towards this aim because research shows that strong social and emotional skills are the key to helping kids succeed in school, be prepared for the workforce, and become positive and healthy contributors to society. www.wingsforkids.org