Archives:

Select:

Evaluating afterschool: What my toddler taught me about evaluation

READ FULL STORY

Evaluating afterschool: What my toddler taught me about evaluation

By Allison Riley, PhD, MSW, Senior Vice President, Programming and Evaluation at Girls on the Run International. Girls on the Run is a physical activity-based positive youth development program that inspires girls to be joyful, healthy, and confident using a fun, experience-based curriculum that creatively integrates running.

The Afterschool Alliance is pleased to present the seventh installment of our "Evaluating afterschool" blog series, which answers some of the common questions asked about program evaluation and highlights program evaluation best practices. Be sure to take a look at the firstsecondthirdfourthfifth, and sixth posts of the series.

My two-year-old daughter and I like to take walks together when I get home from work. Whether we are headed to see the neighbor’s chickens or visit a friend, we always have some goal in mind when we walk out of the door, though my toddler typically doesn’t take the most direct path. Even if I try to rush her along so we can more quickly reach our destination, she is sure to pause when a good learning opportunity comes her way. When I follow my daughter’s lead, our walks are purposeful yet flexible, and I always learn more, too.

As it turns out, my daughter’s approach to a walk translates well to my workday world. As someone who’s spent my career evaluating youth programming, I have learned the importance of having a clear purpose and goals for a project while being flexible and responsive to information gathered during the evaluation process. Let’s look at a recent Girls on the Run study as an example.

Choosing a path

In 2013, Girls on the Run International embarked on a journey to identify evaluation measures that align with our curriculum in preparation for a rigorous external evaluation study of our program’s effectiveness.

When our preliminary study revealed inconsistencies in how the program was being delivered across our program sites nationwide, we had two choices:

  1. Push forward as planned toward our original goal of conducting the external study, or
  2. Alter our path to be responsive to the information uncovered during the preliminary data collection

While it was tempting to proceed as planned, we understood that without addressing these delivery inconsistencies we would not be able to reach our ultimate program impact goal. While it wasn’t in the original plan, we shifted our resources into a National Coach Training initiative designed to equip our network of more than 50,000 volunteers with the tools and resources they needed to provide a supportive climate and implement our curriculum. After developing and launching this strategy we were able to redirect focus back to the external study.

Making it happen

Here are a few strategies to consider to ensure your evaluations have a clear goal while being flexible and responsive to information you uncover along the way.

  1. Build a shared vision with key stakeholders from the beginning. Key stakeholders and program evaluators must have a shared vision of the goals of any evaluation activities and must be ready to be flexible and responsive. At Girls on the Run, being responsive meant extending our evaluation timeline and shifting resources to address the needs revealed during the first phase of the evaluation. Without support from key stakeholders including the board of directors, our academic partner, and our funder, it would have been very challenging to alter our course.
  2. Be serious about learning along the way. As funders and program leaders have begun to rely more heavily on data and measurement, the emphasis on program evaluation has increased. While this growing focus is exciting, it can also lead to a desire to put all of our energy into showcasing impact. With this single-minded focus, program improvement can often take the back seat. Like my daughter on our walks, it is important to place a high value on learning along the way to your destination.
  3. Build in continuous feedback loops. It is tempting to make a plan, launch the evaluation, and then part ways with stakeholders until the report is in hand. In order to keep everyone accountable to the original purpose and goals (and identify any needed adjustments) schedule checkpoints throughout the evaluation project.
  4. If something’s not working, it’s okay to change course. This could mean changing your survey questions or response scales because they aren’t working, or looking at approaches to ensure consistent program delivery before proceeding with the next steps of an external study. Know that you will get to your ultimate destination.
  5. Make the hard decision. It can take courage to step back, slow down, or forge a different path. But the hardest decisions are often the ones that have the greatest payoff when things are all said and done.

For us, being flexible and responsive was not the easy choice, but in the end, we were able to maximize program impact through a focus on program quality and measure that impact with a rigorous, independent study. I suppose my daughter had the right approach all along - purposeful, yet flexible. We will make a great team on the next Take Your Daughter to Work Day.

Data on child well-being: How does the nation measure up?

Understanding how our youth are faring is an important aspect of not only reinforcing, but also strengthening the supports that afterschool provides. Two separate reports released this summer delved into the state of child well-being across the country, measuring our nation’s progress over...

BY: Rina Moss      09/03/19

To elevate equity, diversity, and inclusion, go deeper into your quality standards

By Femi Vance, researcher at American Institutes for Research, Board President for the California School-Age Consortium, Field Consultant for the National Summer Learning Association, and editor of Changemakers! Equity and Access in Out-of-School Time Programs. Implementing high-quality...

BY: Guest Blogger      08/23/19

Evaluating afterschool: The evaluation basics, part III

The Afterschool Alliance is pleased to present the tenth installment of our "Evaluating afterschool" blog series, which answers some of the common questions asked about program evaluation and highlights program evaluation best practices. Be sure to take a look at earlier posts of the...

BY: Guest Blogger      07/19/19

Evaluating afterschool: The evaluation basics, part I

The Afterschool Alliance is pleased to present the eighth installment of our "Evaluating afterschool" blog series, which answers some of the common questions asked about program evaluation and highlights program evaluation best practices. Be sure to take a look at earlier posts of...

BY: Guest Blogger      07/01/19

New report on building data-driven afterschool systems within cities

A data-centric approach has potential to generate actionable information at the systematic and programmatic levels and ultimately to improve the quality and access to afterschool. The Wallace Foundation commissioned a study in partnership with Chapin Hall at the University of Chicago and the Gamse...

BY: Theresa Carr      06/03/19

Data sharing with afterschool leads to student success

With more than 10 million children participating in afterschool programming, the hours between 3 and 6 p.m. hold crucial potential for academic, social, and emotional development that extends well beyond the school day. So how can we ensure that more students have access to high quality afterschool...

BY: Rina Moss      11/13/18

Mapping afterschool in New Mexico

By May Sagbakken, executive director of the New Mexico Out-of-School Time Network All children and youth deserve access to high-quality afterschool and summer learning programs that prepare them for school, college, careers, and life. While many students across the country are receiving valuable...

BY: Guest Blogger      06/05/18

New resources! National and state-level fact sheets on afterschool

As spring finally arrives in Washington, D.C., so does a suite of new resources highlighting key facts and stats on afterschool!   The first resource is a new fact sheet on afterschool, which summarizes positive outcomes for students who participate in programs, as well as the supports...

BY: Nikki Yamashiro      04/13/18

Experts weigh in on the value of afterschool

Last month, we saw afterschool programs across the country open their doors to host Lights On Afterschool events, providing a firsthand look at the broad array of fun, enriching, and engaging activities students take part in at the program and the vital role programs play in their community. A...

BY: Nikki Yamashiro      11/27/17

Looking for evidence on the impact of afterschool?

Look no further! Adding to the research conversation discussing the value of afterschool and summer learning programs, the Afterschool Alliance has released two new fact sheets that provide a sampling of evaluation findings demonstrating the positive impact programs have on students, with subjects...

BY: Nikki Yamashiro      11/09/17

What resources do staff need to support youth mental health and wellness?

By Selena Levy, director of Training at California School-Age Consortium (CalSAC).  May is Mental Health Awareness Month, and we’re highlighting the ways afterschool programs support children and young people as they face challenges including trauma, ACEs, substance use, and beyond....

BY: Guest Blogger      05/23/19

5 steps to effectively implement SEL initiatives in your community

Awareness of the importance of social and emotional learning is beginning to be reflected in policy measures. Practitioners and policymakers alike are continually recognizing that a whole-child approach is necessary for children to be well-positioned to succeed in their academic, personal, and...

BY: Erin Hegarty      02/08/19

Afterschool, community partners featured in Aspen Institute’s "Nation At Hope" report

Today’s youth must navigate a complex, economically competitive, and globally connected world. In efforts to help young people thrive, educators, parents, and leaders have historically focused on academic improvement as the key target for future success. But that’s not the whole...

BY: Dan Gilbert      01/15/19

New issue brief: A big-picture approach to wellness

From empowering students to take charge of their health to training staff to model healthy behaviors, afterschool and summer learning programs across the country are offering safe and supportive environments that promote young people’s healthy eating and physical activity, as well as build...

BY: Nikki Yamashiro      09/07/18

An Ideal Opportunity: The Role of Afterschool in Social and Emotional Learning

High school students in The Possibility Project, New York, N.Y., tackle issues close to their heart, taking on leadership roles to enact change in their communities through performing arts and community action. At Girls on the Run councils located in all 50 states, plus D.C., girls run with their...

BY: Nikki Yamashiro      05/04/18

New research on easy, affordable SEL practices

As the prominence of social and emotional learning (SEL) to support students’ development in school and beyond continues to grow in education circles, challenges implementing SEL programming have also arisen. The latest issue brief in a series supported by The Wallace Foundation, Kernels of...

BY: Nikki Yamashiro      02/20/18

Research shows the benefits of early childhood programs

Over the years, research surrounding participation in early childhood programs—such as preschool, home visiting programs, and parent education programs—has continuously shown improved outcomes for children and families. A research report...

BY: Leah Silverberg      01/25/18

"Building Workforce Skills in Afterschool" highlights promising practices for all ages

The next generation of the American workforce is growing up right now and afterschool programs are vital partners in helping young people discover new passions and work towards their dreams. As in so many other subjects, the variety and versatility of afterschool programming offers opportunities...

BY: Nikki Yamashiro      11/14/17

Afterschool Spotlight: YWCA New Britain - House of Teens

Adapted from our November 2018 issue brief Promoting Civic Engagement Through Afterschool, developed in partnership with the Asia Society Center for Global Education. Civic engagement starts with our nation’s young people. The afterschool field is an essential partner in ensuring...

BY: Nikki Yamashiro      08/16/19

New issue brief showcasing afterschool promoting civic engagement

Earlier this week on my way into work, a father and his daughter, who couldn’t have been older than an elementary schooler, were at the bottom of the metro escalator greeting metro riders with a huge sign that read “VOTE!” That sight brought a huge smile to my face; it was...

BY: Nikki Yamashiro      11/01/18

Reaching 90 percent high school graduation rate by 2020

A recent briefing from Grad Nation marked the release of the 2018 Building a Grad Nation: Progress and Challenge in Raising High School Graduation Rates. The report provides an update on the Grad Nation mission of a 90 percent on-time high school graduation rate for the class of 2020. Examining the...

BY: Leah Silverberg      07/10/18

Afterschool's lasting impact: Q&A with Dr. Milbrey McLaughlin

By Sam Piha, founder and principal of Temescal Associates, a consulting group dedicated to building the capacity of leaders and organizations in education and youth development who are serious about improving the lives of young people. Milbrey McLaughlin has been a leading...

BY: Guest Blogger      06/25/18

"Building Workforce Skills in Afterschool" highlights promising practices for all ages

The next generation of the American workforce is growing up right now and afterschool programs are vital partners in helping young people discover new passions and work towards their dreams. As in so many other subjects, the variety and versatility of afterschool programming offers opportunities...

BY: Nikki Yamashiro      11/14/17

New issue brief showcasing afterschool promoting civic engagement

Earlier this week on my way into work, a father and his daughter, who couldn’t have been older than an elementary schooler, were at the bottom of the metro escalator greeting metro riders with a huge sign that read “VOTE!” That sight brought a huge smile to my face; it was...

BY: Nikki Yamashiro      11/01/18

New issue brief: A big-picture approach to wellness

From empowering students to take charge of their health to training staff to model healthy behaviors, afterschool and summer learning programs across the country are offering safe and supportive environments that promote young people’s healthy eating and physical activity, as well as build...

BY: Nikki Yamashiro      09/07/18

Afterschool after 30 years: Q&A with Dr. Milbrey McLaughlin, part II

By Sam Piha, founder and principal of Temescal Associates, a consulting group dedicated to building the capacity of leaders and organizations in education and youth development who are serious about improving the lives of young people.  Milbrey McLaughlin has been a leading...

BY: Guest Blogger      06/26/18

"Building Workforce Skills in Afterschool" highlights promising practices for all ages

The next generation of the American workforce is growing up right now and afterschool programs are vital partners in helping young people discover new passions and work towards their dreams. As in so many other subjects, the variety and versatility of afterschool programming offers opportunities...

BY: Nikki Yamashiro      11/14/17

New issue brief: Afterschool fostering protective factors

In Colorado, the Montrose Recreation District’s Summer Enrichment Program creates a welcoming environment to help students feel safe to share their feelings. At Jóvenes de Puerto Rico en Riesgo, Inc. (JPRR), also known as Puerto Rico Youth at Risk (PRYR), one-on-one...

BY: Nikki Yamashiro      09/24/19

Afterschool after 30 years: Q&A with Dr. Milbrey McLaughlin, part II

By Sam Piha, founder and principal of Temescal Associates, a consulting group dedicated to building the capacity of leaders and organizations in education and youth development who are serious about improving the lives of young people.  Milbrey McLaughlin has been a leading...

BY: Guest Blogger      06/26/18

Afterschool's lasting impact: Q&A with Dr. Milbrey McLaughlin

By Sam Piha, founder and principal of Temescal Associates, a consulting group dedicated to building the capacity of leaders and organizations in education and youth development who are serious about improving the lives of young people. Milbrey McLaughlin has been a leading...

BY: Guest Blogger      06/25/18

The prevalence of adverse childhood experiences

Close to half of children (45 percent) in the U.S. have experienced at least one adverse childhood experience (ACE)—an experience that could have negative and lasting effects on one’s health and wellbeing, such as depression, drug abuse, and poor physical health. One in 9 children...

BY: Nikki Yamashiro      03/09/18

New polling shows America is for afterschool

Just in time for Lights On Afterschool, the only national rally celebrating afterschool, a new national poll is out that shows strong public support for afterschool. Nearly 9 in 10 adults (89 percent) agree that afterschool programs are important to their community, with bipartisan and nationwide...

BY: Nikki Yamashiro      10/22/18

Why invest public funds in afterschool?

A new infographic released by RAND draws our attention to the key benefits of public funding for afterschool and summer learning programs. Based on their 2017 research report The Value of Out-of-School Programs, which we covered here last year, the infographic reflects the evident need for...

BY: Rina Moss      10/05/18

Mapping afterschool in New Mexico

By May Sagbakken, executive director of the New Mexico Out-of-School Time Network All children and youth deserve access to high-quality afterschool and summer learning programs that prepare them for school, college, careers, and life. While many students across the country are receiving valuable...

BY: Guest Blogger      06/05/18

New resources! National and state-level fact sheets on afterschool

As spring finally arrives in Washington, D.C., so does a suite of new resources highlighting key facts and stats on afterschool!   The first resource is a new fact sheet on afterschool, which summarizes positive outcomes for students who participate in programs, as well as the supports...

BY: Nikki Yamashiro      04/13/18

New issue brief: A big-picture approach to wellness

From empowering students to take charge of their health to training staff to model healthy behaviors, afterschool and summer learning programs across the country are offering safe and supportive environments that promote young people’s healthy eating and physical activity, as well as build...

BY: Nikki Yamashiro      09/07/18

An Ideal Opportunity: The Role of Afterschool in Social and Emotional Learning

High school students in The Possibility Project, New York, N.Y., tackle issues close to their heart, taking on leadership roles to enact change in their communities through performing arts and community action. At Girls on the Run councils located in all 50 states, plus D.C., girls run with their...

BY: Nikki Yamashiro      05/04/18

The prevalence of adverse childhood experiences

Close to half of children (45 percent) in the U.S. have experienced at least one adverse childhood experience (ACE)—an experience that could have negative and lasting effects on one’s health and wellbeing, such as depression, drug abuse, and poor physical health. One in 9 children...

BY: Nikki Yamashiro      03/09/18

"Building Workforce Skills in Afterschool" highlights promising practices for all ages

The next generation of the American workforce is growing up right now and afterschool programs are vital partners in helping young people discover new passions and work towards their dreams. As in so many other subjects, the variety and versatility of afterschool programming offers opportunities...

BY: Nikki Yamashiro      11/14/17