Evaluating afterschool: Answering questions about quality

READ FULL STORY

Evaluating afterschool: Answering questions about quality

By Jocelyn Michelsen, Senior Research Associate at Public Profit, an Oakland, California-based evaluation consultancy focused on helping high-performing organizations do their best, data-driven work with children, youth, and families.

The Afterschool Alliance is pleased to present the fifth installment of our "Evaluating afterschool" blog series, which answers some of the common questions asked about program evaluation. Be sure to take a look at the firstsecondthirdand fourth posts of the series.

Raise your hand if this scenario sounds familiar: you keep up with new research on afterschool by reading articles and newsletters, following thought leaders, and attending conferences—but it is still hard to sort through all the information, let alone implement new strategies. Research often seems out of touch with the realities of programs on the ground, and while many anecdotal examples are offered, how-to guidelines are few and far between.

As an evaluator consulting with diverse afterschool programs across the San Francisco Bay Area and beyond, I frequently hear this frustration from program leaders. There is a real gap between the research and the steps that staff, leadership, and boards can take to build quality in their own programs. Additionally, it can be hard to sift through the research to get to the ‘why’—why implement these recommendations, why invest time and resources, why change?

We’ve taken on some of that sifting to help programs move toward actually using the research. In our brief After School Quality: Moving From Research to Practice, we plumb the literature and distill our firm’s experience to answer some of these challenging questions:

Why does quality matter?

Program quality is the hinge between youths’ afterschool program participation and youths’ positive outcomes. It is not enough for youth to be present at a program – to truly benefit in the short-term and later on in life, youth must have access to afterschool experiences that are of high quality.

As well, program quality increasingly matters to districts and state departments of education (including the California Department of Education). Measuring program quality is increasingly a feature of afterschool funding and as such is growing in importance and visibility within the field.

What does quality look like?

Programs that are high quality weave in positive youth development practices such as relationship building between peers and between youth and staff, platforms for youth choice and voice, chances to develop leadership, opportunities to gain skills and build mastery, and access to well-organized activities that challenge youth in a positive way.

Why should I focus on staff-youth practices?

Program leadership and staff have direct control over the staff-youth interactions within their programs, and as such focusing on how staff interact with and support youth is the best way to understand how programs function and can be refined. The ultimate goal is to scaffold staff to keep helping youth make positive gains by helping them base their practice in well-defined program quality mile markers.

Where do I start?

Depending on your program’s capacity, resources, and goals, there are a number of ways you can start to grow program quality. Creating a culture of feedback and iteration among program staff is a great way to build a strong foundation that will underlie other quality efforts. As well, initiating program observations (also called site visits) using a common observational framework is one of the best ways to assess point-of-service quality over time. Program quality is founded on the tenets of positive youth development, and so any observational tool you are likely to select will be content-neutral – this is great news for programs concerned about matching their program model, needs, content focus, or staff capacity to a specific tool.

How can I learn more?

Public Profit’s After School Quality: Moving From Research to Practice is designed to help programs bring high-quality youth development practices to their own work in a way that is right-sized and feels manageable and sustainable. The brief also links to some free additional resources about selecting an observational tool and how to collect and make sense of your data. Together, these resources will help guide programs through collecting and using data to refine services—and ultimately to work toward achieving even greater positive impacts for youth.

For more information about Public Profit, visit www.publicprofit.net .

Experts weigh in on the value of afterschool

Last month, we saw afterschool programs across the country open their doors to host Lights On Afterschool events, providing a firsthand look at the broad array of fun, enriching, and engaging activities students take part in at the program and the vital role programs play in their community. A...

BY: Nikki Yamashiro      11/27/17

Looking for evidence on the impact of afterschool?

Look no further! Adding to the research conversation discussing the value of afterschool and summer learning programs, the Afterschool Alliance has released two new fact sheets that provide a sampling of evaluation findings demonstrating the positive impact programs have on students, with subjects...

BY: Nikki Yamashiro      11/09/17

Evaluating afterschool: What my toddler taught me about evaluation

By Allison Riley, PhD, MSW, Senior Vice President, Programming and Evaluation at Girls on the Run International. Girls on the Run is a physical activity-based positive youth development program that inspires girls to be joyful, healthy, and confident using a fun, experience-based curriculum that...

BY: Guest Blogger      09/08/17

New poll finds 9 in 10 parents support schools providing afterschool programs

Last night, the Institute for Educational Leadership (IEL) released the 2017 Phi Delta Kappa International (PDK) Poll of Public's Attitudes Towards Public Schools. The overall takeaway from this report, which is PDK’s 49th annual report on Americans’ views toward public...

BY: Nikki Yamashiro      08/29/17

Guest blog: Q&A with an afterschool researcher, part II

Welcome to part II of our Q&A with Neil Naftzger, American Institutes for Research (AIR), about his evaluation work related to 21st CCLC programs specifically and the afterschool field broadly. Below are answers to oneof the questions we asked, with our emphasis added in bold, which...

BY: Guest Blogger      08/18/17

Guest blog: Q&A with an afterschool researcher

In May, the proposed FY2018 budget eliminated funding for the 21st Century Community Learning Centers (21st CCLC), the only federal funding stream dedicated to before-school, afterschool, and summer learning programs. In the budget, a justification given for the elimination of funding was that...

BY: Guest Blogger      07/28/17

Evaluating afterschool: Building an evaluation advisory board

By Jason Spector, Senior Research & Evaluation Manager at After-School All-Stars The Afterschool Alliance is pleased to present the sixth installment of our "Evaluating afterschool" blog series, which answers some of the common questions asked about program evaluation and...

BY: Guest Blogger      07/05/17

Evaluating afterschool: Answering questions about quality

By Jocelyn Michelsen, Senior Research Associate at Public Profit, an Oakland, California-based evaluation consultancy focused on helping high-performing organizations do their best, data-driven work with children, youth, and families. The Afterschool Alliance is pleased to present the fifth...

BY: Charlotte Steinecke      06/06/17

What you need to know about the GAO's afterschool report

The U.S. Government Accountability Office (GAO) released a report on 21st Century Community Learning Centers on April 26 highlighting the benefits of afterschool participation and calling on the U.S. Department of Education to update their performance measures and data collection. The report...

BY: Jen Rinehart      04/28/17

New report: Participation in summer learning programs yields positive outcomes

A new report shows that high levels of participation in summer learning programs can provide positive benefits for low-income students’ math and language arts performance and social-emotional skills. Last week, The Wallace Foundation released Learning from Summer:...

BY: Erin Murphy      09/15/16

"Building Workforce Skills in Afterschool" highlights promising practices for all ages

The next generation of the American workforce is growing up right now and afterschool programs are vital partners in helping young people discover new passions and work towards their dreams. As in so many other subjects, the variety and versatility of afterschool programming offers opportunities...

BY: Nikki Yamashiro      11/14/17

How does afterschool contribute to military readiness?

In 2016, the Council for a Strong America released America Unprepared, showing data that more than 70 percent of young adults in the United States would not qualify for military service due to obesity and other health issues, poor academic performance, drug abuse, or involvement in crime. As a...

BY: Leah Silverberg      08/14/17

5 ways afterschool prepares kids to succeed in the workplace

Cultivating tomorrow’s workforce remains a central part of the discussion about America’s economic future. As today’s children begin to develop the skills they will need in the workplace, experts in the education and afterschool fields are turning their attention to the ways that...

BY: Charlotte Steinecke      05/04/17

What is "The State of Girls" in 2017?

Since the Great Recession, the United States has experienced many demographic, social, economic, and technological changes. A new report from the Girl Scout Research Institute, "The State of Girls 2017: Emerging Truths and Troubling Trends," tracks resulting trends in...

BY: Leah Silverberg      02/17/17

When talking about social and emotional learning, what language works best?

We all know that high quality afterschool and summer learning programs provide kids with the skills that they need to succeed in school and life. While the concept sounds simple, finding the best way to describe these skills and their development is anything but. Some people use phrases such...

BY: Dan Gilbert      01/05/17

New report explores gap between school hours and work schedules

The misalignment between parents’ working hours and kids’ school hours is widely recognized by the afterschool community and working parents everywhere. Years of public polling have highlighted that this issue is top-of-mind with parents and voters. That’s why advocates...

BY: Jen Rinehart      10/25/16

New report: Participation in summer learning programs yields positive outcomes

A new report shows that high levels of participation in summer learning programs can provide positive benefits for low-income students’ math and language arts performance and social-emotional skills. Last week, The Wallace Foundation released Learning from Summer:...

BY: Erin Murphy      09/15/16

5 statistics that inspire optimism in the future of America's youth

A new study has found that “Generation Z,” or the cohort of youth born after 1995 that follows millennials, are healthier and have higher rates of high school completion, despite significant challenges posed by the economy and education costs. The 2016 KIDS COUNT Data...

BY: Robert Abare      07/05/16

New civil rights data reveal nearly 1 in 5 high schoolers are chronically absent

This week, the U.S. Department of Education’s Office of Civil Rights released “A First Look” from the 2013-2014 Civil Rights Data Collection (CRDC), which shows that students of color, students whose first language is not English and students with disabilities are...

BY: Jen Rinehart      06/09/16

The spring edition of the Journal of Expanded Learning Opportunities is here!

The third edition of the Journal of Expanded Learning Opportunities (JELO) has arrived! This spring issue features a conversation about quality programming in afterschool, an article on the role that social emotional learning can play to close the achievement and learning gaps, and an article...

BY: Nikki Yamashiro      06/03/16

Poll: In public education, Americans want more than academics

Phi Delta Kappa International (PDK) recently released the results of their 48th Annual PDK Poll of the Public’s Attitudes Toward the Public Schools. This report, Why school? Americans speak out on education goals, standards, priorities and funding, identifies what...

BY: Erin Murphy      10/13/16

New report: Participation in summer learning programs yields positive outcomes

A new report shows that high levels of participation in summer learning programs can provide positive benefits for low-income students’ math and language arts performance and social-emotional skills. Last week, The Wallace Foundation released Learning from Summer:...

BY: Erin Murphy      09/15/16

New video makes the case for data sharing partnerships

If you have ever wondered what a successful data sharing partnership looks like, or wished that there was a resource available to help you make the case for data partnerships in afterschool, look no further. A new video released by the National League of Cities—in partnership with the...

BY: Nikki Yamashiro      08/23/16

How afterschool systems can connect the dots between data and best practices

Having witnessed the benefits that afterschool programs provide students, parents and communities, many U.S. cities are building afterschool systems to link and sustain high-quality afterschool programs and boost access for those in need. But to ensure afterschool...

BY: Robert Abare      06/08/16

Policy priorities to make data work for students

Student data is a valuable resource for afterschool programs. It helps inform program providers about their students’ needs, the programming that will best support their students and the program’s impact on students’ progress. As important of a role that data play for afterschool...

BY: Nikki Yamashiro      06/02/16

Can the U.S. reach 90 percent high school graduation by 2020?

High school graduation rates have continued to grow over the last decade, reaching a record high of 82.3 percent. This graduation rate is 10 percent higher than at the turn of the century; however, over the last year the rate of increase has begun to slow. If the graduation rate continues...

BY: Erin Murphy      05/13/16

Webinar recap: Afterschool in rural communities

Afterschool programs are an integral partner for rural communities: keeping kids safe, inspiring learning and supporting working families. The Afterschool Alliance recently explored the current state of afterschool in rural communities in our America After 3PM special...

BY: Erin Murphy      05/04/16

New report underscores the high cost of child care

Affordable and accessible high-quality child care is a critical issue for working families across the U.S. Although the benefits of quality child care for both children and their parents are numerous, many families struggle to afford and find child care that meets their needs. The 10th edition of...

BY: Nikki Yamashiro      12/21/16

New report explores gap between school hours and work schedules

The misalignment between parents’ working hours and kids’ school hours is widely recognized by the afterschool community and working parents everywhere. Years of public polling have highlighted that this issue is top-of-mind with parents and voters. That’s why advocates...

BY: Jen Rinehart      10/25/16

Poll: In public education, Americans want more than academics

Phi Delta Kappa International (PDK) recently released the results of their 48th Annual PDK Poll of the Public’s Attitudes Toward the Public Schools. This report, Why school? Americans speak out on education goals, standards, priorities and funding, identifies what...

BY: Erin Murphy      10/13/16

Updated interactive dashboard with data on high-poverty communities

Following the release of our latest America After 3PM report, Afterschool in Communities of Concentrated Poverty, which looks at the role of afterschool programs in areas where there is a high concentration of families living below the poverty line, our interactive web dashboard has been...

BY: Nikki Yamashiro      09/26/16

New report reveals how afterschool aids communities of concentrated poverty

Where you live has direct and indirect impacts on the fundamental resources and opportunities you count on, and which many people may take for granted. Your location affects the quality of schools available to you, your access to healthy and affordable food, and your overall wellbeing and future...

BY: Nikki Yamashiro      08/30/16

5 statistics that inspire optimism in the future of America's youth

A new study has found that “Generation Z,” or the cohort of youth born after 1995 that follows millennials, are healthier and have higher rates of high school completion, despite significant challenges posed by the economy and education costs. The 2016 KIDS COUNT Data...

BY: Robert Abare      07/05/16

Creating high-quality arts programs in national youth-serving organizations

While research has shown that participation in the arts promotes positive youth outcomes, providing quality arts programming can seem like an unobtainable goal to many programs, especially those that mix various art disciplines into daily programming but do not have an arts focus. But a new report...

BY: Leah Silverberg      10/27/17

New report from the Wallace Foundation: Strategies to scale up

The question of how to scale up—taking a successful program, project, or policy and growing it to expand its reach and therefore its impact—has been an important one when thinking about systems change. It is a key component in efforts to make sustainable, positive social gains; a...

BY: Nikki Yamashiro      10/06/17

New resources for STEM in afterschool from the Research + Practice Collaboratory

Check it out: the Research + Practice Collaboratory has some new and updated resources for the afterschool field! If you are not familiar, the Research + Practice Collaboratory works to bridge the gap between education research and STEM education implementation. The Collaboratory’s goal is to...

BY: Leah Silverberg      07/10/17

New report explores gap between school hours and work schedules

The misalignment between parents’ working hours and kids’ school hours is widely recognized by the afterschool community and working parents everywhere. Years of public polling have highlighted that this issue is top-of-mind with parents and voters. That’s why advocates...

BY: Jen Rinehart      10/25/16

How afterschool systems can connect the dots between data and best practices

Having witnessed the benefits that afterschool programs provide students, parents and communities, many U.S. cities are building afterschool systems to link and sustain high-quality afterschool programs and boost access for those in need. But to ensure afterschool...

BY: Robert Abare      06/08/16

How does afterschool contribute to military readiness?

In 2016, the Council for a Strong America released America Unprepared, showing data that more than 70 percent of young adults in the United States would not qualify for military service due to obesity and other health issues, poor academic performance, drug abuse, or involvement in crime. As a...

BY: Leah Silverberg      08/14/17

New resources for STEM in afterschool from the Research + Practice Collaboratory

Check it out: the Research + Practice Collaboratory has some new and updated resources for the afterschool field! If you are not familiar, the Research + Practice Collaboratory works to bridge the gap between education research and STEM education implementation. The Collaboratory’s goal is to...

BY: Leah Silverberg      07/10/17

Poll: In public education, Americans want more than academics

Phi Delta Kappa International (PDK) recently released the results of their 48th Annual PDK Poll of the Public’s Attitudes Toward the Public Schools. This report, Why school? Americans speak out on education goals, standards, priorities and funding, identifies what...

BY: Erin Murphy      10/13/16

5 statistics that inspire optimism in the future of America's youth

A new study has found that “Generation Z,” or the cohort of youth born after 1995 that follows millennials, are healthier and have higher rates of high school completion, despite significant challenges posed by the economy and education costs. The 2016 KIDS COUNT Data...

BY: Robert Abare      07/05/16

School's Out New York City afterschool program fosters success in middle schoolers

A new evaluation of School’s Out New York City (SONYC), supported by the New York City Department of Youth and Community Development (DYCD), finds that the initiative offers New York City middle schoolers a pathway to success through high-quality afterschool programming. Their...

BY: Erin Murphy      05/06/16

Updated interactive dashboard with data on high-poverty communities

Following the release of our latest America After 3PM report, Afterschool in Communities of Concentrated Poverty, which looks at the role of afterschool programs in areas where there is a high concentration of families living below the poverty line, our interactive web dashboard has been...

BY: Nikki Yamashiro      09/26/16

New report reveals how afterschool aids communities of concentrated poverty

Where you live has direct and indirect impacts on the fundamental resources and opportunities you count on, and which many people may take for granted. Your location affects the quality of schools available to you, your access to healthy and affordable food, and your overall wellbeing and future...

BY: Nikki Yamashiro      08/30/16

New civil rights data reveal nearly 1 in 5 high schoolers are chronically absent

This week, the U.S. Department of Education’s Office of Civil Rights released “A First Look” from the 2013-2014 Civil Rights Data Collection (CRDC), which shows that students of color, students whose first language is not English and students with disabilities are...

BY: Jen Rinehart      06/09/16

Webinar recap: Afterschool in rural communities

Afterschool programs are an integral partner for rural communities: keeping kids safe, inspiring learning and supporting working families. The Afterschool Alliance recently explored the current state of afterschool in rural communities in our America After 3PM special...

BY: Erin Murphy      05/04/16