Understanding "evidence-based" programs to make the case for ARP funds

In March 2021, President Biden signed the American Rescue Plan (ARP) into law, providing $122 billion in federal education funds to support students impacted by the pandemic. Since then, state education agencies have submitted their proposed plans for the use of the funds to the Department of Education, including how they plan to use their portion set aside for afterschool and summer learning programs. We’ve seen many states offer diverse solutions—for example, Kansas will provide funding to youth serving organizations such as Boys and Girls Clubs, the YMCA, and 4-H for hands-on, project-based enrichment, and math and reading recovery. Georgia combined the afterschool and summer funds to create the “Building Opportunities in Out of School Time (BOOST)” grant competition for community based providers. New Hampshire has established funding to train camp counselors in how to identify and respond to mental and behavioral health issues.

However, part of the ARP funding requirements specify that the afterschool and summer learning and enrichment programs are “evidence-based.” To better understand what this means, how to talk about the value of programs, and how to effectively make the case for ARP funding, the American Institutes for Research released four evidence guides. Below is a summary of the content in each of the four guides.

Guide 1: Understanding what “research” and “evidence” mean in the context of ARP funding

The first guide, “The Overview,” defines key terms—research and evidence—that are in the federal relief plans and discusses how they are different from one another. Research refers to the process of testing a program or practice to answer a question (i.e. to determine effectiveness), while evidence refers to the information learned from the research that allows us to draw conclusions about intended outcomes. Evidence-based practices, therefore, come from studies with documented evidence of effectiveness.

Guide 2: Outlining best practices and providing examples of what that looks like in programs

The second guide, “What does the research say about afterschool and summer programming?” dives into the evidence base on afterschool and summer learning programs to help programs understand current research and includes corresponding examples of ways to design and implement these quality practices within programs. Six key takeaways were highlighted:

1. Context matters—Afterschool programs should consider family and community voices to best reflect the culture and resources of the community and meet student needs
2. Adults matter—Caring adults are important in the lives young people, and afterschool programs need to hire quality staff and provide them with continued professional development
3. Relationships matter—Relationship-building is the basis for creating safe and supportive environments, and afterschool programs should foster these relationships
4. Safe and supportive environments matter—The structure of the program needs to promote safety and create a welcoming space for all youth
5. Attendance and engagement matter—The benefits of afterschool program participation are tied to attendance levels, so programs should find ways to encourage program participation and retain students
6. Learning and development matter—Afterschool programs have a host of benefits for children who participate, including higher school-day attendance, lower disciplinary referrals, and improved social skills. Programs need to be intentional about the outcomes they hope to achieve and ensure their programming is designed to support these outcomes.

Overall, one of the key important features of afterschool and summer programs is that they are comprehensive and foster whole child development, focusing on both academics and social and emotional learning.

Guide 3: Explaining how to demonstrate a program is evidence-based

The third guide, “What does “evidence-based” mean for afterschool and summer learning programs?” explores what evidence-based actually means for afterschool and summer learning programs. One way to categorize and determine what evidence-based means is using the four tiers of evidence under the Every Student Succeeds Act (ESSA). Programs need to be both well-designed and well-implemented, and using research and evidence from these four tiers can be beneficial. While some funding streams from the Department of Education require a certain level of evidence, others allow any of the four tiers. However, regardless of the evidence requirements, programs need to know how they will improve outcomes.

Guide 4: Recommendations for programs to ensure that they are well designed and well implemented, and a logic model template to help

The final guide, “What can afterschool and summer programs do to leverage this historic opportunity?” offers strategies for afterschool and summer programs to leverage the opportunity that ARP funding has presented to the field. The main strategy highlighted in this guide is creating a logic model to ensure that programs are well designed, well implemented, and align with research and evidence. A logic model is a visual representation of a program’s resources, activities, and expected outcomes, and can be a useful tool to drive program planning, implementation, evaluation, and continuous improvement.

Together, these four guides can help programs make the case for ARP funding by ensuring that programming is grounded in research and evidence-based practices. The guides can be read in order, or as separate documents, depending on one’s needs and interests.