Evaluation is a critical component in designing and operating effective programs and interventions. Program managers and policymakers frequently utilize evaluations as an accountability tool in making decisions about programs to fund, modify, expand or eliminate. 

Kelly McGinn, assistant professor in the Psychological Studies in Education department in the College of Education and Human Development (CEHD), designed the hands-on Program Evaluation course, which she also teaches, to equip students, especially students in the human development and community engagement program (HDCE), with the skillset they need to conduct evaluations. This was one of her first big projects when she joined Temple University as a faculty member in 2018. McGinn currently serves as the HDCE program coordinator. 

Tailored to the need of students, the Program Evaluation course introduces students to evaluation methodology and tools commonly used to assess programs. Students will become familiar with the concepts, methods, and applications of evaluation research, learn how to read evaluation research critically, understand how to use evaluation results to improve program performance, and be able to propose and execute an appropriate evaluation plan to examine program implementation and effectiveness. 

"The goal is to ensure that students are not only learning how to evaluate theoretically, but also applying their knowledge in a practical and meaningful way," McGinn shares. "HDCE students are equipped with a range of courses designed to prepare them for impactful careers. They undergo communication, research, and human development courses, along with this required Program Evaluation course usually taken in their senior year. The interdisciplinary nature of the curriculum empowers students to understand organizational strengths and needs, facilitating positive changes." 

The course is also built with the demands of the job market in mind. HDCE graduates often pursue careers in "helping professions." The diverse interests of students lead them to fields such as school psychology, counseling psychology, social work, applied behavioral analysis, or various non-profit organizations, highlighting the versatility of the HDCE programs. 

"Many students choose to work with children, but outside of the classroom," McGinn shares. "Students are often interested in working in the non-profit space, including education-based non-profits such as after-school-time programs, libraries and museums, non-profits focusing on the school-to-prison pipeline, as well as LGBTQ+ based non-profits." 

It is critical that students understand the importance of program evaluation "because evaluation tools are used and needed across all program areas, in both public and private sectors." 

For programs to be successful, they need answers to many questions. For instance, who is a program intended to serve? What are the outcomes for those who received the intervention? How can a program be improved to better meet its goals? What needs to be adapted for a different population or setting? Why should a program continue to be funded? 

Students taking the Program Evaluation course are actively engaged in thinking and asking these critical questions themselves. They do this through a comprehensive project where they conduct their own evaluations by working with a program of their choice. 

"Often, students are advised to take Program Evaluation together with their internship course. Many choose to work with their internship sites for this project. This real-world connection allows students to contribute meaningfully to the community by collecting valuable data to improve programs," says McGinn. "For students who have not yet had an internship, they can work with an organization that they are connected with or volunteering at. It can also be their place of work. Sometimes they evaluate things right here at Temple, an office within the College of Education and Human Development." 

This semester-long program evaluation project is divided into four parts, providing a structured framework for students. In the first part, students learn to describe the program they plan to evaluate and obtain essential permission to conduct their evaluation at that organization. Subsequent stages involve creating a logic model, developing an evaluation plan, collecting data and finally, analyzing, interpreting, and presenting the results. Throughout these stages, students receive feedback from both peers and McGinn, fostering a collaborative and cumulative learning environment. 

Senior Madison Kerney says the Program Evaluation course taught her "how to ask the right questions concisely and that an effective and thorough evaluation should have a clear focus and an obtainable outcome." 

"I tend to think too big sometimes when it comes to evaluation and asking questions. I get very curious, and I want to know everything. That makes things challenging when conducting an evaluation because it is impossible. If you get too caught up on everything you want to know, your evaluation can become too broad and vague. Find a specific focus and ask one or two questions," Kerney says. "Professor McGinn showed me how to restructure my questions and make them clear to others. Just because I know what I mean to say or ask, does not mean that others do." 

Currently, Kerney works as a part-time special service assistant for the School District of Philadelphia, while maintaining her internship with Heights Philadelphia. She is graduating with her bachelor of science in HDCE in spring 2024 and is expected to graduate from the adult and organizational development +1 accelerated master's program in spring 2025. 

"The course mainly emphasizes two types of evaluations: process and outcome evaluations," says McGinn. "Process evaluations explore whether the program's activities have been implemented as intended. Outcome evaluations measure program effects in the target population" 

Students are encouraged to integrate one or both types of evaluations into their projects. This approach allows them to delve into the activities and outcomes of the programs they evaluate, providing organizations with valuable insights for improvement. 

"Students are collecting actual new data that the organization hasn't already collected, answering evaluation questions that are important to the organization," McGinn further elaborates. "Most of the jobs that our students are interested in tend to require at least a piece of evaluation knowledge. Whether it is collecting and interpreting their own data, or, reading a report and being able to understand what it is saying and making changes and recommendations based on that." 

Carissa George, a junior double majoring in HDCE and criminal justice, shares that her biggest takeaway from the Program Evaluation course is realizing and understanding how crucial program evaluation is in the social services sector. 

"I currently work as a youth conflict specialist intern with systems-involved youth for the Resolutionary Partners program. This course has helped me in this role professionally because I am now able to apply strategies for collecting and analyzing quantitative data to help determine best practices for delivering instruction to this population of youth," says George. 

George says she wants to work in juvenile justice reform, making positive social change by influencing programs and policies, especially in communities like Philadelphia. 

Helping HDCE students understand organizational strengths and needs, as well as preparing them to make an impact, has always been at the center of what faculty and staff in the College of Education and Human Development do. McGinn adds that her vision for the course from the start was to make it as practical and community centered as possible. 

Currently working in research at the Children's Hospital of Philadelphia, senior HDCE student Emily Krask, whose goal is to become a pediatrician, says that she looks forward to taking the lessons from Dr. McGinn's program evaluation course into her work in the future.  

"I have a strong interest in public health research and the implementation of community-based programs. I have no doubt in my mind that, in my career, I will be able to build on the fundamental knowledge I gained in this Program Evaluation course to evaluate and improve programs that I will play an active role in," Krask emphasizes. 

To ensure that the course remains relevant and engaging, McGinn incorporates a participatory evaluation of the course itself, involving students in the feedback process. Modifications have been made to the course in terms of course modalities, how feedback was provided, reading materials and more. 

Teaching and interacting directly with HDCE students allow McGinn to reflect on her own education journey.  

"I always tell people HDCE should have been my major as an undergraduate student. It has always been what I have been interested in, psychology and education together, and HDCE does an excellent job of combining the two," she says. "My personal passion was always education outside of the Pre-K to 12 classrooms, and that is what this major is really about." 

McGinn's approach to the Program Evaluation course design exemplifies a commitment to practical, hands-on learning that empowers students to make a tangible impact in their communities. By bridging theory and practice, this course is a testament to the transformative power of education in shaping future leaders in human development and community engagement. 

"It's fulfilling to see students use their knowledge to help make community impact and change when they can, even at the undergraduate level," McGinn shares. "Hearing the positive feedback from the students and the organizations they presented their results to, and especially knowing that the organizations implemented changes based on their findings, is rewarding."