Evaluative Thinking with Thomas Archibald

James Pann interviews Melanie Hwalek, Ph.D., an evaluation researcher and consultant, to discuss the retrospective pretest (RPT) design, focusing on its practical applications and the findings from her recent research detailed in the paper, “Designing a Questionnaire with Retrospective PrePost Items: Format Matters.” Below, I break down what a retrospective pretest is, when it works...

Empowerment evaluation is an approach that puts evaluation in the hands of the people who run and participate in a program. An external evaluator does not design the study, collect the data, and deliver a report alone. Program staff and community members lead the evaluation, and the evaluator works as a coach. David Fetterman developed...

Whether you're an educator, a student, or simply someone passionate about youth development and educational programs, this podcast episode with Tiffany Berry, PhD., promises to give you insights into the complex world of evaluation. She is the Dean and a full research professor in the Division of Behavioral & Organizational Sciences at Claremont Graduate University, where she also received her Ph.D. Her research interests include Educational Program Evaluation, Educational Curricula, Comprehensive School Reform, after-school services, social-emotional learning, and other areas. She has published over 75 technical evaluation reports and peer-reviewed articles in leading evaluation and youth development journals. See XXXXX for a detailed description of the interview. OUTLINE: 0:00 Overview of episode 2:07 Evaluation models and conceptual models she uses 8:56 Common challenges Tiffany encountered evaluating educational programs 11:59 Challenges in measuring youth development outcomes 17:36 Changing her evaluation approach due to unforeseen circumstances or changes in the program 23:00 How to stay current in the field of evaluation 28:10 Advice for those interested in becoming an evaluator 32:41 Advantages and disadvantages of getting a graduate degree in evaluation versus other disciplines 38:15 Credentialing in evaluation in the US 44:32 The uniqueness of Claremont’s evaluation program 49:40 Claremont’ Doctoral program in evaluation 55:09 Building a professional network in the evaluation field CONNECT WITH JAMES: - Subscribe to this YouTube channel: https://www.youtube.com/user/jamesmpa... - LinkedIn: https://www.linkedin.com/in/pannjames/ Please reach out with comments and questions. Thanks!

In this episode, I talk with Jim McDavid, Ph.D., about his experience with mindfulness and meditation practice, how it has influenced him, and how it affects how he views and practices evaluation. Our conversation also covers practical wisdom, Jim’s interest in the environment, and challenges associated with determining cause and effect in evaluation. Jim is Professor Emeritus at the School of Public Administration, University of Victoria, which he joined as faculty in 1980. He was a recipient of the University of Victoria Alumni Teaching Award, and he received the University of Victoria’s highest academic honor, the UVic Distinguished Professorship Award. He has contributed significantly to the field of evaluation, and the reason I reached out to him related to his work related to mindfulness and evaluation. 00:00 Introduction 01:57 Jim’s definition of mindfulness 05:20 The ethical dimension of mindfulness 08:49 How Jim’s mindfulness practice has evolved over time 11:46 Difference between Transcendental Meditation and Vipassana 15:33 The spiritual dimension of meditation 22:10 Subjectivity of cause and effect 29:34 How mindfulness opens up the mind 35:52 Mindfulness can support evaluation practice 40:32 Connection to practical wisdom 44:45 Importance of being present and really listening in evaluation contexts 49:20 Judgment and evaluation 50:46 Starting a mindfulness practice For more go to: https://evalnetwork.com/mindfulness-meets-evaluation/

Every survey depends on who answers it. A well designed instrument with a weak response rate can produce data that looks precise and points in the wrong direction, because the people who responded differ from the people who did not. This guide covers what counts as a good response rate, why low rates put your...

James Pann, Ph.D., interviews David Victorson, Ph.D., of True North Treks, a nonprofit organization whose mission is to empower young adults and caregivers affected by cancer to “find direction through connection” and mindfulness. As a child, David grew up surrounded by nature and its many restorative benefits. Therefore, when he went on to complete his...

James Pann, Ph.D. interviews Dana Wanzer, Ph.D., assistant professor of psychology in evaluation at the University of Wisconsin-Stout, about her American Journal of Evaluation paper “What is evaluation? Perspectives of how evaluation differs or not from research.” Dana surveyed members of the American Evaluation Association and the American Educational Research Association to find out how...

In this interview, I speak with Amy Gullickson, acting Co-Director and Senior Lecturer at the Centre for Program Evaluation at Melbourne Graduate School of Education. She is also Chair of the International Society for Evaluation Education. We talk about what is the best definition of evaluation and why it is important to have a clear definition. Amy...

Thomas Archibald, Ph.D. is an Associate Professor and Extension Specialist in the Department of Agricultural, Leadership, and Community Education at Virginia Tech. His practice and research are focused primarily on program evaluation and evaluation capacity building. He serves as the Chief of Party/Director of the Feed the Future Senegal Youth in Agriculture project, funded by...

Rakesh Mohan has been the director of the Office of Performance Evaluations (OPE) an independent and nonpartisan agency of the Idaho State Legislature, since 2002. We cover important issues related to evaluation and how evaluation can guiding government and public policy. Under Rakesh’s direction, the office has received many awards, including the American Evaluation Association’s...