•Implementation of a values-engaged evaluation approach was examined.•Evaluators explicitly attended to culture, diversity, and equity.•The case setting was the evaluation of a multi-year STEM ...program.•Evaluators were most successful when using formal and informal communication.
Evaluation must attend meaningfully and respectfully to issues of culture, race, diversity, power, and equity. This attention is especially critical within the evaluation of science, technology, engineering, and mathematics (STEM) educational programming, which has an explicit agenda of broadening participation. The purpose of this article is to report lessons learned from the implementation of a values-engaged, educative (Greene et al., 2006) evaluation within a multi-year STEM education program setting. This meta-evaluation employed a case study design using data from evaluator weekly systematic reflections, review of evaluation and program artifacts, stakeholder interviews, and peer review and assessment. The main findings from this study are (a) explicit attention to culture, diversity, and equity was initially challenged by organizational culture and under-developed evaluator–stakeholder professional relationship and (b) evidence of successful engagement of culture, diversity, and equity emerged in formal evaluation criteria and documents, and informal dialogue and discussion with stakeholders. The paper concludes with lessons learned and implications for practice.
While evaluators have many intersecting identities, ethnicity remains the most salient identity for evaluators of colour. As formal graduate training in evaluation continues to expand, so too does ...the number of students from ethnic minoritized populations, who are in need of specialized mentoring and advising. Drawing from previous research on evaluation, higher education literature, and personal reflections from the author, an Afro-Latina faculty member, this practice note outlines five strategies for mentoring and advising evaluation graduate students of colour. These include considering the impact of vicarious trauma; assisting with the facilitation of peer and mentor “squads”; respecting, honouring, and celebrating students’ culture, religion, and families; being vigilant of microaggressions and practicing microvalidations; and developing mentoring competence. Each strategy is presented along with reflections and practical examples for implementation.
Violence, marginalization, oppression, exploitation, erasure, and injustice are cornerstones of the Black experience in the United States. Despite seemingly insurmountable challenges, Black people ...have worked diligently and competently to earn spaces within the present-day professional arena. While the experiences of Black professionals have been investigated in multiple fields, little is known about Black evaluators. This study aimed to investigate Black evaluators’ experiences in an attempt to understand the intersectionality of their identities, roles, and practice and to ensure voices and contributions of Black evaluators are highlighted. We conducted 26 interviews with Black evaluators across academic, government, nonprofit, philanthropic, and private sectors. We found race and advocate for social justice are central to Black evaluators’ identities, roles, and practice. We developed a thematic framework as a result of our analysis and interpretations of the ways in which Black evaluators’ identities impacted their perceptions of their professional role and practice.
More evaluators have anchored their work in equity-focused, culturally responsive, and social justice ideals. Although we have a sense of approaches that guide evaluators as to how they should attend ...to culture, diversity, equity, and inclusion (DEI), we have not yet established an empirical understanding of how evaluators measure DEI. In this article, we report an examination of how evaluators and principal investigators (PIs) funded by the National Science Foundation's Advanced Technological Education (ATE) program define and measure DEI within their projects. Evaluators gathered the most evidence related to diversity and less evidence related to equity and inclusion. On average, PIs’ projects engaged in activities designed to increase DEI, with the highest focus on diversity. We believe there continues to be room for improvement and implore the movement of engagement with these important topics from the margins to the center of our field's education, theory, and practice.
The Evaluation Journal of Australasia special issue on Values in Evaluation published in December 2019 gave an overview of current discourses associated with valuing in evaluation with the debut of ...'Praxis' papers. This introduction summarizes the three praxis papers in this second Values in Evaluation special issue. The praxis papers contribute to knowledge about the ways in which we can and should engage with values across a variety of evaluation contexts with varying stakeholders.
As the field of evaluation continues to expand, so do evaluation course offerings. However, little is known about the backgrounds of evaluation educators, how they go about planning their courses, ...challenges experienced, or how they attain competency as teachers. This study offers an initial understanding of novice evaluation educators’ experience in developing and teaching introduction to evaluation courses within a university setting. Factors that influence decisions around course content and pedagogical approaches are also examined. Data collection methods include dialogic conversation notes, reflective journals, course syllabi, and student evaluative feedback. Results indicate that perspectives on the purpose of evaluation and how evaluation should be taught are influenced by evaluation educator professional affiliation, experience with evaluation, and student audience. Implications for teaching and training in evaluation are discussed.
There has yet to be consensus within the field on whose and what criteria, or values, should be used to judge program worth. Furthermore, evaluators continue to grapple with decisions about which ...stakeholder values are to be included in evaluation, and if evaluators should be advocating or prescribing certain values or not. This introduction very briefly highlights the discourse associated with valuing in evaluation and summarizes the four articles in the Values in Evaluation Praxis section.
In this chapter, we reflect on the aims and gaps of this volume and the positionality of the co‐editors and contributors. We invite readers to consider how education and training for evaluators are ...or should be conceptualized and actualized, and detail future areas for consideration and attention.
This article considers the volume's intentions, process, and gaps while inviting attention to the impact and influence of CREA and Dr. Hood on evaluators and evaluation. Three of the co‐editors ...author this conclusion by revisiting the key questions and drawing connections between the positions and statements advocated by the contributing authors. The final editor offers final thoughts in the epilogue. CRE is a community, grassroots, and international movement; readers are invited to consider how integrity, courage, and action exist in their everyday evaluation practice.
Guest Editors’ notes LaVelle, John M.; Neubauer, Leah C.; Boyce, Ayesha S. ...
New directions for evaluation,
03/2023, Letnik:
2023, Številka:
177
Journal Article
