Program Evaluation

Program Evaluation

Tashianna Avery, Betsy Fish, Ezohn Smith, and Crystal Stone

Ball State University

March 25, 2018

EDAC 634

Group Members	Roles	Commented On
Tashianna Avery	·   Evaluate feedback and write response ·   Complete table	Group 3-Self-Directed learning
Elizabeth Fish	·   Evaluate feedback and write response ·   Compile individual responses into one response	Group 3 - Self Directed Learning
Ezohn Smith	·   Evaluate feedback and write response
Crystal Stone	·   Group Leader ·   Design evaluation ·   Evaluate feedback and write response. ·   Compile all into one paper	Group 1 - Experiential Learning

Introduction

Our program is titled “Managing Conflict in the Workplace” and the course was designed using Narrative learning theory. The principle idea behind the course is that by sharing stories of various types of conflict that may arise in the workplace, the participants will learn to understand others in conflict situations and develop skills to better deal with the conflict. We reached out to two professionals to evaluate our program, Kismet Saglam and Adam Thocher.

Professional Evaluation

Link to program design paper: https://narrativelearninginadulteducation.blogspot.com/p/program-design.html

Evaluators:

The evaluators were chosen for their experience in education as well as their business acumen to evaluate the program. We chose these individuals because we felt that they would be able to give very well rounded feedback to address both the content and design of the program.

Evaluator 1 – Kismet Saglam, MS Ed, TESOL,

Vice President, Education Services, Kellen

Adjunct Professor, Harper College

Instructor, International TEFL Academy

Evaluator 2 – Adam Thocher, CAE, MBA
Sr. Account Executive, Kellen Company

Executive Director, American Association for Public Opinion Research

Link to full evaluator feedback: https://www.surveymonkey.com/results/SM-RHHVR6WZ8/browse/    

Student Responses

Thoughts on Evaluations

As a whole, our group was very appreciative of the feedback that we received on the program design. As this is a new process for all members of our group, it was good to hear from the evaluators that we were on the right track and very helpful to receive the honest and very analytical feedback. In examining the feedback provided by reviewers, our group was in agreement with the critiques and suggestions that Kismet Salgam and Adam Thocher provided. For example, both reviewers highlighted the need to improve the conflict resolution exercise with either a pre-survey and a post-assessment, or adding more recognition to the idea that mastery happens over time. Similarly, Kismet Salgam suggested that the introductory activity should focus on getting to know job roles. We believe this suggestion should be implemented because it provides facilitators with knowledge on what staff’s duties entail. Additionally, Adam Thocher’s review stated that although we gave our participants the tools to resolve conflict, that mastery of these skills happens over time. Thocher suggested that we implement messages within the program that encourage our participants after the program has completed. As a group, we felt that the feedback from our reviewers provided suggestions that can really improve our program as a whole.

Improvements

From the feedback provided, our group believes that there are several things we can improve within the program. First, we would like to include a pre-and-post test conflict assessment survey within the program. This will allow us to test the validity and growth of the conflict resolution program. To do this, our group would examine our conflict resolution assessment. Next, our group would build a pre-test conflict resolution survey. Finally, our group would look through and take both assessments to ensure that both measures reflect each other.

Second, we would like to create and include an introductory activity that would focus on getting to know job roles. In order to do this, first we will discuss what types of activities would both introduce our participants to the program but also help them gain knowledge regarding their job roles. For example, we might include an activity in which the participants narrate about what they think the role of the job they have been assigned is, pair with a partner and share their responses. This would allow for the participants to narrate their ideas, share and gain feedback from a peer. Finally, we would implement this into the program.

Third, we implement messages within the program that encourage our participants after the program has completed in order for participants to work towards mastery. We would like to do this in three different ways. First, we implement a follow-up training session or make the training two days instead of one full day in order to conduct a pre and post training survey as well as provide time for reflection and ways to gage mastery. Secondly, we would like to implement messages by posting signs around the room that make references to learning. We would meet to discuss what messages we would like to use and consult academic literature on the subject. Next, we would create the signs.  An example would be: “Learning is not a race, Move at your own pace.” Finally, we will add a reflective narrative essay component that would allow participants to narrate their experiences outside of journaling and to also allow them to reflect more on themselves and what they learned from the program. First, our group would identify the best place within the program to add this component. Next, our group would write up a prompt for the reflexive narrative essay. Finally, our group would add this component to the program and get additional feedback from reviewers. We believe these improvements will help our program become more effective and useful to our participants.

Lessons Learned

Throughout the stages of the project so far the most important lesson we have learned is that communication is essential. At the beginning of the group formation, we chose to meet through a series of Zoom conference calls and established a schedule for calls to touch base through the remaining stages of the project. Our group has also remained very active in keeping each other up to date and asking questions through email. The ability to reach out and receive a response quickly has been essential. One area we have learned from though is that when asking for feedback from others, it is important to ensure that the respondents know that we are looking for critical comments as well. One of our respondents was very thoughtful and detailed, providing a great critical feedback to help us improve. The other evaluator, however, provided only minimal feedback with a lot of positive comments. If doing this again we would make sure to include in the directions that we are looking for feedback specifically designed to improve the program so, while positive comments are appreciated, critical comments are very valuable. As a group we appreciated the time and effort that the evaluators took in reviewing our program. While we may not have agreed with all of the comments and feedback, receiving it helped us to consider alternatives and ways our program could be improved. 

Tables

	Evaluators	Ideas for Improving  program design	Revisions/Your responses
1	Kismet Salgam	Intros should focus on getting to know job roles Take conflict assessment afterwards do a pre-course survey instead Ask more open-ended questions Consider changing role play for those reluctant to do so Reflection could be pre-course work 60 second commercial activity	Change the speed dating intro questions to focus on the job roles of the attendees Make the conflict assessment using a tool such as Survey Monkey to track submissions before the training Review all questions asked making them more open-ended or provide an answer scale There are lessons learned when someone is outside of their comfort zone. The role play can take place in small groups. We would not change this exercise. We would not change the reflection time to pre-course work if the assessment is changed to pre-course work. There is value in reflection and discussion. The point of the 60 second commercial is to see if conflict arises among the groups. All groups may not have conflict, but this would depend on the personalities in the group. We believe that this is another good discussion piece, but it can be removed if necessary.
2	Adam Thocher	Recognizing that mastery of conflict resolution happens over time	Adding in a chart and discussing ways to build habits-reminding attendees that it takes time and not to be too hard on themselves