Program Evaluation in Adult Learning Environments

A Little About this Project

Although this case study focused on my role and the product my company was selling, the key components of evaluation remain the same. As you read through this article, consider how you currently evaluate success in your role and in your company. How does it compare to what is discussed below? How robust is the evaluation process at your company? Is there any room for improvement? Ready, let’s start!

Evaluation – Defining Anticipated Outcomes

The evaluation is possibly the most critical aspect of running a program. Because not all problems can be adequately addressed while a program is running, it is imperative that an evaluation process is built into the creation of any program, regardless of the length. This allows you to gather participant data that’s outside the scope of personal success and instead gather data on other equally important components of what makes a program successful. Soliciting this information during or after a course allows program stakeholders to revisit their initial assumptions about what the course or program should accomplish and what it takes to reach those goals. By committing to the creation and implementation of an evaluation plan, you are committing to the continued success of your program.

Philosophy of Evaluation

Whereas assessments should focus on the role of the instructors in ensuring student success, the evaluation should be thought of in terms of gauging program health. The in-class assessments let us know if participants are grasping content and if not, what we can do to adjust our approach. There are other aspects, such as whether the program meets participant expectations, that are not captured through project rubrics or other classroom assessment techniques. This is where the concept of evaluation comes in.

Once complete, it’s important to review the program in its entirety to discover whether the program has served its purpose, whether there are areas of improvement that can be addressed prior to the start of the next program run and whether the program, in its current iteration, provides value to stakeholders. To do this, evaluations need to be:

  • Objective: You, the instructional designer, facilitator or other stakeholder, have invested a substantial amount of time in developing this program. It may be hard to hear that it’s not working the way it’s supposed to. It’s important that fear not keep you from asking those hard, direct and neutral (non-leading) questions.
  • Actionable:Regardless of the way it is collected, evaluation data must be actionable. At times, we may fall into the trap of asking participants completely subjective questions that do not have clear action items associated. Conversely, asking objective questions that only allow for a yes or no may not give you any useful data around suggested improvements.
  • Targeted: Know what you’re evaluating ahead of time and why. While you may be tempted to ask participants how they felt about the location of the school, unless you have the ability to change that, it may be a waste of time – yours and the participant’s – to include it. Questions related to program goals, instructional quality, course content and overall satisfaction all fall under the category of things your stakeholders will want to know. A rule of thumb is to determine key health metrics at the outset of the program and ask questions that speak to those items.

Which aspects of the program are being evaluated and how?

As discussed, what you intend to evaluate should be established prior to the start of the program. This is especially crucial if this is a pilot, as it is likely that there will be areas requiring immediate refinement. For this course, four core areas were identified. In order to determine whether this course was effective, we will evaluate whether participants:

  • Are learning what we expect them to learn
  • Believe the content is delivered in a way that is accessible and easy to understand
  • Can make clear connections between the content they are learning and the content’s relevance to their everyday lives and,
  • Would recommend this program, including the setup, content and instructors, in the future

Feedback will be collected via exit tickets (surveys), administered by the instructor at the end of each session. These surveys will consist of four questions. The exit tickets are meant to be a section by section snapshots evaluating the immediate impact of the course. At the completion of the course, a more comprehensive end-of-course survey will be administered.

Sample Evaluation #1 – Exit Tickets

Purpose Evaluate the quality of content and instruction at the end of each session
Deliverable Survey (hosted in Google docs) – the same form is used throughout the course, differentiated by the submission date.
  1. In one sentence, summarize the most important concept you learned today (Open text)
  2. The material covered during this session is immediately applicable to my job (Likert scale)
  3. The material covered during this session was presented in a clear and approachable manner (Likert scale)
  4. Is there anything else you’d like to add? (Open text)

Sample Evaluation #2 – End of Course Survey

Purpose Evaluated the quality of content and instruction of the overall program
Deliverable Survey (hosted in Google docs)
Questions Overall Experience

  1. Would you recommend this course to a friend of colleague? (NPS)
  2. What is the most important reason you gave us that score?


  1. The material covered throughout this course met my expectations (Likert Scale)
  2. The material covered throughout this course was relevant to my job (Likert Scale)
  3. The material covered throughout this course is immediately applicable (Likert Scale)
  4. I understood what the requirements were for all of my assignments (Likert Scale)
  5. Is there anything you’d like us to know about the content? (Open text)


  1. The instructor presented in a way that made the material easy to understand (Likert Scale)
  2. The instructor provided meaningful feedback (Likert Scale)
  3. I had the tools and resources I needed to participate in class and complete assignments (Likert Scale)
  4. Is there anything you’d like us know about the delivery (Open text)


What happens to the data?

During the Course

During the course, data collected from the exit tickets will evaluate whether the content is relevant and immediately applicable for participants. In addition, it will evaluate delivery techniques and allow program facilitators to provide supplemental materials in areas where the content is lacking. In the event that the issue lies with instruction, alternative methods can be explored, including but not limited to the introduction of blended or self-directed learning tools, additional AV equipment or even a change of venue. The purpose of the evaluations at the end of session and end-of-course is to cover as many modifiable external components as possible and establish a plan of action to be implemented during the current cohort or for the next.

After the Course

All data will be aggregated in an Excel file and used to create a dashboard that identifies trends, weaknesses and strengths of the program. The exit tickets and end-of-course survey responses will be coded and categorized into Content, Delivery, Tools and Environment and then into stages based on the length of time it will take to implement. Quantitative and qualitative data will be used to drive decisions around revising content and/or delivery methods and whether any additional financial investments will need to be made for tools or program development. Finally, data collected will inform what kind of training, if any, would benefit the instructor based on feedback around delivery. A timeline will be established for all improvements and the changes will be communicated to stakeholders, as well as future and former participants.


Using Inter-team Collaborations to Promote Critical Thinking Skills

This is part of a 3-part series focusing on applying adult learning theory in the workplace. To see the other articles, view A Brief Intro to Adult Learning Theory and Self Directed Learning as a Training Solution

Current Problem

Teams across 15 campuses are finding it increasingly difficult to track information and share it with the stakeholders. In addition, there is a lack of standardization in the way programs are run, further complicating alignment and adoption of initiatives to solve this problem.

Proposed Solution

Many of these conversations have started with ‘we can’t’ or ‘we don’t’. We can’t change this system or we don’t use this method. I usually ask probing questions to get to the bottom of these opinions. Is the ‘can’t’ related to something tangible? Is the ‘don’t’ due to a lack of structure or something else? Until a problem can be broken down into it’s smallest parts, a holistic fix cannot be created. Asking teams to think critically about their current processes will force all parties to honestly evaluate the problems they face. I highly recommend using this list of Socratic Questions to get past the surface issues. Much like the worksheet found at the bottom of this page, you can create an evaluation worksheet asking similar guiding questions to reach a conclusion.

Why Promote Critical Thinking?

Human beings draw conclusions from past experience – personal or otherwise. We can become entrenched in views because we have had positive and negative experiences that influence what we think will happen. Perhaps we’ve were involved in an unpleasant outcome related to group work that prevents us from trying something new. Conversely, maybe we heard of a new technology and we want in on it because everyone else is raving about it. Either way, it’s necessary to break down and evaluate this feelings or instincts in order to make logical and well informed decisions. Even if they result is less than stellar, you have begun a process of evaluation that allows you to continue build until you reach success.

What Does it Look Like?

Start by creating a list of specific issues that your team is trying to tackle. If you are a manager, create your own and allow teams to do the same. For each issue, have team members brainstorm about the root causes and 2-3 possible solutions.  You can have participants fill out the worksheet to the right, use the template at the bottom of this page, or create your own.

It is unlikely that everyone will have the same answers, or answers that get to the bottom of the problem. Ask team members to share their view of the problem then use Socratic Questions to whittle it down to it’s simplest form. This might look something like this:

Stated Problem: There aren’t enough people or resources available to complete this project

Follow Up Questions

  • Can you give me an example of a time when this was apparent?
  • Can you describe the scope of the project? In what ways can we leverage the strengths of you and your team members to solve this problem?
  • How can we look at this another way?
Do the same for each section of the worksheet until there is a clear idea of what actionable steps can be taken to begin to solve the issue.

Once that has happened, have teams pair off with members of other teams to get feedback on how they handled similar situations. The idea is to have fresh perspectives challenge existing perceptions and require the problem solvers to re-evaluate, explain and if necessary, defend their beliefs and next course of action.

Will this work for me?

  • There is a question or issue that requires deeper exploration in order to resolve
  • You are looking to engage your audience and get them involved through intellectual contributions
  • You  need to change long held thoughts or processes but are facing resistance (change management)
  • You want to empower your audience to go beyond surface knowledge and use their skills and experience to develop their own ideas

Want to try it out? Use this PDF template to build a framework for applying any adult learning strategy to your current work.

Self Directed Learning as a Training Solution

This is part of a 3-part series focusing on applying adult learning theory in the workplace. To see the other articles, view A Brief Intro to Adult Learning Theory and Using Inter-team Collaborations to Promote Critical Thinking Skills

Current Problem

Where I currently work, we are finding that our instructors need and crave more targeted professional development. As mentioned in the Methodology section, adults want to know that what they are learning is relevant and has immediate application. If there is something that an instructor needs to know to do his/her job, then it is our responsibility to provide it. As a relatively new company, our resource library is limited and does not necessarily address the different skill gaps of our instructors, nor the levels at which they need to address them. In other words, we aren’t practicing differentiated learning. We are taking a one size fits all approach that leaves some instructors feeling lost and others feeling stagnant.

Proposed Solution

At the very simplest – build a searchable repository of professional development resources and pair this with the observations and assessments that are already happening. Ensure that all instructors across all campuses have access to this database and leverage instructor and employee knowledge to further populate it.

It’s extremely important that the resources you provide are accessible, relevant and accurate. Forgetting these simple guidelines can result in frustration (why do I need to learn this?), confusion (what does this mean/this contradicts something else), and resentment (I have to pay/drive an hour out of my way for this information) from learners.

Why Self Directed Learning?

You’re covering a few bases with this approach, most of which are covered in other parts of this site. Above all else, you are providing adults with actionable and applicable items. They can pursue as little or as much as they want to in order to meet the benchmarks set forth by their managers. They can also choose to expand their knowledge outside of the provided resources and are encouraged to improve their in class practice with the support of their instructional leaders.

In a classroom setting, this can be a useful tool for students who often finish early and crave additional knowledge. Conversely, it’s also a great tool for a student who is having trouble grasping a concept that everyone else has mastered. The missing information may be critical to the student’s success, so it cannot be glanced over. Instead, you can provide articles, textbooks, videos or other resources to allow the student to learn the concept on his/her own.

What it Looks Like

For me, this looks like a centralized web-based system that is managed by the instructional leaders across all campuses. Because this will be used by all instructors and is part of the message and culture of the company, it’s important that the information submitted is vetted and that it aligns with what you want to see in the classroom.

For you, this depends on the systems and process already in place for learning. Do you have a learning management system (LMS) like Canvas, Blackboard, Oracle or Schoology? If you answered yes, there might already be add-ons or functionality that allow you to build resource libraries. You may also be able to create videos, quizzes and other assessments that are specific to the goals of your course.

If you answered no, you can still create a system that ticks those three boxes. If you are working in an offline environment, have a listed of links, books, videos, etc to provide to students. Or, if you have books or articles in the classroom, suggest the student has a look during the break. You can also build a simple website, like this one, to act as an extension of the classroom materials.

How can I make this work for me?

SDL is best used when:

  • There are small or simple tasks the audience needs to know before you can move forward. This is sometimes called pre-work and has been used in as a requirement for trainings and classes.
  • A learner is struggling during class but you are unable to a)spend anymore time on the subject, b) provide the depth of knowledge necessary for understanding or c) adjust the lesson to address his/her preferred learning style.
  • A learner is excelling and requires more stimulus to remain engaged in the lesson.

Want to give it a shot? Download this Adult Learning Application Cheatsheet to get started.

Finding the Ideal Solution: 3 Approaches to Problem Solving

I may have mentioned taking a few courses through So far, I’ve found the Problem Solving 101 course invaluable as it explores the topic in depth, including the identification and allocation of resources, objective and subjective input, and evaluation of outcomes. Moreover, introduces three main categories for to consider when faced with problems in the workplace (or everyday life).

First Approach – Pretty Straightforward

The first category is referred to as the “stop it and mop it” scenario. In this situation, there is an event, behavior or condition that you need to stop from happening, as well as clean up the existing damage. Imagine that you are a landlord and your tenants are complaining about mice eating through the walls. To stop this, you hire an exterminator to find out where the mice are coming from and place baits or traps to catch them. To ‘mop it’, or clean it up, you have the exterminator locate likely areas existing mice might nest and place baits there as well. You also hire a contractor to come in and fix all the places the mice have chewed through. With these items in place, the problem should be resolved.

Second Approach – Some Concessions Are Made

The next situation employs the “current reality vs ideal” scenario. In this case, what is happening in the now does not match up with predetermined expectations or goals. To demonstrate, we’ll use student enrollment as an example. You currently have 250 students enrolled in your school but the Department of Education has budgeted you for 300 students. This means you are 50 students short of the goal. There are two ways to approach the issue and the one you chose depends heavily on which holds the most acceptable outcome:

  • Option 1: Adjust the goal. Is it unlikely that you’ll get another 50 students? Is it possible to increase by 25 students instead? How will this affect your goals and budgets for the remainder of the school year? What about next year?
  • Option 2: Attempt to meet the goal. While this seems like the obvious first choice, it may not be possible. In this example, your pool of students is inherently limited by things like zoning, number of students eligible, class sizes and competing schools. This means that even if you recruit all remaining students, you may still fall short. However, this option might be easier to implement when dealing with products or services, as you can increase production, hire more staff, adjust hours, etc.

Third Approach – Hard work for Lasting Payoff

The third approach to problem solving is the “opportunity for change” scenario. This is best applied to a situation where the problem is something with room for improvement. Perhaps you supervise a call center. You discover that employees spend an average of 5 minutes searching the company database for answers to commonly asked questions. While this might not directly translate into dollars lost, it does affect how many customers can be served. It can also have a negative impact on the way customers view your company. After all, how many of us have spent hours on the phone with the cable company trying to do something as simple as reset a router? This sort of issue, under the right management, can be turned into a learning experience for everyone. Determining whether staff needs better resources, guidance or training can improve or eliminate the problem altogether.

To summarize, there are three main ways to categorize and resolve problems. At the most basic level, each categorization requires identifying what the problem is and then determining what the anticipated outcome is. Approaching adverse situations in this manner can help you structure processes and procedures to avoid similar issues in the future.