Elements of a Program Evaluation



What Is Program Evaluation?

By definition, program evaluation is "the systematic application of social research procedures for assessing the conceptualization, design, implementation, and utility of social intervention programs." In practice, program evaluation plays a significant role in program development and assessment. From concept to planning, application to results, the systematic evaluation of each step of a program will serve as a mechanism to developing a realistic program that is clear, comprehensive and measurable. A solid program evaluation will also expedite the dissemination and publication process. Some key areas where incorporating program evaluation methods could be beneficial include:

TOP


Developing An Evaluation Plan

Program development is labor intensive and requires plotting a strategy that will streamline the process. Below is a checklist of questions to ask when formulating a model of the program and establishing an evaluation plan.

Clarify Goals and Objectives
Create a Model of Your Program
Formulate Evaluation Questions
Determine What Type of Evaluation You Want to Conduct
Choose Data Collection Method(s)
Analyze Data
Reporting Findings

TOP


Evaluation Types

There are three types of program evaluation: process, outcome and impact. Each type provides different information. In order to choose what type of evaluation would be most appropriate, you must determine what questions need to answered by the program. Below are explanations of each type of evaluation as well as the question(s) that each should answer.

It is common to use a combination of evaluation types for a program. Process and outcome evaluations are frequently used to measure the effects of a program, however, only a few programs include impact evaluations. If you are uncertain about what type of evaluation you should use, two questions to ask are:

TOP


Selecting A Data Collection Method(s)

Once goals and objectives have been established and the right evaluation questions have been formulated, a method of collecting data must be chosen that will best answer the questions.

Choosing the appropriate data collection tool(s) will be crucial to determining the effectiveness of the program. Data collection tools must be designed to obtain the information required by the evaluation questions. Before deciding which method of data collection to use, the following questions must be answered:

Data Collection Tools

TOP


Analyzing Data

Generally, data analysis techniques will be determined by the evaluation questions and the methods used to collect data. Data analysis can be as simple as calculating percentages or as complex as performing a time series or regression analysis. However, be aware that more complex analysis usually requires assistance from someone who has expertise in data analysis. Because specific aspects of analysis are extremely complex, only three basic types of data analysis are represented below.

The general rule is to begin with the least complex analyses and work toward the most complex analysis technique possible with the available expertise.

TOP


Reporting Evaluation Findings

Once data has been analyzed and interpreted, a report will need to be prepared for purposes of disseminating and publishing program results. Please remember that finding no change among program participants is as important as finding changes and both should be reported.

The standard framework to writing a good evaluation report is:

Evaluation reports should be written so that they are easily understandable to both lay people and professionals, and formatted in a logical, attractive manner. Use the following guidelines when writing the program final report:

TOP


Evaluation Guidelines

  1. Know the stakeholder that will use the results of your evaluation.
  2. Always be aware of the resources available to you.
  3. Design the evaluation to ask the right questions.
  4. Report results and findings in a clear and accurate manner.
  5. Keep your evaluation focused.
  6. When necessary, seek assistance from individuals with the related expertise that is needed.
  7. Develop a plan for documenting and maintaining all evaluation activities and data.
  8. A well done process evaluation provides valuable information, and may be more appropriate for the scope of your program than an outcome or impact assessment.

TOP


Bibliography

Program Evaluation/Research
Sampling
Questionnaire/Survey Development
Validity & Reliability
Statistics

TOP