Evaluation Research

Evaluation and Social Research

Evaluation utilizes many of the same methodologies used in traditional social research, but because evaluation takes place within a political and organizational context, it requires group skills, management ability, political dexterity, sensitivity to multiple stakeholders and other skills that social research in general does not rely on as much. Here we introduce the idea of evaluation and some of the major terms and issues in the field.

Definitions of Evaluation

Evaluation is the systematic assessment of the worth or merit of some object

This definition is hardly perfect. There are many types of evaluations that do not necessarily result in an assessment of worth or merit -- descriptive studies, implementation analyses, and formative evaluations, to name a few. Better perhaps is a definition that emphasizes the information-processing and feedback functions of evaluation. For instance, one might say:

Evaluation is the systematic acquisition and assessment of information to provide useful feedback about some object


Both definitions agree that evaluation is a systematic endeavor and both use the deliberately ambiguous term 'object' which could refer to a program, policy, technology, person, need, activity, and so on. The latter definition emphasizes acquiring and assessing information rather than assessing worth or merit because all evaluation work involves collecting and sifting through data, making judgements about the validity of the information and of inferences we derive from it, whether or not an assessment of worth or merit results.

The Goals of Evaluation


Most evaluations provide feedback that can be used for decision-making or policy formulation through the provision of empirically-driven feedback. Qualitative input by stakeholders is increasingly becoming very important.

Evaluation Strategies

'Evaluation strategies' encompass four major groups of evaluation:

Scientific-experimental models 

·         historically dominant and take their values and methods from the sciences

·         prioritize on the desirability of impartiality, accuracy, objectivity and the validity of the information generated

·         quasi-experimental designs

·         objectives-based research that comes from education

·         econometrically-oriented perspectives including cost-effectiveness and cost-benefit analysis, and;

·         theory-driven evaluation (Holistic assessment: Taking contextual factors and. causal mechanisms into consideration in assessment. Program theory: stakeholders' implicit and explicit assumptions on what actions are required to solve a problem and why the problem will respond to the actions (Chen, 2005).

Management-oriented systems models

·         PERT = the Program Evaluation and Review Technique

·         CPM = the Critical Path Method

·         The Logical Framework or "Logframe" model developed at U.S. Agency for International Development

·         UTOS model where U stands for Units, T for Treatments, O for Observing Observations and S for Settings; and the CIPP model where the C stands for Context, the I for Input, the first P for Process and the second P for Product.

These management-oriented systems models emphasize comprehensiveness in evaluation, placing evaluation within a larger framework of organizational activities.

Qualitative/ anthropological models

·         Emphasize the importance of observation, the need to retain the phenomenological quality of the evaluation context, and the value of subjective human interpretation in the evaluation process

o   naturalistic or 'Fourth Generation' evaluation;

o   the various qualitative schools = critical theory and art criticism approaches; and,

o    the 'grounded theory' approach of Glaser and Strauss

Participant-oriented models

·         emphasize the central importance of the evaluation participants, especially clients and users of the program or technology. Client-centered and stakeholder approaches are examples of participant-oriented models, as are consumer-oriented evaluation systems.

There is no inherent incompatibility between these broad strategies as each of them brings something valuable to the evaluation table. In fact, in recent years attention has increasingly turned to how one might integrate results from evaluations that use different strategies, carried out from different perspectives, and using different methods.

Types of Evaluation

There are many different types of evaluations depending on the object being evaluated and the purpose of the evaluation. Perhaps the most important basic distinction in evaluation types is that between formative and summative evaluation. Formative evaluations strengthen or improve the object being evaluated -- they help form it by examining the delivery of the program or technology, the quality of its implementation, and the assessment of the organizational context, personnel, procedures, inputs, and so on. Summative evaluations, in contrast, examine the effects or outcomes of some object -- they summarize it by describing what happens subsequent to delivery of the program or technology; assessing whether the object can be said to have caused the outcome; determining the overall impact of the causal factor beyond only the immediate target outcomes; and, estimating the relative costs associated with the object.

Formative evaluation includes several evaluation types:

• needs assessment determines who needs the program, how great the need is, and what might work to meet the need
• evaluability assessment determines whether an evaluation is feasible and how stakeholders can help shape its usefulness
• structured conceptualization helps stakeholders define the program or technology, the target population, and the possible outcomes
• implementation evaluation monitors the fidelity of the program or technology delivery
• process evaluation investigates the process of delivering the program or technology, including alternative delivery procedures

Summative evaluation can also be subdivided:

• outcome evaluations investigate whether the program or technology caused demonstrable effects on specifically defined target outcomes
• impact evaluation is broader and assesses the overall or net effects -- intended or unintended -- of the program or technology as a whole
• cost-effectiveness and cost-benefit analysis address questions of efficiency by standardizing outcomes in terms of their dollar costs and values
• secondary analysis reexamines existing data to address new questions or use methods not previously employed
• meta-analysis integrates the outcome estimates from multiple studies to arrive at an overall or summary judgement on an evaluation question