603-10 EVALUATION AS A FORMATIVE PROCESS

Community Learning Development Resource 603 – 10

 

EVALUATION AS A FORMATIVE PROCESS

 

Evaluation has at least two main purposes:

  1. To evidence the effectiveness and efficiency of our practice, performance and achievements in order to plan for and action improvements and developments in the outcomes for our clients. This is an internal quality improvement feedback loop.
  1. To demonstrate the effectiveness and efficiency of our practice, performance, achievements and cost for someone else – usually a stakeholder, strategic body or funder. This is an external accountability feedback loop.

 

Some reflective comments about the two purposes:

  • Is it better that we concentrate on doing the former and that we try to do this continuously – always seeking to learn from our practice and to improve? Is it also true that if we do this effectively our performance will naturally develop and improve?
  • Is it also true that by continuously seeking to evaluate for self-improvement (A FORMATIVE MODEL) we will also generate the evaluative evidence, which we can provide for enquiring funders or stakeholders? By having effective ongoing monitoring, evaluation and review  the evidence can also complies with the requirements of external stakeholders?
  • If we only evaluate when someone tells us to, because they want evaluative evidence of what we have done with their funding (A SUMMATIVE MODEL), surely this on its own is poor evaluative practice by us?

 

APPLYING THE FORMATIVE MODEL TO ANY EXPERIENCE, EVENT OR ACTIVITY

  • This requires us to be able to record, reflect and analyse:
    • What was planned?
    • What happened?
    • How and why did it happen?
    • How successful was it – input, process, output and outcome?
    • What can we learn from it?
    • How can we improve it?
    • Can we improve how we evaluate it?
    • Should we present this evidence of practice to a Stakeholder audience? (Summative Evaluation for us but hopefully formative for them!)

 

Evidence-based practice is about continuous formative evaluation prompting action planning for continuous improvement leading to more efficiently provided inputs, processes and outputs, and more effective outcomes.

 

EXAMPLE – Capacity Building Project workers plan to find out about other agencies and workers active in each of their target communities – and plan  after 6 weeks to evaluate their progress and achievements in this work.

 

Performance Indicator

What is to be achieved

Evidence

Of success

Performance Standard

Targets to be met
  • workers have an understanding of the work of key agencies in their target communities

 

  • effective links are made with relevant agency workers.

 
  • knowledge gained of the agencies, workers and the nature of their work
  • contacts recorded and arrangements made to meet and network
  • an analysis of the potential networking usefulness
  • Agencies identified will include those engaged in adult learning, health and community planning
  

 

  • Workers have met with relevant workers from the above named agencies.

 

Action planning (Formative evaluation outcomes) – examples:

–         More time is to be given to contacting agency workers and networking – progress to be reviewed in three months time

–         Planning to set up a collaborative community focussed networking meeting with public health workers – first meeting within six months

Summative evaluation outcome – examples:

–         Interim review report to Partnership group identifies the agencies and workers contacted by each Project worker by the end of the six week period

–         Numbers and variety of adult learning providers active in each target community are described in a report to the Partnership.

 

MULTIPLE ACCOUNTABILITY

  • We are accountable for our performance, practice and achievements, to our employers, colleagues, agencies and funders on the one hand and on the other hand, we are accountable to local people.
  • There are sometimes tensions for us and our work, between these two different lines of accountability.
  • We need to consider how formative and summative evaluative outcomes can contribute to effective accountability relationships with our agencies and funders, etc., and with local people.

 

PROMPTS FOR REFLECTION AND DISCUSSION

  • Which evaluative outcomes, formative or summative, are more likely to have a direct impact on improving our practice?
  • How might a summative report to a funding body have an impact on the quality of the experience our clients have with us?