Who can use the tool?
What resources are needed?
Development, ownership and support
Third sector examples
Further sources of information
Prove It! was developed by The New Economics Foundation in partnership with Groundwork UK and Barclays PLC to provide a method for measuring the effect of community regeneration projects on the quality of life of local people. This tool was originally conceived as a handbook to help those managing neighbourhood renewal projects look beyond the physical and environmental changes that had taken place (e.g. the number of trees planted; amenities created) and be able to highlight the positive outcomes of regeneration that can often go unnoticed. Prove It! is best suited:
- To helping understand the effects of small or medium-sized projects, (as opposed to larger-scale regeneration programmes).
- To looking at the effects of projects that involve local people as workers, volunteers or beneficiaries.
- To organisations concerned with local community involvement.
- To those interested in evaluating a project’s effect on social exclusion and other quality of life issues.
Prove it! seeks to make data collection itself part of the process of regeneration, with local people involved in a project’s evaluation as well as its delivery. Prove it! was designed to be manageable and possible to use within the limited resources that small-to-medium–scale projects have available to them to ensure that evaluation becomes part of the culture of an organisation, rather than a burden. The evaluation process itself can contribute positively to the desired outcomes of the project.
As a way to assimilate some of the principles of effective impact evaluation practice into a project’s day-to-day running, a Prove it! Toolkit was developed in 2004 and updated in 2009. This is made up of a series of MS Word and Excel files containing instructions for running participative workshops, designing simple questionnaires and inputting data that allow a thorough and robust exploration of a project’s impact. The aim is to make it easier for organisations to take those first steps towards undertaking effective impact measurement.
The Prove it! (2000) handbook describes the process of involving communities in agreeing on the most important issues, deciding on indicators and collecting data. It also provides the rationale for this type of participative evaluation and community engagement. The complementary version of Prove it! represented by the elements of the Prove it! Toolkit (2004) described below, brings together much of this into a series of documents describing activities and illustrating templates and spreadsheets which simplify the most important stages for those who may not have the time and resources to undertake the more in-depth approach.
The Prove it! Toolkit incorporates three main tools:
- A Storyboard exercise for understanding how a project’s intended activities will lead to change.
- A Survey Questionnaire to be used at the start and end of the project.
- A Poster Evaluation exercise in order to reflect at the end of a project on its impacts and the lessons that have been learnt.
In addition there are notes, guidance and templates provided to assist project managers in planning the evaluation process and presenting findings.
The Prove It! handbook describes the process of involving communities in agreeing on the most important issues, deciding on indicators and collecting data. It also provides the rationale for this type of participative evaluation and community engagement. The kit described below condenses much of this into a series of documents describing activities and illustrating templates, including spreadsheets, which simplify the important steps for those who may not have the time and resources to undertake the more in-depth version.
- The Storyboard provides a focus for talking to local people before the project has taken place. This can inform the organisation’s thinking on the whole project. A group of people involved with the project or affected by it are invited to discuss eight questions that together describe the hypothesis, or ‘story’ about how they think the project will make a difference. Once the hypothesis has been established people involved (or potentially benefiting from the project) are better able to choose the indicators that will demonstrate whether or not the project has made a difference. When the storyboard is used during and/or after the project it provides a way for local people to be involved in the evaluation as well as the delivery of an initiative, project, or activity.
- The Prove It! toolkit provides a Survey Questionnaire that focuses on capturing the impacts of the project on people and their communities. This contains an MS Excel file of ready-made questions for the survey that can be adapted for use by any project. There is some flexibility in the survey if desired, with the option for organisations to add some questions. The toolkit also gives guidance as to whom to survey and on issues involved in conducting interviews with local people. Once the first round of surveys has been completed (ideally before the project has been implemented), the toolkit provides a series of ready made data-entry sheets directly linked to the Questionnaire, which automatically convert the responses into graphs that can be cut and pasted into other documents for analysis and presentation.
- The Poster Evaluation exercise is a ready-made workshop that offers people who have been involved or affected by the project, the opportunity to look back over the project, describe the impacts it has had, and to highlight the lessons learnt. It is based on using a large, interactive poster with a timeline upon which participants attach self-adhesive notes identifying the high points and low points of the project’s history. The exercise is designed to acknowledge unexpected consequences of the project as well as intended outcomes. A facilitator uses a set of instructions to guide people through a series of stages, each focusing on different aspects of the project’s outputs and outcomes. This part of the toolkit is based on another evaluation tool developed by nef in partnership with the Shell Better Britain Campaign, which is known as Look Back Move Forward or LBMF.
- This tool can help organisations answer the questions:
“How can we go about measuring and documenting the ‘outcomes’ of our work?”
“How can we measure the ways in which communities’ and individuals’ quality of life is changing as a result of our endeavours?”
- Measuring social outcomes can help to demonstrate the full value of regeneration or other local improvement activities to external bodies, such as funders. There is widespread recognition among policy-makers that evaluation often fails to involve local people, which Prove It! effectively addresses.
- Prove It! can provide a comprehensive story or hypothesis for a project. With a strong hypothesis established at the start, a better case can be made at the end that a particular intervention has brought about the changes.
- The tool is relatively simple to use and has some scope for flexibility and adaptability to an organisation’s specific needs and can be used for a wide variety of projects, not just regeneration.
- Its participative process can help involve local people and help contribute to the building of trust in the community so that the collection of data becomes part of the process of regeneration itself. This can help to build capacity of local groups and people and galvanise further action in other areas.
- Prove it! may be useful as a complement to other evaluation tools. Where the Prove it! questionnaire is not sufficient or appropriate for collecting data, other methods should be sought to demonstrate change and distance travelled.
- As with all participative forms of evaluation, there is a need for caution in labelling certain people or groups of people as ‘local’ or representative of ‘the community’. There is potential for the exclusion of voices of groups or individuals in the local community as well as the potential for local pressure groups dominating the evaluation.
- A participative process may bias the answers of participants – people may tell you what they think you want to hear.
- Prove It! works best if there is a confident member of staff with some experience with working in a participative way with the community and stakeholders involved.
- Prove It! seeks to measure only the effects of a particular project or initiative. It may not identify activities within other people’s lives that can have an effect on social capital and quality of life in a local area.
Who can use Prove It!?
Any third sector organisation working on projects and initiatives involving a local community can use Prove it! It can meet the needs of regeneration and community development organisations including Development Trusts.
It works best with smaller projects and less well for large projects or those with no community involvement.
Prove It! may also be useful for project officers from agencies, local government and other decision makers who are likely to commission evaluations of specific initiatives and use the results to inform policy decisions.
What resources are needed?
Prove it! requires leadership from someone who will plan and manage its use from start to finish. This role should naturally fall to a project’s manager or someone within the organisation who can be the ‘evaluation champion’. Because evaluation is necessarily a long-term, ongoing process, the champion needs to be able to take responsibility for keeping track of the evaluation with enough information on file so that someone else can continue the process if they have to move on.
Of the three sections of the Prove it! Toolkit described above, the poster evaluation exercise is the only part that recommends the involvement of an outside facilitator to lead the session. This provides an opportunity for someone not directly involved in the project to review whether the findings from the evaluation make sense.
Proficiencies or skills
No specialist skills are required to use the tool, but it is helpful if the evaluation champion has previous formal or informal experience in collecting and presenting data for monitoring and evaluation, and an understanding of the processes and the problems associated with interviewing. Knowledge of or a background in involving people in participatory processes would also be useful in deciding how to facilitate the participative elements of the tool.
Prove it! has been designed to be integral to a project as opposed to something that is undertaken after it has been completed. To avoid any extra burden to their workloads this must take place as part of existing project activities. Because of the emphasis on self-appraisal (without necessarily relying on outside consultants) many of the data collection activities need to be undertaken by staff within the project.
It is vital that at the start of a project (and therefore at the start of its evaluation) the evaluation champion agrees with staff, volunteers or partners the extent to which they will be involved in the evaluation process. An evaluation planning template is included in the materials to help guide this discussion. There is no hard and fast rule as to how much time to spend on evaluation, but best practice suggests using 10 per cent of the initiative’s or project’s budget or time.
Sufficient staff time is needed to manage the involvement of a project’s beneficiaries in various aspects of the evaluation process, for example in the distribution and collection of questionnaires, or in undertaking a storyboard and poster evaluation session. The original Prove it! handbook (2000) describes in more detail how to involve local people in shaping and taking part in other aspects of an evaluation.
Courses, support, and information
The Prove It! handbook provides useful support and information. nef may be able to provide limited telephone and email support if staff are available. The handbook can also be used by evaluators or outside consultants to guide an assisted process.
Development, ownership and support
The Prove it! (2000) handbook has been designed to provide information, support and guidance.for undertaking a participatory evaluation. Likewise the Toolkit is a stand alone set of documents that can be used as the template for an outcomes evaluation. However, if training or additional assistance is required, this can be obtained from nef by emailing email@example.com
Third sector examples
Some organisations that have used Prove It!:
- Groundwork UK
- British Trust for Conservation Volunteers
- British Waterways
- Lea Rivers Trust
- Brinnington Community First SRB (Liverpool)
Further sources of information
The complete Prove it! Toolkit can be downloaded here for free.
The handbook, Prove it! Measuring the effect of neighbourhood renewal on local people, can be downloaded for free from the Prove It! website.
The handbook also includes appendices with further information on indicators, surveys and social capital, of which there is also a rich and varied literature in the public domain.
“Prove It! turns up different answers, exploration of softer outcomes and a chance for discussion.”
“It assists – it’s not a burden. It isn’t dry either – it helps you to be imaginative about the project.”
“Prove It! Works best with projects that have a clear aim, a clear start and a clear finish. Harder with bigger projects.”
It’s good for longer-term projects because we can track progress over time”
* All quotes are taken from ‘Prove it! Its development and its potential for evaluating community-based regeneration projects’, nef, published 2004.
|Previous: PQASSO||Back to Contents||Next: Quality First|