guide:implementation:evaluation
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision Last revision Both sides next revision | ||
|
guide:implementation:evaluation [2018/10/23 11:44] Camille [Are we reaching our objectives ?] |
guide:implementation:evaluation [2018/11/02 18:05] Caroline [A Matrix to stay the course] |
||
|---|---|---|---|
| Line 2: | Line 2: | ||
| ===== The evaluation process ===== | ===== The evaluation process ===== | ||
| - | During the implementation of a Heterotopia Tour you have to monitor and evaluate different aspects of the project. Frequent monitoring allows you to prevent risks, to manage problems in time, to ensure quality, and to anticipate possible situations that can influence your project. | + | During the implementation of a Heterotopia Tour you have to monitor and evaluate different aspects of the project. Frequent monitoring allows you to **prevent risks**, to **manage problems in time**, to **ensure quality**, and to **anticipate** possible situations that can influence your project. |
| General monitoring and evaluation plan can include: | General monitoring and evaluation plan can include: | ||
| Line 11: | Line 11: | ||
| | Management – budget use\\ Assess:\\ - Budget efficiency (quality-price value)\\ - Fair budget allocation (did any partner had justifiable over-costs that should be covered? )\\ - Budget efficiency | Monthly\\ Getting closer to the start of the Tour: more frequently\\ During the Tour: daily | - Budget plan divided in two parts: original budget plan and real budget\\ - Reporting on budget use, with financial documents to verify the costs\\ - Online partner meetings\\ - Coordination staff meetings | | | Management – budget use\\ Assess:\\ - Budget efficiency (quality-price value)\\ - Fair budget allocation (did any partner had justifiable over-costs that should be covered? )\\ - Budget efficiency | Monthly\\ Getting closer to the start of the Tour: more frequently\\ During the Tour: daily | - Budget plan divided in two parts: original budget plan and real budget\\ - Reporting on budget use, with financial documents to verify the costs\\ - Online partner meetings\\ - Coordination staff meetings | | ||
| | Partnership – local partners\\ - partners’ motivation and commitment level\\ - timely and quality implementation of tasks/quality services\\ - quality of communication between partners | Monthly\\ Getting closer to the start of the Tour: more frequently\\ During the Tour, in case of partners who are providing accommodation: daily | - Online and in-person meetings\\ - Agreement stating the conditions and provisions of collaboration | | | Partnership – local partners\\ - partners’ motivation and commitment level\\ - timely and quality implementation of tasks/quality services\\ - quality of communication between partners | Monthly\\ Getting closer to the start of the Tour: more frequently\\ During the Tour, in case of partners who are providing accommodation: daily | - Online and in-person meetings\\ - Agreement stating the conditions and provisions of collaboration | | ||
| - | | Activity programme’s efficiency\\ Assess:\\ - The activities contribute to reach the participants’ learning objectives\\ - The activities contribute to reach the project objectives\\ - The methods used are suitable for participants\\ - The time schedule of the activities is suitable for participants | Before the Tour: bi-weekly, starting from the preparatory activities\\ During the Tour: daily\\ After the Tour: bi-weekly, till the end of the follow-up activities | - Online and in-person consultations with youth workers during the preparation\\ - Daily feedback from participants during the Tour\\ - Daily monitoring meetings with facilitators during the Tour\\ - Mid-term and final evaluation session | | + | | Activity programme’s efficiency\\ Assess:\\ - The activities contribute to reach the participants’ learning objectives\\ - The activities contribute to reach the project objectives\\ - The methods used are suitable for participants\\ - The time schedule of the activities is suitable for participants | Before the Tour: bi-weekly, starting from the preparatory activities\\ During the Tour: daily\\ After the Tour: bi-weekly, till the end of the follow-up activities | - Online and in-person consultations with youth workers during the preparation\\ - Daily feedback from participants during the Tour\\ - Daily monitoring meetings with facilitators during the Tour\\ - Intermediate and final evaluation sessions | |
| - | | Facilitators’ work\\ Assess:\\ - Quality of support provided for participants before, during and after the Tour\\ - Team working skills\\ - Professional efficiency\\ - Respect for the pedagogical methodology\\ - Flexibility and immediate adaptability to participants’ needs | Before the Tour: bi-weekly, starting from the preparatory activities\\ During the Tour: daily\\ After the Tour: bi-weekly, till the end of the follow-up activities | - Online and in-person consultation with youth workers during the preparation\\ - Daily feedback from participants\\ - Daily monitoring meetings with facilitators\\ - Mid-term and final evaluation session | | + | | Facilitators’ work\\ Assess:\\ - Quality of support provided for participants before, during and after the Tour\\ - Team working skills\\ - Professional efficiency\\ - Respect for the pedagogical methodology\\ - Flexibility and immediate adaptability to participants’ needs | Before the Tour: bi-weekly, starting from the preparatory activities\\ During the Tour: daily\\ After the Tour: bi-weekly, till the end of the follow-up activities | - Online and in-person consultation with youth workers during the preparation\\ - Daily feedback from participants\\ - Daily monitoring meetings with facilitators\\ - Intermediate and final evaluation sessions | |
| - | | Participants well-being\\ Assess:\\ - Physical well-being\\ - Psychological well-being\\ - Emotional well-being | Before the Tour: bi-weekly, starting from the preparatory activities\\ During the Tour: daily\\ After the Tour: bi-weekly, till the end of the follow-up activities | - Online and in-person consultation with youth workers during the preparation\\ - Daily feedback from participants\\ - Daily monitoring meetings with facilitators\\ - Mid-term and final evaluation session | | + | | Participants well-being\\ Assess:\\ - Physical well-being\\ - Psychological well-being\\ - Emotional well-being | Before the Tour: bi-weekly, starting from the preparatory activities\\ During the Tour: daily\\ After the Tour: bi-weekly, till the end of the follow-up activities | - Online and in-person consultation with youth workers during the preparation\\ - Daily feedback from participants\\ - Daily monitoring meetings with facilitators\\ - Intermediate and final evaluation sessions | |
| - | | Participants’ learning achievements\\ Assess:\\ - How much the preparatory activities help to create a solid base for learning\\ - How much the Tour contributes to reach participants’ learning objectives \\ - Which objectives have been reached and which not yet\\ - Competencies developed\\ - Level of responsibility and contribution of participants\\ - Changes in attitudes and behaviours of participants\\ - How much the follow-up activities help participants to put into use the competencies acquired during the Tour | Before the Tour: bi-weekly, starting from the preparatory activities\\ During the Tour: daily\\ After the Tour: bi-weekly, till the end of the follow-up activities | - Daily reflection sessions with participants\\ - Non-formal evaluation methods\\ - Daily monitoring meetings with facilitators\\ - Mid-term and final evaluation session | | + | | Participants’ learning achievements\\ Assess:\\ - How much the preparatory activities help to create a solid base for learning\\ - How much the Tour contributes to reach participants’ learning objectives \\ - Which objectives have been reached and which not yet\\ - Competencies developed\\ - Level of responsibility and contribution of participants\\ - Changes in attitudes and behaviours of participants\\ - How much the follow-up activities help participants to put into use the competencies acquired during the Tour | Before the Tour: bi-weekly, starting from the preparatory activities\\ During the Tour: daily\\ After the Tour: bi-weekly, till the end of the follow-up activities | - Daily reflection sessions with participants\\ - Non-formal evaluation methods\\ - Daily monitoring meetings with facilitators\\ - Intermediate and final evaluation sessions | |
| - | | Level of achievement of objectives\\ \\ Assess:\\ - The objectives of the activities have been reached or not\\ - The objectives of the participants’ have been reached or not\\ - The objectives of the project have been reached or not | During the Tour: daily\\ After the Tour: monitoring moments 2 weeks, 1 months, 2 months, 6 months after the Tour | - Daily evaluation of activities\\ - If necessary, ad-hoc evaluation after the activity\\ - Mid-term and final evaluation with participants\\ - Final report on indicators that measure the level of achievement of project objectives | | + | | Level of achievement of objectives\\ \\ Assess:\\ - The objectives of the activities have been reached or not\\ - The objectives of the participants’ have been reached or not\\ - The objectives of the project have been reached or not | During the Tour: daily\\ After the Tour: monitoring moments 2 weeks, 1 months, 2 months, 6 months after the Tour | - Daily evaluation of activities\\ - If necessary, ad-hoc evaluation after the activity\\ - Intermediate and final evaluations with participants\\ - Final report on indicators that measure the level of achievement of project objectives | |
| - | Besides monitoring the aspects that you and your partners can control, it is important to think about the external factors as well: we suggest you to make a risk analysis and risk prevention and management plan before the project. You can analyze the following aspects: | + | Besides monitoring the aspects that you and your partners can control, it is important to think about the **external factors** as well: we suggest you to make a **risk analysis** and **risk prevention and management plan** before the project. You can analyze the following aspects: |
| | Possible risk/situation | How does it influence the project | Level of probability | Level of damage/how serious is the problem it can cause | How to prevent it | How to manage it in case it happens | | | Possible risk/situation | How does it influence the project | Level of probability | Level of damage/how serious is the problem it can cause | How to prevent it | How to manage it in case it happens | | ||
| - | As you see, the 1st table helps you to ensure quality and to get right back on track after a difficulty; the 2nd table facilitates problem solving and crisis prevention. | + | As you see, the 1st table helps you to ensure quality and to get right back on track after a difficulty; the 2nd table facilitates **problem solving** and **crisis prevention**. |
| For information about impact evaluation please see //Chapter 6 – Follow ups and dissemination//. | For information about impact evaluation please see //Chapter 6 – Follow ups and dissemination//. | ||
| - | {{ :guide:implementation:22365245_10213029595407244_4275051684887115183_n.jpg?direct&600 |}} | + | |
| - | ===== Are we reaching our objectives ? ===== | + | |
| + | |||
| + | ===== A Matrix to stay the course ===== | ||
| A **Logical Framework Matrix** created during the project design phase can be useful to monitor and evaluate during implementation whether you have reached the project’s objectives or not. | A **Logical Framework Matrix** created during the project design phase can be useful to monitor and evaluate during implementation whether you have reached the project’s objectives or not. | ||
| - | For creating a Logical Framework Matrix (details to how to do it here: [[http://www.tools4dev.org/resources/logical-framework-logframe-template/]]) you will need to: | + | |
| + | For creating a Logical Framework Matrix you will need to: | ||
| * Set clear **objectives** | * Set clear **objectives** | ||
| Line 36: | Line 40: | ||
| * Analyze closely which **circumstances** you need for a successful implementation | * Analyze closely which **circumstances** you need for a successful implementation | ||
| - | {{:guide:implementation:s1040013.jpg?direct&600 |}} | + | {{:guide:implementation:s1040013.jpg?direct&260 |}} |
| + | You can find details to how to do it here: [[http://www.tools4dev.org/resources/logical-framework-logframe-template/]]). | ||
| - | ===== Tools to take the scope of one's learnings ===== | + | As for any evaluation, be **factual and upfront** regarding the results. You don't need to convince yourselves that you are doing a good job; you create tools to identify what is going well and what isn't. Keep in mind that **problem solving begins with the identification of problems**, as well as real celebrations arise from honest assessments. |
| - | ==== Mapping out one's skills and competences with a Radar ===== | + | ===== Tools to take the scope of one's learnings ===== |
| - | + | ||
| - | Another tool that can be used is the **radar of skills and competences**. | + | |
| - | Ask participants to take **15 minutes before the Tour starts** to examine what their **expectations** in terms of skills and competences development are. | + | |
| - | // 1. What kind of skills do you expect to learn or improve during the Heterotopia Tour?// | + | Participants' learnings are at the core of any Heterotopia Tour. We expect from it to inspire them and to enable them to acquire knowledge and skills in various fields; some they expected, and some they didn't. |
| - | // 2. Identify how strong is your expectations to learn or improve each of them and set an indicator for each.// | + | |
| - | For example, participant "A" could write: "I really expect to learn about some ecoconsruction techniques during the Tour" and sets an indicator of 7 out of 10, corresponding to his/her expectations in terms of learning about those techniques. Likewise, they would also like to improve their English speaking skills, but their expectations are lower than for eco-construction ; he/she sets an indicator of 5 out of 10. | + | For this reason, it is decisive to develop and implement **tools and methods** that will allow them to **acknowledge their learning** and provide you information about the efficiency of your program from a pedagogical point of view. |
| - | {{contributions:radar.png?200 |}} | + | From our experience, we advice you the following: |
| - | **Ask participants the same question at the end of the Tour and compare the answer**. What did they expect to learn at the very beginning, and where are they at now that it's the end of the Tour? Does it match or not? Are there huge differences? Are those either positive or negative? That way you can evaluate how well the journey meets the participants' expectations. | + | [[guide:implementation:evaluation:tools_and_methods1|Mapping out one's skills and competences with a Radar]] |
| - | As facilitators, if you find the time for it and you are eager to readapt the Tour on the go, it might be interesting to do a mid-term assessment (that can take the shape of a Sharing Circle, as described previously, for example) and see **to which extend the Tour answers to the participants' expectations**. Even though it is very challenging to modify the content of the Tour, **it might be possible to some extend to adapt the animation or the facilitation**, for example, by focusing more either on theory or practice depending on the participants' needs. | + | [[guide:implementation:evaluation:tools_and_methods2|Reflecting on one's learning path with the Travel Diary and the Hero's Journey]] |
guide/implementation/evaluation.txt · Last modified: 2018/11/05 15:55 by Cecilia
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
