Find your way
Follow us on Facebook
Sidebar
guide:implementation:evaluation
This is an old revision of the document!
Table of Contents
Monitoring & evaluation
The evaluation process
During the implementation of a Heterotopia Tour you have to monitor and evaluate different aspects of the project. Frequent monitoring allows you to prevent risks, to manage problems in time, to ensure quality, and to anticipate possible situations that can influence your project.
General monitoring and evaluation plan can include:
| Aspect to monitor | Minimum frequency | Tools for monitoring |
|---|---|---|
| Partnership – implementing partners Assess: - partners’ motivation and commitment level - fair task division - timely and quality implementation of tasks - respect and solidarity among partners - quality of communication between partners | Monthly | - Online and in-person meetings - Gantt chart with tasks, responsibilities and deadlines - Quality assurance guidelines with qualitative and quantitative Indicators - Written and signed agreement between partners about conditions and provisions of collaboration - Written reports |
| Management – logistics (accommodation, food, transport, materials) Assess: - timely and quality implementation of tasks - quality of services | Monthly Getting closer to the start of the Tour: more frequently During the Tour: daily | - Gantt chart with tasks, responsibilities and deadlines - Feedback from participants and from facilitators - Coordination staff meetings |
| Management – budget use Assess: - Budget efficiency (quality-price value) - Fair budget allocation (did any partner had justifiable over-costs that should be covered? ) - Budget efficiency | Monthly Getting closer to the start of the Tour: more frequently During the Tour: daily | - Budget plan divided in two parts: original budget plan and real budget - Reporting on budget use, with financial documents to verify the costs - Online partner meetings - Coordination staff meetings |
| Partnership – local partners - partners’ motivation and commitment level - timely and quality implementation of tasks/quality services - quality of communication between partners | Monthly Getting closer to the start of the Tour: more frequently During the Tour, in case of partners who are providing accommodation: daily | - Online and in-person meetings - Agreement stating the conditions and provisions of collaboration |
| Activity programme’s efficiency Assess: - The activities contribute to reach the participants’ learning objectives - The activities contribute to reach the project objectives - The methods used are suitable for participants - The time schedule of the activities is suitable for participants | Before the Tour: bi-weekly, starting from the preparatory activities During the Tour: daily After the Tour: bi-weekly, till the end of the follow-up activities | - Online and in-person consultations with youth workers during the preparation - Daily feedback from participants during the Tour - Daily monitoring meetings with facilitators during the Tour - Intermediate and final evaluation sessions |
| Facilitators’ work Assess: - Quality of support provided for participants before, during and after the Tour - Team working skills - Professional efficiency - Respect for the pedagogical methodology - Flexibility and immediate adaptability to participants’ needs | Before the Tour: bi-weekly, starting from the preparatory activities During the Tour: daily After the Tour: bi-weekly, till the end of the follow-up activities | - Online and in-person consultation with youth workers during the preparation - Daily feedback from participants - Daily monitoring meetings with facilitators - Intermediate and final evaluation sessions |
| Participants well-being Assess: - Physical well-being - Psychological well-being - Emotional well-being | Before the Tour: bi-weekly, starting from the preparatory activities During the Tour: daily After the Tour: bi-weekly, till the end of the follow-up activities | - Online and in-person consultation with youth workers during the preparation - Daily feedback from participants - Daily monitoring meetings with facilitators - Intermediate and final evaluation sessions |
| Participants’ learning achievements Assess: - How much the preparatory activities help to create a solid base for learning - How much the Tour contributes to reach participants’ learning objectives - Which objectives have been reached and which not yet - Competencies developed - Level of responsibility and contribution of participants - Changes in attitudes and behaviours of participants - How much the follow-up activities help participants to put into use the competencies acquired during the Tour | Before the Tour: bi-weekly, starting from the preparatory activities During the Tour: daily After the Tour: bi-weekly, till the end of the follow-up activities | - Daily reflection sessions with participants - Non-formal evaluation methods - Daily monitoring meetings with facilitators - Intermediate and final evaluation sessions |
| Level of achievement of objectives Assess: - The objectives of the activities have been reached or not - The objectives of the participants’ have been reached or not - The objectives of the project have been reached or not | During the Tour: daily After the Tour: monitoring moments 2 weeks, 1 months, 2 months, 6 months after the Tour | - Daily evaluation of activities - If necessary, ad-hoc evaluation after the activity - Intermediate and final evaluations with participants - Final report on indicators that measure the level of achievement of project objectives |
Besides monitoring the aspects that you and your partners can control, it is important to think about the external factors as well: we suggest you to make a risk analysis and risk prevention and management plan before the project. You can analyze the following aspects:
| Possible risk/situation | How does it influence the project | Level of probability | Level of damage/how serious is the problem it can cause | How to prevent it | How to manage it in case it happens |
As you see, the 1st table helps you to ensure quality and to get right back on track after a difficulty; the 2nd table facilitates problem solving and crisis prevention.
For information about impact evaluation please see Chapter 6 – Follow ups and dissemination.
A Matrix to stay the course
A Logical Framework Matrix created during the project design phase can be useful to monitor and evaluate during implementation whether you have reached the project’s objectives or not.
For creating a Logical Framework Matrix you will need to:
- Set clear objectives
- Describe the expected results that will contribute to reach the objectives
- Describe the specific activities that will produce the expected outcomes
- Set reasonable quantitative indicators that will measure the level of success
- Decide which proof you can provide to justify the reached results
- Analyze closely which circumstances you need for a successful implementation
You can find details to how to do it here: http://www.tools4dev.org/resources/logical-framework-logframe-template/).
As for any evaluation, be factual and upfront regarding the results. You don't need to convince yourselves that you are doing a good job; you create tools to identify what is going well and what isn't. Keep in mind that problem solving begins with the identification of problems, as well as real celebrations arise from honest assessments.
Tools to take the scope of one's learnings
Participants' learnings are at the core of any Heterotopia Tour. We expect from it to inspire them and to enable them to acquire knowledge and skills in various fields; some they expected, and some they didn't.
For this reason, it is decisive to develop and implement tools and methods that will allow them to acknowledge their learning and provide you information about the efficiency of your program from a pedagogical point of view.
From our experience, we advice you the following:
Mapping out one's skills and competences with a Radar
You can use a radar of skills and competences to help participants to define their learning expectations and to assess to which extend they reach them.
When the Tour starts, ask participants to take 15 to 20 minutes to examine what their expectations in terms of skills and competences development are.
- 1. What kind of skills do you expect to learn or improve during the Heterotopia Tour?
- 2. Identify how strong is your expectations to learn or improve each of them and set an indicator for each.
For example, participant “A” could write: “I really expect to learn about some eco-construction techniques during the Tour” and sets an indicator of 7 out of 10, corresponding to his/her expectations in terms of learning about those techniques. Likewise, they would also like to improve their English speaking skills, but their expectations are lower than for eco-construction ; he/she sets an indicator of 5 out of 10.
Ask participants the same question at the end of the Tour and compare the answer. What did they expect to learn at the very beginning, and where are they at now that it's the end of the Tour? Does it match or not? Are there huge differences? Are those either positive or negative? That way you can evaluate how well the journey meets the participants' expectations.
As facilitators, if you find the time for it and you are eager to re-adapt the Tour on the go, it might be interesting to do an intermediate assessment (that can take the shape of a Sharing Circle, as described previously, for example) and see to which extend the Tour answers to the participants' expectations. Even though it is very challenging to modify the content of the Tour, it might be possible to some extend to adapt the animation or the facilitation, for example, by focusing more either on theory or practice depending on the participants' needs.
Reflecting on one's learning path with the Travel Diary
The Travel Diary is a tool which has been put to the test during 3 Heterotopia Tours in different formats, which purpose is to help the participants reflect on their learning path in three steps.
Its final version is inspired from the “Hero’s Journey”, a common template of stories with specific acts and archetypes describing the evolution of a person. (For more info look up Joseph Campbell: The Hero with a Thousand Faces.)
The Travel Diary we used was divided in three parts:
Step 1: Departure. Before the Tour.
Used during preparatory activities. Helps to reflection on:
- Motivation of the participant
- Fears of the participant
- Resources of the participant (internal and external)
- Support: mentors, accompanying persons
- Expectations and general objectives of the participant
Step 2: Initiation. During the Tour.
Used during the first day, daily reflection, intermediate and final evaluation. Helps to reflect on:
- Specific learning objectives of the participant
- Obstacles that the participant faced, ways of overcoming them
- Support the participant found during the Tour
- Learning process, adjustments that have to be done
- New competencies: knowledge, skills and attitudes
- Learning outcomes, personal achievements
Step 3: Return. After the Tour.
Used during assessment and follow-up activities. Helps to reflect on:
- How to put the newly acquired skills in use in the participant’s ordinary life
- How to bring benefits to the community through the participant’s experience
- Longer-term impact on the participant
Example of the Travel Diary is downloadable from here: https://www.dropbox.com/s/iews83w6f6k5ilg/heterotopiatour_traveldiary%23Sicily-en.pdf?dl=0
Guide for facilitating activities with the Travel Diary:
| The Hero's Journey - Acts | The participants' journey - The Heterotopia Tour | |
|---|---|---|
| ACT 1. Ordinary World | 1. Ordinary world – Hero’s everyday life, description of hero’s character: his nature, his capabilities, way of living. | The Identity of the participant, his/her context, way of living, already existing competencies. |
| 2. Call to Adventure – The call ultimately disrupts the comfort of the Hero's Ordinary World and presents a challenge or quest that must be undertaken. It can be a threat, or any kind of discomfort. | Motivation of the participant: discomfort caused by something in theit ordinary world, effecting them or people important for them; something they wants to change in their life. | |
| 3. Refusal of the call – The Hero has second thoughts or even deep personal doubts as to whether or not he is up to the challenge. When this happens, the Hero will refuse the call and as a result may suffer somehow. | A memory of passiveness when the participant first recognize that there is a challenge to face; fears, doubts, reasons for not acting. The effect of this passiveness on the participant’s life. | |
| 4. Meeting the mentor – The Hero meets a mentor who provides guidance: they could be given an object of great importance, insight into the dilemma they face, wise advice, practical training or even self-confidence. | People and situations that pushed somehow the participant to take a step and act on their challenge, make a first move towards the change they wants to see. (They can be people and situations from already before the selection as participant, and also the partner organisations who are supporting and preparing the participant. | |
| Threshold | 5. Passing the Threshold – The Hero is now ready to act upon their call to adventure and truly begin their quest, whether it be physical, spiritual or emotional. They may go willingly or may be pushed, but either way they finally cross the threshold between the world they are familiar with and that which they are not. | Setting objectives and committing to the project. The Moment of starting the Heterotopia Tour! Entering in an unfamiliar situation. |
| ACT 2. Special World | 6. Tests, Allies, Enemies – The Hero begins their journey. The Hero needs to find out who can be trusted and who can't. They may earn allies and meet enemies who will, each in their own way, help prepare him for the greater ordeals yet to come. This is the stage where their skills and/or powers are tested and every obstacle that he faces helps us gain a deeper insight into his character. | Test: The obstacles that participants face during the Tour. Obstacles that can be overcome, if they try and look for support. After each test participants evaluate what they have gained (e.g. during the daily reflection groups, reflecting on the learning outcomes of the day). Allies: Support participants find during the Tour, which can be various things: peer support, support from the youth leaders and facilitators, the supporting environment, already existing qualities and knowledge they brought with themselves.Enemies: Things, people, circumstances that obstacle participants to reach their objective, and in overall to feel well. |
| 7. Approach To The Inmost Cave – The Hero enters into the Cave to face their biggest challenge. They may once again face some of the doubts and fears that first surfaced upon their call to adventure. They may need some time to reflect upon their journey and the treacherous road ahead in order to find the courage to continue. | Mid-term evaluation moment will be used to represent the Approach to the inmost cave. Participants will look back on the path they’ve already done and will look on the upcoming days, analyzing the changes they have to make in order to reach the set objectives. |
|
| 8. The Supreme Ordeal – it may be a dangerous physical test or a deep inner crisis that the Hero must face. They must draw upon all of their skills and their experiences gathered upon the journey in order to overcome their most difficult challenge. This is the high-point of the Hero's story and where everything they holds dear is put on the line. Either they win or fail, their life will never be the same again. | Final evaluation moment: Looking back to the most difficult moment during the tour, the moment that participants identify as “life changing”. To identify this moment, participants will reconstruct the whole journey and reflect on its different elements, moments, difficulties, the gained competencies etc. | |
| 9. Reward (Seizing the Sword) – After defeating the enemy, surviving death and finally overcoming their greatest personal challenge, the Hero is ultimately transformed into a new state, emerging from battle as a stronger person and often with a prize. The Reward may come in many forms: an object of great importance or power, a secret, greater knowledge or insight. | Final evaluation moment: Learning outcomes of the participants (skills, attitudes, knowledge, other). Assessing the changes in the character: before the tour and at the end of the tour. |
|
| ACT 3. Return to the Ordinary World | 10. The Road Back – The Hero's journey is not yet over and they may still need one last push back into the Ordinary World. The moment before the Hero finally commits to the last stage of his journey may be a moment in which he must choose between their own personal objective and that of a Higher Cause. | Returning to the ordinary life to participants after the Tour can be shocking and disturbing – they have changed, but things at home are still the same. This is the moment where we have to remind them (through follow-up and reintegration activities) that they left because they wanted to trigger some changes in their life: it is a reflection moment on how they can use the gained knowledge and competencies to serve also their communities and to trigger changes. Reinforcing the “social” and “collective” aspect of the project. |
| 11. Resurrection – This is the climax in which the Hero must have their final and most dangerous encounter with death. The final battle also represents something far greater than the Hero's own existence with its outcome having far-reaching consequences to his Ordinary World and the lives of those he left behind. Ultimately the Hero will succeed, destroy their enemy and emerge from battle cleansed and reborn. | Evaluation moment after the project (during the local assessment sessions): To what level participants have reached their overall objective? (Not the learning objectives, which are the “rewards”, but the change they wanted to see.) On “their way to return”, so since they came back home, have they used the things they gained during the Tour? | |
| 12. Return with the Elixir – This is the final stage of the Hero's journey in which they return home to their Ordinary World a changed man. They will have grown as a person, learned many things, faced many terrible dangers and even death but now looks forward to the start of a new life. Their return may bring fresh hope to those he left behind, a direct solution to their problems or perhaps a new perspective for everyone to consider. | Follow-up activities: participants put in action their ideas and initiatives, bringing positive societal changes. (Can be implemented and monitored in long term.) | |
Mapping out one's skills and competences with a Radar
Reflecting on one's learning path with the Travel Diary and the Hero's Journey
guide/implementation/evaluation.1540371119.txt.gz · Last modified: 2018/10/24 10:51 by Marina
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
