Informatika | Gazdasági Informatika » Results based Programming, Management and Monitoring, RBM Guide

Source: http://www.doksinet RTC Consultants UNESCO Bureau of Strategic Planning ( BSP ) RESULTS BASED PROGRAMMING,MANAGEMENT AND MONITORING (RBM) GUIDE 1) Introduction By decision of the Director General and the Executive Board, staff and managers in each Sector and Programme of UNESCO are henceforth required to apply Results-Based Programming, Management and Monitoring (RBM) to the design and implementation of all UNESCO’s activities, namely Major Programmes, Programmes, Subprogrammes, Main lines of action, actions and activities. This Guide introduces a number of the fundamental concepts that underpin RBM. In essence, RBM is a quite straightforward way of thinking about three basic questions that are fundamental to any organisation: What do we wish to achieve ? What will we do to reach that goal ? How will we know whether we have achieved our goal ? One of the core values of a modern approach to RBM is a clear move away from downward-driven and resource-based

command-and-control management systems, towards new models based on collective responsibility and delegation, interaction and collective accountability. 2) What is RBM? a) RBM is a method to help reach and assess achievements RBM is about a managerial philosophy and approach which underpins decision-making and provides for regular feedback and adjustment. It helps moving the focus of managing and decision-making from a tradition of assessing what has been done and how it has been done, to a concentration on selecting objectives to be met, and assessing what was achieved as an implementation of such objectives. For that purpose, it focuses on 1 Source: http://www.doksinet what an organization intends to achieve in terms of the nature of the real changes that programming activities are designed to bring about, and the relevance of such activities; medium-term ways of identifying and showing staff and managers, as well as all stakeholders, how the implementation of that programming

actually “made a difference” in bringing about such changes. For example, if what is being sought is to build a policy-making capacity within a number of national governments, or to enhance the capacity of societal institutions to respond to a recognized problem, the RBM approach is not to focus on the conferences, the training, the seminars or the networks that may have been undertaken. Instead, the RBM approach will assess whether that policy-making capacity was enhanced; or whether those societal institutions succeeded in grappling with the problem. Some of UNESCO’s bilateral and multilateral partners have adopted their own unique approaches to RBM over the last decade. For them, RBM has helped to improve the quality of decision-making and enhance the effectiveness of programming. As each organization has unique characteristics, mandates and responsibilities, the way RBM is introduced and the way it evolves will necessarily differ from one body to the next. b) RBM is a

participatory, flexible and living method RBM seeks to develop at the beginning of the planning process a complete understanding of what is to be achieved, by whom and how achievements will be assessed. It also seeks to build a collective understanding of the scope of activities and how they contribute to the larger, more strategic objectives and goals that an organisation may set. It therefore calls for a three-way flow of information within an organisation : between staff and managers, among every staff equally and, above all, with stakeholders Negotiation and validation among colleagues, and between staff and managers, are vital parts of a fully-developed approach to RBM, which is based on openness, collective problem-solving and collective decision-making. These values reflect an ethos where it is natural to discuss upwards, downwards and horizontally, and to do so in a transparent and timely manner. To that end, the negotiation and validation elements of SISTER are indeed

electronic proxies for the philosophy of RBM. RBM is a fundamentally flexible approach to planning and decisionmaking. The tools and techniques needed to respond to the three primary questions listed at the start of this Guide will vary, not only among organizations, but also within an organization. Different types of organizational elements will need different approaches and different tools. For example, administrative and service-delivery elements of an organization will address the three questions differently from the way entities dealing with policy, forecasting and planning issues will. But all parts of the Organization must address all three questions. RBM is a flexible way for every part of UNESCO to plan, assess progress and take decisions. 2 Source: http://www.doksinet The very concept of “results” will for example vary according to the level at which it is considered : in the context of UNESCO, the design of a “ result” is required at each of the six existing

levels of programming, and this is also the core principle along which SISTER has been designed. • • • In the context of the medium term strategy ( C/4 ) , specific OUTCOMES must be formulated for each strategic objective and sub-objective, which gives the Major Programmes and Programmes their direction and programmatic guidance In the context of the biennial programme and budget ( C/5 ), EXPECTED RESULTS are articulated, pertaining to each main line of action. In the workplans, more OPERATIONAL RESULTS are formulated for each and every action and activity, including also performance indicators All of these elements are to be entered into SISTER It is obvious that the nature, magnitude, meaning of “results” cannot be the same between the outcomes of a medium term strategy and the series of outputs making the result of an activity. Nevertheless, it is crucial that all these results build a chain of meaningful achievements, bridging the gap between the permanent goals of

UNESCO and what the Organisation actually achieves in its current daily action. The method of nesting Logframes, from Major Programme down to activity and vice versa is a powerful one to make such a programming a coherent and efficient one. SISTER is based on such a logic, and contributes to relate distinct levels to each other. Making results fit together and add up into major outcomes for the Organisation as a whole is what designing strategies is all about. This is the core task of senior management, and indeed permeates all programme management. Nesting Logframes : the logics of SISTER support the RBM approach to programming MP P Goal Purpose Goal Output Purpose Programme Output Sb progr; OUTCOMES SP Goal Purpose Output MLA MLA Goal Purpose Output Action “expected results” Action activity Goal Purpose Output Activity Goal Purpose Output “ operational results” 3 Source: http://www.doksinet Specific indicators are required for each type of “ result”. They

also serve as entry points for evaluation and monitoring assessments . They must concentrate on achievements, not on compliance. The feedback and analysis which are core components of RBM imply that new solutions must be welcomed and that an organization and its programming cannot remain static in an ever-changing world. RBM thus reflects the acceptance of continuous improvement as part of an organization’s basic culture RBM is well adapted to UNESCO’ s requirements 3) The Programme Logic of RBM - A Simplified View RBM is a simple approach to programming logic that is inherently intuitive, where one step builds immediately on the earlier. This graphic illustrates the process and emphasizes that it is cyclical. The first step - setting objectives and expected results - is, and must be, inextricably linked with the final - assessing actual results. From objectives to outcomes : the RBM programming cycle Constant assessment and revision 6) Outcomes What approaches will 1)

objectives and Expected results What did you achieve in terms of the medium term changes you sought? ( C/4 focus) 5) Actual you use? With Whom do you intend to partner ? What do you intend to achieve or change? How does this relate to UNESCO priorities and themes? ( C/5 focus ) 2) Strategies How well did you How much financial/ do in terms of what human resources will you achieved, versus you require ? resultswhat you intended to achieve or to change ? How do you measure it ? 3) Inputs Performance indicators 4) Outputs What will actually be done in terms of programming? Necessarily, this drawing oversimplifies the complexities inherent in the programme logic that underpins the reality of UNESCO. Its purpose is not to map all the steps of programme logic, or to factor in other necessary management tools such as costefficiency analysis and internal auditing to ensure regulatory compliance, etc. It is just to show the necessary relationship between the articulation of Expected

Results and what is actually achieved as compared with what was intended. Equally, it shows how the practice of negotiation and validation mentioned earlier and enshrined in SISTER must occur at each step as part of a on-going dialogue between all levels 4 Source: http://www.doksinet contributing to a coherent series of results, from primary output to major outcomes : the output of an activity must be logically linked with how the action of which this activity is part and parcel will itself further the attainment of the Main Line of action’s Expected Result in the first place, and so on. This whole cycle is directed towards an approach to management that goes beyond an examination of what was done and how many resources were consumed, towards one which concentrates on the assessment of the impact of programming simply, i.e what was achieved Goals/ objectives have to be kept in a given relation to purposes, outputs and acts, so as to form together an integrated whole. It

constitutes the main planks of the Logical Framework approach, so that acts contribute to the purpose through an output, and the purpose fits to the goal. In such a framework, one could denote the result as being the level of approximation between output and purpose. Each of the four concepts take its exact meaning from its relation to the three others, opening the way for a very creative and yet extremely rigorous approach to programming by collective thinking and joint designing of strategies. Goals Conditions Purpose Conditions outputs Conditions Acts Conditions Performance Indicators Means of verification RESULTS 4) The Immediate Focus - The Main Line of Action Out of the six levels of UNESCO’s programming • Major Programme o Programme  Sub-Programmes • Main line of action o Action  Activity the Main Line of Action (MLA) is the place where outputs from actions and activities are aggregated and contribute to build up outcomes for major programmes. This is

where the C/5 articulates expected results against which the Organisation’s action will be monitored and evaluated, where strategic and operational factors intersect. 5 Source: http://www.doksinet For UNESCO the ability to determine what is being achieved first becomes truly practical at the level of the Main Line of Action. It is at this level where the number and duration of individual types of programming come together in a sufficient critical mass to begin to make it possible to look towards achievements. The building blocks of programming ( actions ) begin to be aggregated in sufficient volume so as to clarify the relationship between what is being sought, what is being done and what has been achieved – in essence the responses to the three questions that opened this Guide. The higher levels of UNESCO programming serve to house related Main Lines of Action whereas at the lower levels - action and activity - the focus turns to individual instances of programming. Main

Lines of Action naturally work together to contribute to broader programme objectives. The articulation of sub-objectives for each Major Programme in the draft C/4 illustrates the way how Main Lines of Action come together to form a totality that reflects accomplishment by a sector, which can themselves be monitored in terms of results and achievements as well. In addition, all MLAs need to take into account to what extent and how they relate to the strategic objectives of the medium term strategy ( C/4 ) and respond to the two cross-cutting themes : the contribution of information and communication technologies to the development of education, science, and culture, and the construction of a knowledge society and the eradication of poverty, especially extreme poverty. In this way, the collective problem-solving nature of RBM will facilitate the assessment of how Main Lines of Action collectively contribute to UNESCO-wide strategies and goals, thereby sharpening the focus and impact of

all that UNESCO does. 5) Developing a common terminology for RBM at UNESCO The following short list of terms may be a starting point for UNESCO, and thus a common understanding of the basics of RBM at the MLA. An Expected Result “ Expected results describe a change in a situation and/or benefits to end-users/clients due to activities or interventions of the Organisation. Expected results are specific and can be captured either in quantitative or qualitative terms”. Expected results can be designed at each and every level of programming, taking different faces according to what should be expected from each level – mainly outputs at the lowest level, merely outcomes when in comes to higher strategic levels. Strategies Strategies describe the broad ways in which one intends to work towards the achievement of an Expected Result. They address conceptual issues such as the nature and modalities of UNESCO’s work ( whether catalytic, or normative, or as a clearing house, etc), whether

it will be directed more towards action or towards reflection, the range of actions to be taken for that purpose at the appropriate level, and the means required for that purpose. They need to articulate how UNESCO’s action comes 6 Source: http://www.doksinet into play amongst other bodies’ role in the same field of intervention, and to take into account all stakeholders ‘ roles and interests. An Input inputs are human, financial and other resources necessary for achieving results and producing outputs An Output Outputs are final products, services or general activities delivered to end users/ clients. Such outputs are only sought for the purpose of contributing to the achievement of an expected result. Producing an output is by itself meaningless and useless. It only gains relevance by contributing adequately to an expected result, at the lowest possible input cost. An Outcome Outcomes are overall changes in situations and/or benefits to end users/clients, either

qualitative or quantitative, in the area where UNESCO has intervened, that may or may not be substantially due to the Organisation’s activities but were set out earlier as Expected Results and as such benefited from the Organisation’s efforts. An Actual Result an Actual Result is the collective assessment within the span of a biennium of the kind of, and degree of, change brought about in the field of action earlier articulated in the Expected Results set out for the MLA – or for any other level of programming, the term “ result “ being adequately used at each of them as a generic concept. Performance indicators Tools measuring whether and/or to what extent an expected result or outcome has been achieved Means of verification The design of any indicator should specify how the required information will be gathered, checked, ascertained and built up into a significant indication. The corresponding resources should be foreseen from the outset, so as to make the indicator an

effective one. 6) Monitoring, Review and Programme Evaluation • • • Programming is subject to three main types of feed-back-providing operations : Monitoring Review Evaluation As soon as the implementation phase begins, monitoring phases in, based on periodical assessments both by programme sectors and by central services. SISTER is conceived to support such a monitoring process, by requiring each responsible officer to crisply report, on a quarterly basis, upon the progress of the element he/she is in charge of, and assess how the result progressively takes shape. This requirement calls 7 Source: http://www.doksinet for some form of indicators and milestones to be designed, so as to anchor the assessments to verifiable evidence. The delivery-orientation of the monitoring process can foster continuous improvement of implementation and raise quality control. The process of Review takes place at a less frequent rate. As the monitoring, it is a task to be performed by staff

and managers themselves, as opposed to Evaluation, which is entrusted to external auditors or evaluators. Performance Indicators are often viewed as the cornerstones of such a process, and should as such be built in the initial programming as appropriate. Information on programming and monitoring recorded in SISTER by the responsible officers themselves provide the foundation for this attempt to measure the relation between what was intended and was is being achieved Evaluation, by nature, comes later as a major source of lessons to be learnt and of open discussion of all aspects of the whole process, in the context of a longer period of time, frequently an entire programming cycle of a number of years spanning a number of bienniums. It requires external and neutral competence, and a full set of information about all aspects of the element, policy or action under scrutiny. It focuses both on impact of what has been achieved ( the real measure of outcomes ) and on the quality of the way

is was done. SISTER embodies the concepts of both Review and Monitoring as it seeks periodic reporting on progress, both in terms of the assessment of outcomes and also the allocation of human and financial resources. The reports generated by the system as well as its open access, can contribute to the requirements of evaluation and reporting, thus encouraging a greater degree of collective decision-making and transparency of action. Review and evaluation also form major components of reporting by the Director General to governing bodies, be it in the form of the C/3, the EX/4 or evaluation reports, which henceforth must be closely based on all information entered into and registered by SISTER. 7) Expected results at the MLA level At the level of the MLA, the expected results which have been articulated for programming probably appear as a mix of three basic varieties: ♦ Clearly quantifiable expected results - ones which set out concrete expected changes such as "increasing the

number of qualified professionals from X to Y over a set period of time”. ♦ Broad quantifiable Expected Results - such as changing participation rates in a population, for example increasing the population’s access to professional training from X to Y over a longer time frame. ♦ More qualitative Expected Results - ones which articulate change in terms of building capacities, changing attitudes, altering personal or organizational behavior. 8 Source: http://www.doksinet They all need different types of indicators. 8) Performance indicators Within the context of RBM, results are mainly expressed and measured by Performance Indicators. The key concept underpinning the development of Performance Indicators is that they are just that - indications of change. The primary use of Performance Indicators is to guide management and staff decision-making, and to enable sensitive, flexible and timely mid-course corrections to programming as well as assessments. This core purpose does

not require sophisticated statistical tools, but reliable signals that tell much, directly or indirectly, about the real facts on which one undertakes to have leverage. A fair balance is to be sought between the cost ( time, money ) to build and maintain them, and the real need to have them at hand. 9) Types of Performance Indicators The following briefly introduces the general types of performance indicators that can be used to assess progress towards the achievement of Expected Results - these are the tools needed to answer the third question: “How do we know whether we are achieving/ have achieved our goal ?” a) Direct Statistical Indicators Direct statistical indicators show progress when expected results are cast in terms of readily quantifiable short-to-medium term changes. For example, if the expected result is to increase the number of professionals trained in a skill over a time period of a biennium or less, it should not be difficult to secure direct quantifiable data on

a regular basis. Care must be taken to ensure that the time span of the expected result lends itself to the collection of such data for use in review. An example drawn from Major Programme II illustrates the nature of this type of indicator and its requirements. If the Expected Result is to: “arrange for the training of 3,200 scientists in theoretical physics and mathematics”, then the Direct Statistical Indicator would be simply a count over the term of a biennium of the number for whom training has been arranged. The responsible MLA will have to put into place a means to track progress towards the achievement of this Expected Result on a regular basis – most likely every four or six months. Doing so only once a year, in essence at the “half way point” of the biennium, would not be sufficient, as the assessment of only one “snapshot in time” would not permit mid-course corrections should they be required – the core of the concept of continuous improvement b) Proxy

Indicators Proxy indicators are normally quantitative, but do not directly relate to the expected result. A Proxy is used to show “ progress It should be used when getting the full data is too time-consuming, or when the timeliness of complete data would fall outside the need for Review. However, there must to be a prima facie connection between the proxy and the expected result. 9 Source: http://www.doksinet An example drawn from Major Programme III illustrates how a Proxy Indicator could be used to assess progress in what might seem to be an intangible situation. If the Expected Result is: “greater awareness promoted among the general public and decisionmakers about the major challenges of the future.”, a good Proxy Indicator might be to collect data on the number of times public figures in the subject countries/organizations spoke of the challenges of the future or made reference to UNESCO’s work in the area, the number of times the mass media reported on future

challenges and/or the number of times specialized or scholarly publications addressed the challenges of the future. In this case, collecting data every six months would be satisfactory. In the longer term, Programme Evaluation should offer statistical data to ascertain more accurately the totality of the factors and variables at play. c) Narrative Indicators In many instances where the expected result may be qualitative (changing attitudes, building capacities, etc), a non-statistical approach may be the only way possible to develop an indication of “progress” . Narrative indicators largely focus on the “process of change” - asking stakeholders what they did as a result of the assistance. This technique works especially well in instances where capacity-building, training, conferences, network development and workshops are the planned outputs. Given UNESCO’s work, narrative indicators may be especially relevant. However, when dealing with stakeholders, care needs to be taken

to avoid a focus simply on “satisfaction”. Rather, the focus should be on what happened as a result of the intervention/participation. It should also be noted that narrative indicators can seldom be easily quantified over the short term. An example drawn from Major Programme I illustrates how narrative indicators could be used. If the expected result is to “Enhance national capacities for policy formulation in adult and life-long learning”, then a valid narrative indicator might be a follow-up questionnaire to be circulated among those individuals who participated in training, or conferences or other activities to ask them what they did in their countries as a result of UNESCO’s actions. Such a questionnaire should not be a survey of client satisfaction. It should ask: ”What did you do as a result of UNESCO’s actions ?” It also should be administered to stakeholders several times – at least once a year - in order to develop a “baseline” and thus begin to assess

the continuum of change. It may be that oral interviews could be used in lieu of a formal written response. Narrative indicators enable an organization to begin to explore complex interrelationships among factors without recourse to extremely expensive statistical research. In this way, UNESCO could demonstrate “partial success” even if other factors may have prevented the overall “enhancement of the national capacity”. This example also illustrates how a proxy Indicator could be combined with a narrative indicator. In this case, a reliable proxy indicator might be to enumerate the number of instances where countries have implemented new polices or programmes related to adult or life-long learning. The proxy has not measured “enhanced capacity”; rather it has shown domestic programmatic progress. 10 Source: http://www.doksinet Given the extent to which many of UNESCO’s expected results may seem to be intangible, narrative indicators offer the prospect of great

utility. However, they should not be seen as a wide-spread substitute for data that can be quantified to some degree. Narrative Indicators are extremely difficult to quantify as their focus is the “story”. For the purposes of Review at the level of the MLA, care needs to be taken to use the qualitative data derived from them as a “ snap shot” that will facilitate decision-making and continuous improvement within the context of a biennium – and not as a final judgement. 10) Investing In Performance Indicators Developing performance indicators for Review will be fruitless unless an investment is made to ensure that the data can be systematically collected. At the MLA, this implies that sufficient time and money be set aside to follow up on what has been done. For example, the use of narrative indicators that seek to contact stakeholders implies that UNESCO needs to invest time and money in such follow-up; and that such follow-up become a planned activity, and not an ad hoc or

informal process. Similarly, the use of Proxy Indicators and Direct Statistical Indicators requires a regularly-scheduled approach to data collection and the identification in advance of data sources. This sort of follow-up must be an integral element of programming design. Another common problem which has beset a number of organizations that have adopted RBM is the tendency to articulate expected results and set out performance indicators before ascertaining data sources. 11 Source: http://www.doksinet The RBM Guide Exercise We will now begin to look at the particular Main Line of Action of you and your colleagues. The purpose of this Guide is to provide an opportunity for discussion and, through discussion, for learning about the fundamentals of RBM Take enough time to consider all questions, to explore alternatives and to discuss what RBM means for you and your colleagues. Remember, there are no “correct” or “ textbook” answers. Going through the following questions

might help you in the process of programming Step 1: Your Main Line of Action a) What is your Main Line of Action ? How many other Main Lines of Action are there within your Major Programme ? What is the relationship of your Main Line of Action to others within your Major Programme? Does your Main Line of Action work with other Major Programmes and if so, how? b) What does your Main Line of Action attempt to achieve in the broadest sense what are the broad goals of your Main Line of Action ? Hint: Think about what the Main Line of Action is trying to achieve; and not about what you and your colleagues plan to do. c) How does your Main Line of Action relate to one or more of UNESCO’s strategic objectives? d) How do you see your Main Line of Action contributing to the two cross-cutting themes: the contribution of information and communication technologies to the development of education, science, and culture, and the construction of a knowledge society and the eradication of poverty,

especially extreme poverty Hint for c) and d): Consider what you identified as the broad goal (s) for your Main Line of Action and discuss how they can specifically contribute to broader goals enshrined in higher levels of programming. Discuss the real linkages between your MLA and the Major Programme ‘s Strategic objectives? Step 2: Setting Your Specific Expected Results - What Kind of Change Do You Intend to Achieve? NB At this step, early interaction with step 4 is highly recommended 12 Source: http://www.doksinet In Step 1 you identified the broad goals for your Main Line of Action. Now it is time to articulate your specific Expected Results - What you intend to change as a result of UNESCO’s programming or intervention? a) For your Main Line of Action what tangible change or changes are you seeking ? years? What kind of change(s) are you seeking within a frame of two years? What kind of change(s) are you seeking over a longer time frame of five or six Hint: In the C/5,

a series of expected results has been set out for your Main Line of Action. Review this list to see whether these “Expected Results” speak in terms of the changes to be achieved, or whether they speak only in terms of actions to be undertaken – which are outputs, and not results. If you find some “expected results” that are only outputs, consider what the action, through the activities it comprises, is designed to change / improve, and then re-articulate the “expected result” in terms of an anticipated achievement or a desired change. Are there additional expected results that could be added? b) How will these changes impact on UNESCO’s mainstreaming issues ? Step 3: Identifying the Broad Strategies and Modalities You Intend to Use The next step in this exercise is to discuss the strategies and modalities that you intend to use to give effect to the specific Expected Results you have just outlined in Step 2 for your Main Line of Action. a) Do you intend to adopt a

largely reflective, or a more active approach to achieving your Expected Results? In others words: Do you intend to undertake largely active programme delivery, or do you intend to largely focus UNESCO’s programming efforts and/or interventions on research and information gathering? If you intend to do both, what is the rough balance - more towards reflection, or more towards actions? b) Do you intend to work with others to achieve your Expected Results and if so, with whom ? What do you see as UNESCO’s primary roles in any partnership or joint action? What do see as the main role or roles for others ? Who is the lead organization, UNESCO or some other body ? Hint: Here the aim is to identify the partners in the change process, inside of UNESCO as well as external partners, and not the recipients of assistance or 13 Source: http://www.doksinet interventions unless they themselves have an active partnership role to play in the change process. Step 4: Your Stakeholders It is

very important to understand who the Stakeholders are (recipients and/or beneficiaries) of UNESCO’s programming efforts or interventions and the relationship of these Stakeholders to the attainment of the specific Expected Results you outlined in Step 2. a) For each of the specific Expected Results you outlined earlier, identify who will be the Stakeholders of UNESCO’s actions or interventions ? Hint: The direct Stakeholders of UNESCO’s actions may not necessarily be the same as those who have been identified in the Expected Result. For example, the direct Stakeholders of a capacity-building programme may be teachers or public administrators. But, if the change being sought relates to changing a specific condition within the general population – the beneficiaries – policy makers and the media may well appear as the key stakeholders, yet indirect Stakeholders, since their commitment might well be the key variable. Also, consider the medium term versus the longer term Expected

Results you have set. Are the Stakeholders the same ? b) Have you consulted with the Stakeholders about UNESCO’s plans? c) Do the Stakeholders have a role to play themselves in bringing about the achievement of the Expected Results ? In the medium term ? Over the longer term ? Which kind of a role? Step 5: Identifying Your Inputs a) Identify the financial and human resources that UNESCO will need to invest to achieve its Expected Results. Over what time frame ? Will the planned investment exceed one biennium ? b) Identify the human and financial resources being brought by others and those that may be contributed by the Stakeholders themselves. Are these other investments assured and/or linked firmly to those of UNESCO? What if any conditions have been placed on these investments of others? Are the investments of others clearly linked to UNESCO’s Expected Results? In other words: Will UNESCO’s primary investment result in a multiplier effect that contributes to the attainment of

UNESCO’s own Expected Results? 14 Source: http://www.doksinet Other than those investments currently planned for, are further investments from others or specific conditions needed to achieve the Expected Results ? Hint: At a time when collaborative efforts and partnerships are becoming more and more the rule, it is very important to understand the nature of the contributions that each partner brings to a combined effort. Equally importantly within the context of an RBM approach to planning and decision-making, it is vital to know whether the investments of others (extrabudgetary contributions) are being directed towards the same Expected Results as those set by UNESCO. Step 6: Your Outputs - What You Intend to Do to Achieve Your Expected Results The next step in this exercise may appear to be the simplest, but in some ways may be the most important in terms of developing a consistent logical connection between the Expected Results identified in Step 2, and their eventual

achievement. This issue of doing should be split in two very different questions : do WHAT ? do HOW? The first one is to be spelled in terms of OUTPUTS. The other leads to designing ACTIVITIES, the combination of which delivers these outputs in the best possible conditions of cost-effectiveness, quality, quantity, relevance, accuracy, timeliness. Moreover, you should discuss not only what you intend to do - namely your activities and the outputs they produce (and those of others) - but, more importantly, consider how and to what extent what you do will contribute to attaining your Expected Results. Very simply put, the question becomes: “How will your actions make a difference or promote the desired changes”? a) For each (if applicable) of the Expected Results you cited in Step 2, list the types of activities or interventions that UNESCO will undertake to achieve them. Hint: Be specific about listing all your planned outputs and those activities that produce such outputs - the

conferences, research, data collection, network building, publications, seminars, training and capacity-building activities, whatever. Your task here is get a firm understanding of the breadth of the programmatic activities within the various actions your Main Line of Action is made of, that respectively relate to your Expected Results. b) Do any other elements of UNESCO undertake activities or interventions in support of your Main Line of Action’s stated Expected Results ? If so, what are their planned outputs ? c) If you are working in partnership with other bodies to achieve UNESCO’s stated Expected Results, list what they intend to do ( What? and how? ) to further the achievement of the UNESCO Expected Results. 15 Source: http://www.doksinet Hint: In listing the outputs of other bodies, be sure that their outputs relate to UNESCO’s Expected Results. d) Now for each of the outputs you have listed, discuss in as much detail as possible how the output will actually “make a

difference”. In other words, explain how UNESCO’s outputs, and/ or those of others, cause or result in the changes that were identified in the Expected Results. Hint: You are being asked to show the logical connection between Output and Expected Result. For example, it is not enough to say: “We will hold a training program or publish a paper, therefore local capacity will be increased”. The issues really are: Will the training program or paper actually make a difference and how will it do so? What will UNESCO’s actions do for the Stakeholders - both the direct and the indirect Stakeholders? Do not be surprised if you are unable, in every instance, to show a logical connection between what you plan to do ( the activities, ending up into Outputs) and what you intend to achieve (your Expected Results). Exercises like this frequently result in uncovering inconsistencies and slippage between the two. This is one of the prime benefits of an RBM exercise like this, as it allows

staff, professionals and managers alike to sharpen the logic behind UNESCO’s programming and to maximize and concentrate UNESCO’s impact on sectoral and Organization-wide objectives, themes and priorities Step 7: Actual results and Indicators As was described earlier, the Main Line of Action is the level of UNESCO’s programming typology where there is sufficient critical mass to begin to get a sense of the progress you have made. So, don’t try to develop indicators for each and every one of your outputs. Rather, look at the totality of the expected results for your Main Line of Action and develop a few good, practical and reliable indicators that relate to the totality of the expected results of your MLA. Also recall that indicators are just signposts and that a good timely qualitative indicator can be far more useful for planning and reporting, and for enabling mid-course corrections over the life of a biennium , than waiting a long time for “perfect” third-party

statistical data. a) Consider the Expected Results for your Main Line of Action and discuss how to ascertain whether your programming activities and the outputs they deliver- are contributing to the attainment of these Expected Results. In essence, discuss what you would consider to be Performance Indicators that you will need in order to assess the achievements of the Main Line of Action. Consider this issue in the context of three time frames: Within one year or less, so as to permit mid-course correction Within the span of one biennium Within a longer time frame of five or six years. 16 Source: http://www.doksinet Hint: What you are being asked to do is develop the means to assess the extent of the change that has occurred. Different time frames will probably result in different means to assess the level of change that has occurred. For example, if the Expected Result is to change a condition within a society at large and the planned activities are a series of conferences,

capacity-building workshops, etc bringing about given outputs, the way you might be able to assess “what has happened” within the shorter time periods may be to ask those who participated in UNESCO’s activities - your direct stakeholders- what they did as a result of their participation within a UNESCO activity, or how they used UNESCO’s products. Over the longer time frames, you might wish to consider how to gauge the actual extent of change within the society as a whole. b) Now, for each of the indicators that you have identified, discuss how you intend to collect the information you need and the level of human and financial resources that you will require to do so. Hint: Recall that indicators are just that - “indications”. So, look for practical ways of getting information about outcomes by combining narrative sources, other qualitative data and statistical data when available. Also consider the frequency with which you will collect the data to ensure that there can be

a process of continuous improvement throughout the duration of a biennium. Remember, what you are looking for are the signposts of change and that change in itself is likely to be incremental. Also look for ways of collecting data that can be repeated time after time, so that you can begin to build a “baseline” and begin to see whether a continuum of change is evolving as a result of UNESCO’s work. 17 Source: http://www.doksinet Results-Based Programming, Management and Monitoring ( RBM) Guide For UNESCO 18 Source: http://www.doksinet September, 2001 RTC Consultants, Ottawa UNESCO Bureau of Strategic Planning ( BSP) 19 Source: http://www.doksinet 20 Source: http://www.doksinet Page -21-