Making a difference. Everyone involved with a nonprofit generally does so because they want to have an impact and make a difference in the world. This is the case for individuals working at nonprofits – whether employees or volunteers – and for individuals or organizations donating to nonprofits. An individual working or volunteering at a nonprofit wonders if their time and skills are being used effectively, having an impact and making a difference. Likewise, a donor is concerned that the funds or property contributed to a nonprofit are being used in a way to have the most impact and make a difference in the world.
To determine whether a particular nonprofit is having an impact or making a difference, they must evaluate whether they are achieving mission. Every nonprofit has a mission statement that defines the organization’s reason for existence, and this statement should be the basis for all strategy and programming. There are a number of different ways to measure an organization’s effectiveness at accomplishing its mission. Financial performance measures have been addressed in previous months. This Topic of the Month will cover Program Evaluation – whether a nonprofit organization’s programs are accomplishing its mission. Program effectiveness can be measured and evaluated at a number of different levels – inputs, outputs, outcomes and impact – and should be communicated to those interested in the nonprofit organization.
Every nonprofit organization should have a “vision” and “mission statement” – vision relates to the results the organization hopes to achieve and the mission relates to the purpose of the organization or its reason for existence. Both are important in framing the organization’s programs and measuring the effectiveness of programs. Fundamental mission questions for every nonprofit are:
- What does success – achieving mission – look like?
- How do we (and others) know if the organization is achieving mission?
Nonprofit organizations tend to use either a reactive or proactive approach to measure mission and evaluate their effectiveness. A reactive evaluation approach focuses on looking back at activities performed and determining what worked well, what didn’t, and what lessons were learned for next time. The generally more effective approach is proactive evaluation. Instead of reacting to past performance, proactive evaluation determines how to measure the success of the program activity before the activity happens. This approach provides not only the possibility of measuring progress as the mission’s strategy is implemented, but also
allows for the comparison of one nonprofit organization with others that provide similar services. In addition, planning for the evaluation prior to the activity is not only helpful in solidifying the program activity’s goals but it determines in advance what data should be collected.
Measure What Matters
Many nonprofit organizations collect and report various data about the organization’s programs and activities, but may not in fact be measuring the effectiveness of its programs relative to its mission. There are a number of ways to measure the effectiveness of a nonprofit organization. Generally, there are two types of data that can be collected in evaluating program effectiveness: quantitative and qualitative.
One type of data is not superior to the other and both are important to evaluate the success of a nonprofit’s programs.
Quantitative data is expressed through numbers. Because quantitative data is inherently objective, it is often used to clarify qualitative data, which is more subjective in nature. Qualitative data is expressed through stories or quotes, received from questionnaires, testimonies, and interviews. Data that is qualitative is generally subjective in nature and presents a more experiential sense of whether a program participant had a successful or positive experience in the organization’s program.
While both quantitative and qualitative measures are important, it is critical to focus on measuring what matters. Gather the right information.
Measuring the Mission: Using a Logic Model
The key to measuring the mission of the nonprofit organization is to break down the all-encompassing mission statement into smaller activities with measurable outcomes for each. Using a logic model can assist in the process.
The diagram below illustrates the nonprofit organization’s logic model – i.e., “What will the program do and what is the expected result?” It demonstrates what steps need to take place before the final objective is achieved.
Inputs: These are resources that are needed for the program to operate and include funding, space, volunteers, and staff. Some nonprofit organizations measure and report inputs – e.g., increase in the number of facilities or locations, the increase in the number of volunteers, etc.
Activities: Activities are the specific areas of programming and should directly relate to the funding outlined in the inputs section. Activities should also include those things that need to happen before the program is launched, i.e. pre-program activities such as hiring, training, recruiting staff, advertisement/awareness. Note that activities can be very simple or complex and may include many or few items depending on the nature and scope of the programs. The next step is to project or estimate your intended results. This is done through looking at outputs and outcomes.
Outputs: Outputs are what is produced by the program – e.g., activities implemented, work performed, products or services delivered. Outputs can be internal or external, are often quantitative in nature, and focus on “what we did” (internal output) and “who we reached” (external output). It is important to have an output for every activity. Qualitative outputs can be the feedback received from participants in the program.
Outcomes: Outcomes are the direct results or consequences of the outputs. This is also known as the difference between the past and the present as a result of the program activity and output. Note that not every output has an outcome, and that an outcome is not the final impact of the nonprofit’s programs.
Impact: Impacts are the final intended effects of the program, the ultimate measure of whether a nonprofit organization is effective in achieving its mission. Impacts represent the significant changes, improvements or benefits to society that have occurred as a result of the nonprofit’s programs.
Thinking through a logic model – the cause and effect of relationships between inputs, outputs, outcomes and impact – will highlight both what is needed to successfully run a nonprofit’s programs and how the nonprofit organization will measure its effectiveness. In developing each of these areas, it may be helpful to begin with the end in mind. That is, begin with the intended impact of the nonprofit’s programs– the final intended effects of the organization’s activity – and work toward understanding its inputs. For example, if you run a job- training program, and your desired impact is that program graduates will attain full-time employment, you can work backwards to develop the outcomes and outputs against which progress can be measured toward this goal. In this scenario, an output could be having each participant complete three mock interviews, and an outcome would be having each participant be interviewed for current open positions at five companies. From there, you can work backward again to determine the additional training through activities that needs to take place as well as the resources (inputs) that you need in order to complete the program.
Program evaluation can be a time-consuming process, but not necessarily so if the organization is focused on gathering the right information and has systems in place to evaluate the information. While there may be many things to measure, it is important to focus on the key strategic priorities and the relevant measures of success for those priorities. The appropriate level of analysis and measurement will vary depending on the size of the nonprofit, the number and scope of programs, and the requirements of it stakeholders.
Communicating Program Effectiveness
There are many stakeholders that are interested in whether a nonprofit organization is achieving its mission – donors, employees, volunteers, management, the Board of Directors, government, funding agencies, local officials, and the public at large. The nonprofit should consider the different groups of people and the information they may want to know about the organization. By understanding the motivation and interests of each group that is affiliated with the organization, the nonprofit organization will be able to focus on what is important to measure and what is important to communicate to each group to show that the nonprofit is having an impact, making a difference, and achieving mission.
Articles for Further Reading:
- Key Steps In Outcome This article provides a tutorial on how to create outcomes for your organization and also includes examples of other organizations’ outcomes. http://www.urban.org/UploadedPDF/310776_ KeySteps.pdf
- David Parmenter, Key Performance Indicators for Government and Non Profit Agencies: Implementing Winning KPIs (Hoboken, NJ: John Wiley & Sons, Inc., 2012).
- Marc J. Epstein and Adriana Rejc Buhovac, Management Accounting Guideline: Performance Measurement of Not- For-Profit Organizations (Mississauga: The Society of Management Accountants of Canada, New York: The American Institute of Certified Public Accountants, 2009).