Assessing Gen Op Support

general operating support / vol.2

assessing the impact

geo critical issues series

assessing impact
1725 DeSales St. NW, Suite 404 / Washington, DC 20036
tel: 202.898.1840 fax: 202.898.0318 web: www.geofunders.org

Grantmakers for Effective Organizations is a coalition of more
than 350 grantmaking organizations committed to building
strong and effective nonprofit organizations. GEO promotes
grantmaking practices that improve nonprofit results.

the
More information on GEO and a host of resources and links
for grantmakers are available at www.geofunders.org.

© 2008 Grantmakers for Effective Organizations

This publication may not be reproduced without permission.
To obtain permission, contact GEO at 202.898.1840 or
info@geofunders.org. This publication is available in electronic
format at www.geofunders.org.

GEO would like to thank the following individuals for their
feedback on this publication:

■ Paul Brest, The William and Flora
Hewlett Foundation

■ Nancy Burd, The Burd Group

■ Jacob Harold, The William and Flora
Hewlett Foundation

■ Amanuel Melles, United Way of
Greater Toronto

■ Clara Miller, Nonprofit Finance Fund

■ Edward Pauly, The Wallace Foundation

■ Sam Reiman, McCune Foundation

g e n e r a l o pe r at i n g s u p p o rt vo l . 2 / a s s e s s i n g t h e i m pac t

introduction
Grantmakers increasingly are turning to general operating support to ensure that
nonprofits have the resources they need to achieve their goals. Grantmakers for
Effective Organizations documented this trend in an Action Guide published in
2007.1 However, many grantmakers still have reservations about shifting more of
their support for grantees into the “unrestricted” column.

Even though they may share GEO’s belief that these flexible dollars can boost
effectiveness and impact for nonprofits, many grantmakers struggle to answer a
question that has become a significant barrier to the wider use of general operating
support in philanthropy:

How are we going to measure its impact?

This supplement to GEO’s General Operating Support Action Guide was created to
demonstrate how some grantmakers are assessing the impact of general operating
support. While preparing this publication, GEO was reminded repeatedly of the
many schools of thought on this topic. Some in the field feel that rigorous assess-
ment of these grants is counterproductive and antithetical to the nature of general
operating support; the idea, they say, is to give this money to organizations that are
doing good work and then “get out of the way.”

Others argue that the jury is still out on whether, and in what circumstances,
general operating support can contribute to grantee effectiveness. While the
decision to provide these flexible funds can be supported by common sense and

1
General Operating Support: GEO Action Guide. Grantmakers for Effective Organizations. 2007. www.geofunders.org.

© G R A N T M A K E R S F O R E F F E C T I V E O R G A N I Z AT I O N S | 1

positive experience, clear-cut evidence of concrete benefits is lacking. As a result,
some say that grantmakers should adopt a more rigorous, evidence-based approach
to assessment. At the very least, the field of philanthropy needs to be candid about
the limitations of the prevailing forms of assessment.

GEO comes down somewhere in the middle of this debate. GEO is a strong
proponent of general operating support, based on our belief that grantmakers are
effective only to the extent that their grantees achieve meaningful results. While
GEO agrees that grantmakers should do everything they can to “get out of the way”
and ensure that they are not placing added burdens on grantees, we also understand
the desire for more information about the impact nonprofits can achieve as a result
of general operating support. For this support to gain wider acceptance, grantmak-
ers need to be able to talk about how it affects organizations, what it allows them to
do differently, and in what instances it delivers the best results for organizations and
communities alike. And that requires some level of assessment.

To evaluate general operating support, grantmakers should have a clear strategy
behind what they expect it to accomplish. For example, if a grantmaker gives
general operating support to build nonprofit capacity, that’s what evaluators will
assess. If the goal is to broaden grantees’ impact, then that assessment will require
a different set of questions.

2 | © G R A N T M A K E R S F O R E F F E C T I V E O R G A N I Z AT I O N S

This document is not a step-by-step guide to assessment of these grants, although
it offers various strategies and actions to consider. Rather, GEO has identified a
number of common themes that emerge in conversations with grantmakers about
this topic. We also have identified two prevailing approaches to assessment: one that
emphasizes pre-grant assessment and one that relies more on assessment during and
after the time the grant is made. We have divided our discussion into four parts:

1 Getting Started: Thinking Differently About Assessment

2 Before the Grant: Vetting Grantees and Clarifying Expectations

3 During and After the Grant: Impact Measures and Reporting

4 Eyes on the Prize: Keeping Things in Perspective

general operating
support defined :
GEO has defined general operating support as funding in support of a nonprofit
organization’s mission rather than specific projects or programs.2 Organizations can
use these funds at their discretion to cover an array of expenses, from program costs
and salaries to administration, office expenses, technology, training, fundraising and
marketing, and more.

2
General Operating Support: GEO Action Guide. Grantmakers for Effective Organizations. 2007. www.geofunders.org.


part

1

Getting Started
Thinking Differently About Assessment

........

g e n e r a l o pe r at i n g s u p p o rt vo l . 2 a s s e s s i n g t h e i m pac t

Compared with grants for specific programs or projects, general
operating support requires grantmakers to give up some control over where the
money goes. This does not mean, however, that grantmakers have to give up on
the expectation that their investments will yield demonstrable results. Rather,
grantmakers need to think about assessment in a different way.

This means changing the focus from program-level outcomes to the social impact
of the organization as a whole. The guiding questions become: How is the organiza-
tion delivering on its mission? How does the organization set goals to track its prog-
ress? And to what extent is general operating support contributing to its success?

Taking a Humble Approach
Grantmakers should keep in mind that the impact of any individual grant —
for program or operations — ultimately depends on the size of the grant in relation
to organization and program budgets. One grant may account for a relatively small
portion of the organization’s overall budget. For example, if the grantmaker is pro-
viding $25,000 to an organization with a budget of $1 million, it’s hard to pinpoint
exactly the result of that support.

Bottom line: “Claiming credit” is never an open-and-shut case for grantmakers.
After all, it is the nonprofit, not the grantmaker, that actually is doing the work.
Recognizing this, many grantmakers that have set out to track the impact of their
general operating support acknowledge that a little bit of humility is in order.


“It is the nature of this type “We are in the process of looking at what we can
of support that you are definitively say in terms of evaluating the grants
we are providing, but we understand the limita-
buying into the organization’s
tions,” said Shawn Mooring, program officer with
goals, as defined by the The Philadelphia Foundation.
organization itself.”
Even the Blue Shield of California Foundation,
PA U L B R E S T
one of the few grantmakers that has conducted
TheWilliam and Flora Hewlett Foundation
in-depth evaluations of how its general operating
support is impacting grantees, acknowledges the limitations. “I haven’t come across
any way to evaluate the impact of this work that gives you black-and-white data,”
said Brenda Solórzano, director of health care and coverage with the foundation.

This doesn’t mean grantmakers shouldn’t try to assess the impact of these funds,
only that they should keep in mind the difference between “attribution” and
“contribution.” The evaluation question should then become: “How did these
flexible dollars contribute to this organization’s success?”

Buying Into Grantee Goals
Grantmakers also should keep in mind that the primary purpose of providing gen-
eral operating support is to give grantees flexibility to pursue their goals as they see
fit. Therefore, imposing conditions on precisely how these funds will be spent —
or on precise outcomes that grantees will achieve with this support — is antithetical
to the true intent of general operating support.

As noted in the GEO Action Guide, grantmakers usually opt for general operating
support when their own goals are substantially aligned with those of the grantee,


and when the grantmaker’s own due diligence aﬃrms that the grantee has the
capacity to achieve its goals. It follows, then, that grantmakers let grantees
determine how those goals will be achieved.

“It is the nature of this type of support that you are buying into the organization’s
goals, as deﬁned by the organization itself,” said The William and Flora Hewlett
Foundation CEO Paul Brest in an interview. “You may press them on
indicators and how they will know when they have achieved their goals, but the
spirit is that you start where they are.”


part

2

Before the Grant
Vetting Grantees and Clarifying Expectations

........


When a grant is made in support of a specific program or project, the results
usually can be tracked in clients served, units of affordable housing built, or similar
measures. With general operating support, the goal is to support the broader mis-
sion of the organization. This can change how a grantmaker does due diligence and
up-front assessment. The focus on organizational as opposed to program outcomes
has prompted many grantmakers to take a more comprehensive up-front look at the
operations and mission of prospective grantees.

The Women’s Opportunities Re- “high performance standards”
source Center in Philadelphia has in leadership, management and
been working in recent years to other areas, based on the organiza-
diversify its funding, with the goal tion’s lifecycle stage (e.g., startup or
of securing more general operat- adolescent).3
ing support. “These flexible funds
“It was a thorough process, and
are so valuable to us,” said Lynne
these questions really made us think
Cutler, founder and president
about where we are going as an
of the nationally recognized
case organization,” said Cutler. The online
nonprofit, which works to help
study low-income individuals build
assessment was followed by a site
visit by The Philadelphia Foundation
their income and assets.
staff with the staff and board of
Among the grantmakers that have the nonprofit.
delivered flexible funds to WORC
Cutler doesn’t think the up-front
is The Philadelphia Foundation. In
work involved in securing the general
2007, the grantmaker awarded a
operating support was an unreason-
one-year general operating grant of
able amount. “It’s just a different
$50,000 to WORC after the nonprofit
focus where they’re taking a more
completed a detailed application.
in-depth look at the organization
The online questionnaire asked
as a whole, and not just a particular
WORC leaders to consider where
program,” she said.
the organization stood in relation to

3
To learn more about The Philadelphia Foundation’s eligibility requirements and to see its assessment tool,
visit http://www.philafound.org/page22339.cfm.


Performing Critical Due Diligence
Gary Yates, president and CEO of The California Wellness Foundation, said that
his organization’s Responsive Grantmaking program, which provides general operat-
ing support to nonprofits working to improve the health of underserved popula-
tions, is founded on a rigorous up-front assessment of potential grantees. “We do a
lot of hard due diligence and site visits,” he said, noting that the foundation “steps
out of the way” once the grant is made, keeping post-grant reporting requirements
to a minimum.

The Philadelphia Foundation, intent on building a high-performing nonprofit
sector, uses a rigorous due diligence process. The process includes a “lifecycle”
analysis, plus site visits with the board and staff. As described in GEO’s publica-
tion, Imagine, Involve, Implement: Transforming Grantmaker Practices for Improved
Nonprofit Results, the lifecycle analysis looks at four dimensions of organizational
capacity: leadership, adaptive capacity, management and operations (See sidebar
page 14).

Nancy Burd, formerly vice president for grantmaking services with the foundation,
said the self-assessment responses are intended as a self-reflective, diagnostic and
learning tool for nonprofits. By identifying capacity strengths and challenges, the
assessments also can guide the development of a successful applicant’s organiza-
tional effectiveness goals.

Using the lifecycle reference as a guide, the nonprofit selects a stage in which
they best fit and completes an application that asks specific questions about their
organizational practices aligned with the lifecycle. Then, the foundation conducts
in-depth site visits before making a final decision. The Philadelphia Foundation
provides general operating support grants of up to $50,000 — with individual
grants maxing out at 10 percent of an organization’s operating budget.

At the time the decision is made, according to Burd, The Philadelphia Foundation
feels confident enough to step back and let the grantee do its work as it sees fit.

“When we make a general operating support grant, we are supporting high-
performing nonprofits at a particular ‘age and stage’ that have proven themselves


“When we make a general to be learning institutions,” Burd said. “We are
operating support grant, investing in strong businesses and have every
reason to believe they will continue to address the
we are supporting high-
ordinary and extraordinary challenges that may
performing nonprofits at a arise.”
particular ‘age and stage’
Among the other providers that emphasize up-
that have proven themselves
front assessment over post-grant reporting is
to be learning institutions.” The Whitman Institute. “We view this as 100
NANCY BURD
percent unrestricted money,” said John Esterle,
Formerly with The Philadelphia Foundation The Whitman Institute’s executive director, noting
that the grantmaker is not interested in detailed
reporting on how the money was spent.

“As a proactive funder, we do a lot of homework before we meet with prospective
grantees. Mission alignment and leadership are key factors for us. We also ask grant-
ees to talk about what success will look like for them over the next year and how
they’ll know they achieved it,” said Esterle.

Setting Goals to Clarify Expectations
While many grantmakers are content with affirming that general operating sup-
port grantees share their goals and have the capacity to achieve them, others have
grantees spell out some of the specific goals and objectives that the support will help
them achieve, as well as the metrics they will use to assess their impact. This is the
“negotiated general operating support” model advocated by The William and Flora
Hewlett Foundation and detailed in the GEO Action Guide.

The California Wellness Foundation, for example, links all grants to three objec-
tives identified by the grantee. “We ask them, ‘What do you want to achieve with
this funding?’” explained Fatima Angeles, director of evaluation and organizational
learning with the foundation.

Similarly, nonprofits requesting community grants from the Saint Luke’s Foun-
dation in Cleveland, whether for program or general operating support, have to
complete a logic model spelling out short-term and long-term objectives.


“We return their model at the end of the grant with instructions to report on what
they were able to accomplish or learn with our support,” said Saint Luke’s Founda-
tion President and CEO Denise San Antonio Zeman. “It is great when they meet
their objectives, but it is equally valuable when they learn something in the process
that will improve their work going forward,”

Another proponent of the negotiated support model is the New York-based F.B.
Heron Foundation, whose grantmaking consists primarily of general operating
support for organizations seeking to build wealth among low-income populations.
With $318 million in assets and a staff of 14, Heron reaches an agreement with
general operating support grantees on a specific set of measurable objectives.

John Weiler, senior program officer with the foundation, said that all grant objec-
tives are based on the grantee’s own planning documents. The grantmaker is not
imposing its own expectations on the organization as a pre-condition for awarding
general operating support.

For The Edna McConnell Clark Foundation, general operating support is based
on detailed business plans developed by grantees. The plans include a set of per-
formance metrics (that the grantee chooses and the grantee’s board approves) that
both the grantmaker and the grantee can look at over time to assess how things are
going. Here’s an excerpt from one of the grantees’ plans:

By 2012, Youth Villages plans to expand its capacity 50 percent, increasing
the number of youth served each year from 9,500 to 15,000. In addition to
deepening its operations in North Carolina, Mississippi, Massachusetts, and
Alabama, Youth Villages plans to expand into two or three new states.4

In most cases, the foundation will connect the grantee with a third-party consultant
who can help the organization map out its future goals and then develop and refine
its plans to reach them. Foundation President Nancy Roob emphasized that the
business plans are not imposed on the organizations from on high. “We are clear
from the start that while we have a stake in the plans of their organizations, they are
the client. It is their plan,” Roob said.

4
For more information on The Edna McConnell Clark Foundation’s grantees and their business plans,
go to www.emcf.org/portfolio/grantees/index.htm.


Considering Lifecycle Stages
Susan Kenny Stevens, a consultant to Denver’s
Rose Community Foundation, advocates that
grantmakers and grantees take a “lifecycles”
approach to assessment, and, depending on the
nonprofit’s self-described stage, adopt capacity
B E F O R E T H E G R A N T:
improvement measures that are appropriate to that
stage. According to Stevens, author of Nonprofit What Grantmakers Can Do
Lifecycles: Stage-based Wisdom for Nonprofit Capacity,
nonprofits can be found at any one of seven Make sure you are picking the right
lifecycle development stages, from “idea” and partners for general operating support by
“start-up” through “growth” and “terminal.” Peter conducting thorough due diligence.6
York of the TCC Group, which provides evaluation Encourage nonprofits to develop business
and other consulting services to nonprofits and plans with clear goals. Offer consulting and
foundations, pointed out that organizations at other support as needed so they can create
different lifecycle phases might use general operat- these plans.
ing support in different ways. For example, smaller
Make sure grantees “own” their goals and
start-up organizations might tend to put more of
plans. Don’t impose your own vision or goals
these funds directly into programs, while more
on them.
mature nonprofits might devote more of the funds
to infrastructure and “organizational needs.”

The key is to understand where grantees are devel-
opmentally, York explained, and not to apply a one-
size-fits-all model to screening potential grantees
and tracking their progress. TCC Group developed
the Core Capacity Assessment Tool, an online self-
assessment for nonprofits that looks at four dimen-
sions of capacity as well as organizational culture.
Every organization using CCAT receives a “lifecycle
score,” along with a capacity-building plan based
in part on the organization’s place in a five-stage
lifecycle continuum.5

5
For more information, see www.tccccat.com
6
For more information on due diligence, see Liza Culick, Kristen Godard and Natasha Terk. The Due Diligence Tool:
For Use in Pre-Grant Assessment. Grantmakers for Effective Organizations. 2004. Available at www.geofunders.org.


The Philadelphia Foundation
Uses Self-Assessment
for Prospective General
Operating Support Grantees

The Philadelphia Foundation’s general operating support
application serves as a self-administered assessment, based
on the TCC group’s Core Capabilities Assessment Tool, that
helps nonproﬁt organizations identify capacity strengths and
challenges. In completing the full application for general
operating support, organizations rate themselves on a vari-
ety of organizational capacity elements. In this excerpt,
organizations are asked to rate their adaptive capacity.

For the complete assessment tool, see www.philafound.org.


T H E P H I L A D E L P H I A F O U N D AT I O N E X A M P L E :

Adaptive Capacity Tool
High Performance Standards Is this an organization practice? Detailed Example
(Yes/No) Provide a brief
explanation where appropriate.

Organization develops moderately broad
and deep connections with community
leaders, funders, and constituents and
learns about needs through these
relationships.

Organization undertakes a formal
self-assessment process annually (perhaps
using an organizational self-assessment
instrument), identiﬁes needs for improving
the management and governance of the
organization, and incorporates this thinking
into a strategic planning process.

Program staff develops simple systems
for gathering and using data about
programmatic outcomes.

Staff and board develop simple systems
for integrating and using data from needs
assessment, organizational assessment,
program evaluation, and other sources,
and how it relates to organizational
improvements.

Organization develops simple systems
for storing, organizing, disseminating,
and using its knowledge.

Organization develops connections with
other organizations and forges more
formal collaborations with some of them,
such as by coordinating program delivery
and sharing resources.


part

3

During and
After the Grant
Impact Measures and Reporting

........


Most grantmakers that provide general operating support do not require
grantees to prepare a detailed accounting of how their dollars were spent. Rather,
the focus of their tracking and assessment efforts rests on how well the organization
is achieving its goals and, if the grantmaker expects to see capacity improvements as
a result of the grant, measures of organizational capacity. According to Paul Brest of
The William and Flora Hewlett Foundation, this means looking at indicators that can
reveal the extent to which a nonprofit is making progress toward self-identified goals.

“Grantmakers and nonprofits should focus on substantive outcomes,” Brest said.
“At the heart of this is clear goals, an evidence-based strategic plan for achieving
them, and milestones for knowing whether the organization is on course and
having impact.”

The East Bay Asian Local Develop- “We are much stronger as an
ment Corporation is a 33-year-old organization because these flexible
community development corporation dollars allow us to build our staff
that builds affordable housing and and infrastructure and respond to
community facilities in and around neighborhood needs so that we can
Oakland, Calif., while also provid- have more of an impact on our
ing services to low-income community,” said EBALDC Executive
case residents. Director Lynette Lee.
study
Using general operating sup- According to Weiler at the F.B. Heron
port from the F.B. Heron Foun- Foundation, the impact of general
dation, the Evelyn and Walter Haas, operating support on EBALDC
Jr. Fund and others, EBALDC has itself shines through in the organiza-
expanded its staff, invested in com- tion’s enhanced fiscal strength, the
puter and software upgrades, offered increased focus of EBALDC’s board
enhanced professional development and staff on succession planning and
opportunities for all employees, and other strategic priorities, and its
launched new pilot programs. demonstrated leadership on
community development issues
on a statewide basis, as well as in
other measures.


Using Organizational Assessment Tools7
General operating support shifts the focus of measurement from specific program
or project outcomes to the organization’s achievements as a whole. Additionally, it
focuses on the organization’s capacity to continue to deliver results into the future.
At the Endowment for Health in New Hampshire, this means using the CCAT to
assess the overall capacity of grantees (and to track changes in capacity over time).

“It’s really about asking Similarly, the Blue Shield of California
the right questions and Foundation evaluates the impact of its general
operating support to community clinics by
engaging in a continuing
keeping the focus on what the clinics are doing
conversation with grantees to make their organizations stronger and more
about how this support is effective. A “baseline survey” in 2006 asked
questions about grantee activities and investments
impacting their capacity
in areas such as professional development for staff,
to do their work.” board development, strategic planning, technol-
SAMIN DADELAHI ogy, advocacy, financial management and more.
Wyoming Community Foundation The grantmaker plans a follow-up survey in 2008
to assess how things have changed.

“We’re giving these organizations funds they have a hard time getting anywhere
else. And the reason we’re doing it is because we want them to build capacity,”
said Solórzano. “So the evaluation question is whether that is actually happening,
whether they’re investing in areas like planning and board development, which we
know can make these organizations stronger and more stable.”

Of course, not all grantmakers have the capacity to conduct such wide-ranging
evaluations. But they can take steps to make the connection between their general
operating support and changes in grantee capacity. The Wyoming Community
Foundation, which awarded grants and scholarships totaling $2.5 million in 2006,
started providing general operating support in the past five years. Today, according
to Senior Program Officer Samin Dadelahi, more than half the foundation’s grants
are for general operating support.

7
For a comprehensive guide to organizational assessment for grantmakers, see A Funder’s Guide to Organizational Assessment:
Tools, Processes and Their Use in Building Capacity. Grantmakers for Effective Organizations and the Fieldstone Alliance,
co-publishers. 2005. Available for purchase at www.geofunders.org.


To assess the impact of this support, Dadelahi said the foundation asks grantees to
answer a set of “really simple questions” in their final reports, while providing other
information, including approved and actual budgets. Among the questions:
Did these funds strengthen the capacity of your organization — and how?

“It’s really about asking the right questions and engaging in a continuing conversa-
tion with grantees about how this support is impacting their capacity to do their
work,” Dadelahi said.

Grantmakers, however, should be wary of an overemphasis on self-reporting by
grantees. Evaluators regularly note that self-reports tend to be positively biased;
grantees, understandably, will often want to put the best light on their work.
Wherever possible, grantmakers should therefore couple any self-reporting with
third-party assessments — while keeping in mind the challenges inherent in
attributing specific outcomes to a specific stream of funding.

Encouraging Learning and Improvement
Traditional measures of organizational capacity and outputs (finances, board leader-
ship, clients served, etc.) often do not paint a complete picture of the impact of
general operating support. Another measure many grantmakers use can be stated as
a question: “What is the organization learning?” One of the crucial ways to assess
the impact of general operating support is to look at the extent to which organiza-
tions are using data and information from their ongoing work to improve outcomes
over time.

David Hunter, former director of evaluation and knowledge development with
The Edna McConnell Clark Foundation and now an independent consultant,
said that grantmakers providing general operating support should care about two
kinds of results. First, they want to see that the organization is getting stronger and
developing the capacity to do its work better and more effectively over time. Does
it have access to reliable funding streams? Is it tapping into other sources of unre-
stricted funding? Is there growing depth on the staff so that the chief executive has
the necessary support?


Second, grantmakers want to see that the organization is learning important
lessons about what does or does not work to achieve progress in its given field,
and that it can share that learning with others. Toward that end, Hunter encourages
grantmakers to work closely with grantees to build their capacity to track outcomes
through qualitative and quantitative measures. “We need to make absolutely certain
that these organizations don’t keep repeating trial-and-error mistakes,” Hunter said.

At the F.B. Heron Foundation, program staff members assess learning among
general operating support grantees by asking a standard set of questions about
how they are using data and information, and how the staff and board have
changed strategies, goals or both based on that information (See sidebar page 21).
The foundation’s goal, according to Weiler, is to gauge the grantee’s commitment
to continuous improvement in its work. A grantee’s use of data is one of the seven
criteria that program staff use to assess grantee progress and to inform decisions on
future support.

A similar commitment guides the work of the Icicle Fund, a grantmaker serving a
sparsely populated four-county area in north central Washington state. “These are
small nonprofits, many of them run on an all-volunteer basis,” said Joan Alway,
executive director of the fund. “They don’t have time to do a whole lot of reporting
or analysis.”

As a result, the focus of the Icicle Fund’s application and reporting requirements is
on “learning as opposed to monitoring,” Alway said. Grantees are asked at the start
what they want to learn in the course of their work. For example, one arts orga-
nization might want to test the viability of a new ticketing system, while another
nonprofit may be curious about how to get its board more engaged in fundraising.
Then, in a midyear and final annual report, as well as in conversations during the
course of the grant, the grantee can reflect on what it is learning.

“That is probably the most important thing we can encourage and do in this work:
help these organizations develop a culture that is self-reflective and intentional,”
Alway said.


F.B. Heron Foundation Tool Assesses
Organizational Learning Among Grantees
Program officers at the F.B. Heron Foundation use their Impact Spectrum Assessment Tool
to help determine the extent to which general operating support grantees are learning from
their ongoing work and using that learning to get better results. Using this tool as a guide,
program officers assess grantees’ current learning practices and place them on a continuum,
ranging from “aspiration” to “learning organization/continuous improvement.”

Organization Name: PPlan/Initial activities:
1. Leadership has identified the steps, time frame
and resources (e.g., financial, internal and
In making a process assessment of a grantee, external expertise) needed to assess impact.
staff should consider three components, each 2. Specific outcomes have been identified, along
of which has corresponding text below: with the mechanisms to verify them.
3. The organization has taken initial steps to
1. Commitment to tracking impact
implement the plan (though efforts may be
or outcomes;
limited to pilot projects, or target specific
2. Utilization of information/data; and
program areas).
3. Changes in how the group does its work.

PAspiration: PBroader system/strategies
implemented:
1. Management’s commitment to tracking
1. Resources have been mobilized to track
impact is unclear.
impact.
2. Some basic statistics (e.g., activities, service
2. Data and information on impact/outcomes
use) are compiled.
have been assembled, analyzed and reported.
3. Information about impact is anecdotal or
3. A system is being implemented that will
inconclusive.
provide consistent review of program impact.

PMindset: PLearning organization/continuous
1. There is clear management “buy-in” to the improvement:
importance of impact, and leadership can
1. Managers use data routinely in decision-
articulate the difference between units of
making and organizational development.
service and impact indicators/results.
2. Feedback mechanisms are in place for the
2. The organization has identified approaches
ongoing collection, analysis and use of
or examples of effective ways to document
information.
impact.
3. The organization cites examples of how it
3. Leadership articulates issues and problems
has changed due to impact analysis.
to be resolved to address impact.


Change Indicators:
ONE GRANTMAKER’S ANALYSIS

Organizations participating in Rose Community Foundation’s BOOST program,
which combined general operating support with capacity-building support,
reported a number of changes and “organizational firsts” that enhanced their
capacity to fulfill their missions. Looking across five areas of organizational capacity,
the grantees’ achievements offer insights on indicators that other grantmakers can
use to track the impact of their general operating support.

Mission and Programs
■ Clarified mission, vision and values
■ Eliminated programs that did not fit mission or were not financially viable
■ Improved and added programs to better align with mission, financial needs and
strategic direction
■ Implemented program evaluation
■ Expanded partnerships and collaborations

Management
■ Added new positions and hired people that were the right fit for the growth stage
■ Freed up the executive director’s time to focus on strategic issues
■ Created new branding and marketing materials
■ Implemented new succession plans, job descriptions, employee evaluation and
management teams
■ Paid more attention to financial management on a regular basis, and shared
financial information with managers

Governance
■ Implemented new job descriptions, committee structures and officers
■ Improved board recruiting processes
■ Added new board members
■ Conducted board self-assessments
■ Increased board engagement in fundraising and financial management


Financial Resources
■ Used financial management tools (such as
program-based budgets) to better understand
financial reality of each program
DURING AND
■ Established strong finance committee A F T E R T H E G R A N T:
■ Shared a “dashboard” of key financial informa-
tion regularly with the board What Grantmakers Can Do
■ Pursued new funding opportunities Work with grantees to identify
■ Hired development director for the first time “organizational indicators” that can point
■ Hired in-house financial staff for the first time to the impact of general operating support.

■ Used multiyear planning and budgeting Remember the difference between
■ Upgraded accounting software “attribution” and “contribution.” Don’t
expect definitive impact data, especially for

Administrative Systems relatively small grants.

■ Implemented new policies and procedures for Consider where organizations are in their
human resources, financial management and development. Benchmarks might be differ-
other areas ent for start-up vs. mature organizations, for
■ Strengthened technology: Web site, computers example.
and database upgrades
Focus on learning as an impact measure.
■ Purchased accounting and fundraising software Consider how general operating support
grantees are using data and information
to improve results over time.

Be specific with your questions. To elicit
useful information, avoid asking vague and
generic questions.

If you aren’t going to use it, don’t ask!
If you or your grantee cannot use the data
collected in response to your questions,
it is common sense not to ask.


part

4

Eyes on the Prize
Keeping Things in Perspective

........


In a series of focus groups as part of GEO’s Change Agent Project,8 nonprofit
leaders and grantmakers alike identified increased levels of general operating support
as one of the key changes grantmakers can make to improve nonprofit results. Other
surveys and reports have reached the same conclusion.9 General operating support
helps organizations become more effective.

As grantmakers consider how to assess the impact of general operating support, they
should make sure their assessments do not stand in the way of the broader goal of
improving the capacity of nonprofits to deliver meaningful results.

This means using assessment as a platform for promoting learning and continuous
improvement among grantees. It also means using assessment as a platform for
a stronger grantmaker-grantee relationship and ensuring that assessment doesn’t
become an added burden for grantees.

Christine Smith is president of Over people and still providing quality
the Moon Child Care in Dubois, Wyo. child care,” she said.
In 2007, the facility, which serves
At the end of the grant, Smith will
from 12 to 40 children at a time, was
put together a brief report for the
awarded its first general operating
Wyoming Community Foundation
support grant from the Wyoming
showing how the dollars were spent,
Community Foundation.
with the possibility of renewing the
case The $10,000 grant has pro-
study grant for another year. But the staff at
vided crucial support for the
the foundation doesn’t have to wait
organization.
until then to know what’s happening
“Our biggest deficit is in at Over the Moon.
operations,” Smith said, explaining
“I am in touch with the foundation
that Over the Moon cannot charge
a lot. We have a great relationship,”
“big-city fees for child care” in a small
Smith said. She noted that program
community like Dubois. She uses the
officer Dadelahi has been working
unrestricted funds to cover rent, sala-
closely with her to find other sources
ries and utilities. When asked to as-
of funding for the child care center.
sess the impact of general operating
One result: the Wyoming Community
support, she said it shows in the fact
Foundation recently awarded Over
that Over the Moon can keep going.
the Moon an additional $10,000 from
“We are still open and still employing
a donor-advised fund maintained by
the grantmaker.

8
For more information on GEO’s Change Agent Project, see GEO’s publication Listen, Learn, Lead: Grantmaker Practices
that Support Nonprofit Results, available for download at www.geofunders.org.
9
See, for example, Rick Cohen. A Call to Action: Organizing to Increase the Impact and Effectiveness of Foundation Grantmaking.
National Committee for Responsive Philanthropy. 2007. Available at www.ncrp.org.


Building a Better Grantmaker-Grantee Relationship
Understanding the impact of general operating support is about more than
identifying the right indicators and creating an efficient reporting and tracking
system. It’s also about opening up a more transparent, trusting relationship with
grantees. This, in turn, gives the grantmaker a better idea of the challenges these
organizations face, how the unrestricted dollars are helping (or not) and what
types of additional support grantees may need.

“We put a premium on developing strong working relationships with grantees,”
said Esterle of The Whitman Institute, which provides general operating grants of
$25,000 to $50,000 per year. Esterle argued that general operating support actually
can deepen the relationship between the grantmaker and the nonprofits it funds.
The reason: The act of giving these grants is founded on trust.

The Whitman Institute does not require formal reports from grantees. “I don’t see
any need for them to write a special report just for us. That isn’t a great use of their
time,” Esterle said.

Instead, The Whitman Institute is in regular contact with grantees — “an ongoing
conversation,” Esterle called it — about what they are learning and how they are
incorporating that learning going forward. Esterle added that the institute seeks op-
portunities to convene grantees so they can share what they are learning with each
other and make new connections.

Another grantmaker that believes deeply in general operating support as a plat-
form for a stronger relationship with grantees is the Endowment for Health in
New Hampshire. According to Vice President and Chief Operating Officer Mary
Kaplan, the endowment does not accept unsolicited proposals for general operat-
ing support but rather approaches prospective grantees to explore the possibility of
providing $25,000 per year in general operating support for three to five years.

The organization has a deliberate strategy aimed at building a relationship with
the grantee, starting with the expectation that endowment representatives (the vice
president of programs, the president and, ideally, one trustee) will attend a grantee
board meeting to discuss the grant and mutual expectations.


To keep tabs on what’s happening during the grants, the endowment visits with
the grantees’ boards of directors every year or two. “We want real feedback on how
things are going,” said Kaplan. She said the meetings generally produce good, hon-
est information about the impact of this support. Kaplan added that the Endow-
ment for Health is less interested in having “an audit trail” showing how the general
operating funds are being used than in developing a clear sense of the funds’ impact
on the organization’s ability to fulfill its mission. Toward that end, the endowment
has been working with an evaluation consultant to assess the impact of its pilot
general operating grant program, which began in 2005.

Avoiding Information Overload
While they are looking for useful information from grantees about the impact of
general operating support, grantmakers should be concerned about not creating
a significant amount of added work for grantees.

A recent report from Project Streamline — a collaborative initiative of the Grants
Managers Network, GEO and others — documented 10 ways in which grantmak-
ers’ current application and reporting requirements create “significant burdens on
the time, energy and ultimate effectiveness of nonprofit practitioners.” Number
six on the list, titled “Reports on the Shelf,” is about how grantmakers often ask
for information that isn’t of any real use to them or their grantees as they strive to
improve effectiveness.10

Clara Miller, president and CEO of the Nonprofit Finance Fund, questions
whether general operating support should (or can) be evaluated at all. “These
specialized measures, conversations, convenings, questionnaires and self-assessments
all add cost, reducing the ‘net grant’ for grantees,” she observed. “What we can
evaluate is the overall result of the grantees’ work (output), not the grant itself (in-
put).” She added that assessment of general operating support defeats the purpose
of providing these flexible funds.

“I believe general operating support is meant to support the terrific work a trusted
grantee is already doing,” Miller said. “The minute we ask for anything specific, or

10
Jessica Bearman. Drowning in Paperwork, Distracted from Purpose: Challenges and Opportunities in Grant Application and Reporting.
Project Streamline, a collaborative initiative of the Grants Managers Network. 2008. www.projectstreamline.org


“These specialized measures, demand anything more from a grantee than the
hard work of maintaining existing, high-quality
conversations, convenings,
programs and services, it’s not general operating
questionnaires and self-assess- support any more.”
ments all add cost, reducing
Others say grantmakers are justified in trying to
the ‘net grant’ for grantees.
understand the impact of their general operating
What we can evaluate is the support dollars — but that they should be care-
overall result of the grantees’ ful not to impinge on grantee effectiveness in the
work (output), not the grant process. Brest, for example, said there is a tendency
among grantmakers to always add to the reporting
itself (input).”
requirements on grantees without taking anything
CLARA MILLER away. He said the Hewlett Foundation currently is
Nonprofit Finance Fund weighing how to “ease the administrative burden”
of reporting on grantees and the foundation’s own
program staff. His ultimate goal is to develop a “dashboard-like” system that would
allow a program officer to glean how a grantee is faring by glancing at a condensed set
of indicators.

“We need to make sure we only ask for information that is important,” Brest said.

Hunter agreed, suggesting that grantees already should be collecting most of the infor-
mation that would help illuminate the impact of general operating support grants. In
fact, the true test of whether grantmakers are asking for too much, he said, is whether
they are asking for information that is of no use to the grantee itself as it works to
strengthen its capacity and effectiveness.

“If you are doing this well, you should never ask the grantee for data that it doesn’t
need in order to make good management decisions,” Hunter said.

These data can be useful to grantees in other ways as well. For example, good,
easy-to-grasp information about the impact of general operating support can help
nonprofits make the case for these types of grants from other funders.


Seeking Grantee Feedback
Assessing the impact of general operating
support has to be a two-way street. As grant-
makers ask questions of their grantees, they
also need to allow grantees to ask their own
questions and offer candid feedback about
what’s working and what could improve. As
GEO observed in the report Listen, Learn, EYES ON THE PRIZE:
Lead: Grantmaker Practices that Support Non-
profit Results, grantmakers’ reluctance to solicit What Grantmakers Can Do
grantee feedback can do real harm to their
relationships with grantees — and to overall Keep in touch with grantees. Ask what they
are learning in the course of their work — how
grantee effectiveness.11
it’s going, what challenges they’re

Some grantmakers incorporate opportunities for facing, etc.

feedback into their assessments of general oper- Don’t go overboard. Ask for information that
ating support. For example, the Endowment for grantees can provide with relative ease, and
Health is planning to bring together all general that is useful for them.
operating support grantees to reflect on the pro-
Use general operating support as a plat-
gram and to hear suggestions for improving it. form for providing other support. Offer
links to capacity-building support and other
Social Venture Partners Seattle has shown a sim- sources of funding.
ilar determination to gather grantee input. In
addition to providing general operating support Make evaluation a two-way street. Ask for

grants that average $45,000 per year over five feedback on your grantmaking practices and
procedures and how they’re helping (or not).
years, it offers strategic consulting conducted by
volunteer donors and paid consultants. It’s an Work with grantees as learning partners.
intensive process, and SVP Seattle is determined Create a joint grantmaker-grantee working
to keep checking in with grantees about how it’s group or advisory group to promote a shared
going and whether they feel it’s worth the effort. understanding of how to measure the impact
of general operating support and articulate
The latest “Investee Satisfaction Report” its values.
compiled by the grantmaker showed that the
nonprofits were generally pleased with their re-
lationship with SVP Seattle.12 At the same time,

11
Listen, Learn, Lead: Grantmaker Practices That Support Nonprofit Results. Grantmakers for Effective Organizations. 2006.
Available at www.geofunders.org.
12
See www.svpseattle.org/our-impact/advancing-nonprofits/SVP%20Portfolio%20Report%202006-2007.pdf.


the survey turned up concerns and suggestions about how SVP Seattle could do a
better job — for example, by being clearer with grantees about what is expected of
them during the grant and how to take full advantage of the grantmaker’s financial
and nonfinancial support.

“General operating support is a critical part of building a relationship of trust
with these organizations,” said Susan Fairchild, SVP Seattle’s grants and
advocacy manager.

conclusion
The Blue Shield of California Foundation’s 2006 evaluation of its effort to provide
general operating support to community clinics yielded important insights. Among
these: The foundation’s unrestricted grants were having a greater positive impact on
the operations of small clinics, which tend to serve a higher percentage of uninsured
patients, than on larger ones. Because the foundation places a priority on reaching
underserved people, this finding prompted it to change its grantmaking formula.
Now it gives larger grants to smaller clinics, which often struggle to cover expenses,
so they can continue to provide services to the uninsured.

In describing how grantmakers can go about strengthening grantee results,
GEO often asks the question, “Effectiveness for what?” A better question for the
purposes of this publication is “Assessment for what?” Why should grantmakers
concern themselves with assessing the impact of general operating support on
grantees’ capacity to fulfill their missions?

As the Blue Shield of California Foundation example shows, the assessment of these
grants can produce learning, which in turn can be used to broaden the social impact
of grantmakers’ work. Assessment also can provide the information, perspective and
stories grantmakers need to make the case for the broader use of general operating
support as a tool for improving nonprofit results.

GEO looks forward to hearing from other grantmakers about their assessment
practices, and to keeping the community informed as our learning continues.


appendix
Assessing General Operating Support
GRANTMAKER STRATEGIES

BEFORE THE GRANT
Grantmakers emphasizing upfront assessment can consider the following strategies.

■ Grantmaker conducts rigorous due diligence of prospective general operating
support grantees, which may include site visits with board and staff or other
in-person meetings to reach clarity on the organizations’ goals and plans. —
The California Wellness Foundation, The Philadelphia Foundation, The Whitman Institute

■ Grantee completes online self-assessment evaluating its operations against high
performance standards based on its lifecycle stage. — The Philadelphia Foundation

■ Grantee completes facilitated self-assessment that establishes a “capacity starting
point” that can be used as a benchmark to measure progress in the course of the
grant. — Rose Community Foundation

■ Grantmaker and grantee reach agreement on a speciﬁc set of self-identiﬁed,
measurable grant objectives. — The California Wellness Foundation,
F.B. Heron Foundation

■ Grantee submits detailed “business plan” to the grantmaker (often developed
with help of third-party consultant). This plan includes performance metrics that
the grantee intends to track over time. — The Edna McConnell Clark Foundation

DURING AND AFTER THE GRANT
Grantmakers emphasizing post-grant assessment can consider these approaches.

■ Grantmaker uses the TCC Group’s Core Capacity Assessment Tool to assess the
overall capacity of general operating support grantees and track changes over time.
— Endowment for Health

■ Grantmaker conducts “baseline survey” to learn more about grantee activities and
investments in areas such as professional development for staff, board development,


strategic planning, technology, advocacy, financial management and more. A follow-up
survey two years later assesses how things have changed.
— Blue Shield of California Foundation

■ Grantmaker requests grantees to orient reporting toward simple questions about
the effect of general operating support on organizational capacity.
— Wyoming Community Foundation

■ Grantmaker bases benchmarks and evaluation methods on the lifecycle stages of
general operating support grantees. — Rose Community Foundation

■ Grantmaker asks grantees specific questions about the extent to which they are
engaged in learning for continuous improvement. — F.B. Heron Foundation

■ Grantmaker explicitly asks grantees what they want to learn in the course of their work
— and then encourages reflection on learning during the grant. — Icicle Fund

EYES ON THE PRIZE
Grantmakers can use these approaches to ensure they are not placing unnecessary
burdens on grantees.

■ Grantmaker eschews formal reporting in favor of “ongoing conversations” with grant-
ees about what they are learning in the course of the grant. — The Whitman Institute

■ Grantmaker sets out to build strong relationships with grantees through staff
and board participation in grantee board meetings before and during the grant.
— Endowment for Health

■ Grantmaker convenes grantees to assess the impact of general operating support on
their operations and to hear suggestions for improvements. — Endowment for Health

■ Grantmaker surveys grantees to keep track of their progress and to pinpoint
concerns and suggestions. — SVP Seattle


We would like to extend a special thank-you to the
foundations that have supported GEO with major general
operating support grants over the past ﬁve years:

■ Bill & Melinda Gates Foundation

■ Charles and Helen Schwab Foundation

■ The David and Lucile Packard Foundation

■ The Edna McConnell Clark Foundation

■ Evelyn and Walter Haas, Jr. Fund

■ Ewing Marion Kauffman Foundation

■ Fannie Mae Foundation

■ Ford Foundation

■ The James Irvine Foundation

■ Surdna Foundation

■ The UPS Foundation

■ The Wallace Foundation

■ The William and Flora Hewlett Foundation

■ W.K. Kellogg Foundation


1725 DeSales St. NW, Suite 404 / Washington, DC 20036

tel: 202.898.1840 fax: 202.898.0318 web: www.geofunders.org

Design by Hamilton Hughes Design
Writing by William H. Woodwell, Jr.
Cert no. SW-COC-002504

Assessing Gen Op Support

More Related Content

Viewers also liked (20)

Similar to Assessing Gen Op Support (20)

Assessing Gen Op Support