The notes about monitoring and evaluation at the Robert Wood Johnson Foundation are typical of the approach used in the not-for-profit program environment. Over time the methodology has become more and more academically sophisticated, but also more costly, more time consuming and arguably less and less relevant. Most of the analysis is done in a manner that makes it difficult to get a truly objective result, and it is rare for the result to be validated in any way. At the macro level, the question of whether use of resources has been effective in improving quality of life in society remains unanswered.

Assessing our impact

The Robert Wood Johnson Foundation believes that better information about the effectiveness and impact of our philanthropy leads to better decision-making and, as a result, more constructive social change in the areas in which we work. RWJF has led the field of program evaluation from our inception as a national philanthropy, both in commissioning rigorous evaluations by independent experts and by applying their findings to improve our programs.

The Foundation also has pioneered ways to report on our grants in formats intended for a range of audiences, such as our Program Results Reports and our annual anthology, To Improve Health and Health Care. These efforts, combined with an annual program assessment report, make up a family of tools that examine and communicate about the impact of our work.

RWJF values ongoing, rigorous examination of our work for several reasons:

* ... As stewards of private resources that we must use for the public good, our efforts to assess the results and lessons from our work help us understand and improve the ways we invest those resources to have the greatest impact.
* ... By communicating regularly and openly, we seek to be clear and transparent with the public about the progress and results of our grantmaking.

These multiple approaches to assessment are an important source of knowledge for our staff and Trustees as we commit ourselves to continual improvement and learning.

Evaluation of specific programs

Program evaluations are a cornerstone of our efforts to learn from our work. In any given year, RWJF conducts between 30 to 40 active program evaluations and invests about 5 percent of our annual grantmaking budget on evaluations. Evaluations are different in size and scope. They range from small program assessments that rely to a great degree on expert judgment, to multimillion dollar mixed-method evaluations of program outcomes. We design them to capitalize on the greatest opportunity for learning, given the nature of the program.

Evaluation of program strategies

To understand the impact of our large portfolios of grantmaking, which focus on a particular goal or set of goals, we set short-, long-, and medium-range targets in specific program areas such as health insurance coverage, quality and equality of health care, childhood obesity, and public health. At the same time, we develop performance indicators to measure progress toward those targets. Performance indicators aim to clarify the underlying strategic approach of our grantmaking in each area and provide the leadership of RWJF with short-term, intermediate and long-term benchmarks to indicate progress in implementing that approach. RWJF management and our Board of Trustees receive periodic reports on these indicators. We also develop retrospective assessments of a group of programs that were designed to have an impact on a specific problem.

Evaluation of RWJF's organizational performance

Each year, we examine our performance as an organization and present that assessment to our Board of Trustees. The report incorporates our performance indicators and commissioned surveys of our grantees, health policy experts and the public to discover the most pressing health and health care issues facing our nation, their perspectives about our work, and the ways they utilize the knowledge gained from our programs and research.

Program Results Reports

Since 1996, RWJF has been producing Program Results Reports on most of our funded projects and national programs. We commissions writers to produce reports to increase the transparency of RWJF's funding and promote learning from past work. Findings and lessons from reports on similar topics are combined into Program Results topic summaries.

To Improve Health and Health Care: The Robert Wood Johnson Foundation Anthology

RWJF supports a book series published annually by Jossey-Bass to disseminate what RWJF has learned from various aspects of its grantmaking. The writers include award-winning journalists, RWJF staff members and outside evaluators. The authors sift through written and other records, interview key players, and make site visits to produce an interesting, jargon-free chapters that let readers know why RWJF decided to fund the activity, what the program's activities were (or are, in the case of existing programs), what has been accomplished, how the program fit with RWJF overall strategy, and what lessons can be drawn.