The NSF’s Broader Impacts Criterion

For researchers applying to the National Science Foundation, the motivation to consider ways to engage in education and outreach is very explicit – they can’t get funding without it.

The NSF’s “Broader Impacts” criterion stands side-by-side with its “Intellectual Merit” criterion as the two most essential frames for review of any proposal to any NSF directorate for research funding. The Intellectual Merit criterion addresses the potential of the proposed research project to advance the field in significant and important ways. Reviewers examine the rationale for the approach, its potential outcomes, its feasibility, the experimental design, management plan, and the qualifications of the team requesting the funding.

The Broader Impacts Criterion, or “BIC,” as it is sometimes known, addresses the larger societal context of the research, such as the recruitment and training of researchers, dissemination of research findings, the path from research to useful applications, attention to various societal implications of the research, and educational outreach to K-12, undergraduate, and the public. These can include efforts to recruit youth, women, and minorities to careers in science and technology, efforts to explore potential benefits and possible long-term implications of the research, efforts to engage community audiences in learning and discussion on the research and its impacts; and efforts to transfer knowledge to commercial enterprise and industry. In a sense, the BIC helps to remind program officers, review panelists, and researchers that the funds they steward are from the nation’s taxpayers and are meant to be an investment for the benefit our all our citizens and our future.

While, many researchers are intrinsically motivated to engage in education and outreach activities, it is often this bottom-line motivation – the requirement of funders – that tips the scale and turns intention into action.

Partnering with museums and other informal education institutions I think is really important and its been very useful for us, as you know the National Science Foundation is one example is an agency that requires outreach as part of their grants, and so this is a great way of taking care of that obligation while also hooking into the infrastructure that [science museums] provide.

– Mike Falvo, faculty, UNC Chapel Hill

The Broader Impacts requirement for NSF grants recognizes the need to go beyond just doing good research and to make the impact and results of our research known to a larger audience. At a fundamental level, this money is tax-payer money and hence we have an obligation to let the general public know what their money is being used for. Sharing new and exciting research is also a terrific way to get more K-12 students interested in pursuing careers in this area. The broader impact requirements should not be seen as a burden rather as an opportunity to invest in outreach activities.

– Amy Moll, faculty, Boise State University

Addressing the Broader Impacts Criterion
All proposals to NSF for research funding must include explicit plans to make meaningful contributions in one or more of the following five areas. A science museum partnership could be useful in most of these areas.

(1) Advance discovery and understanding while promoting teaching, training, and learning with innovative connections of research and education… [including] opportunities to involve undergraduate and high school students in research experiences [and] participate in the professional development of K-12 teachers…

(2) Broadening participation of underrepresented groups by involving members of underrepresented groups (women, African Americans, American Indians including Native Alaskans, Hispanics, Native Pacific Islanders, and persons with disabilities) in research and education activities at all levels. …[in order to increase] the pool of future talented educators and promising researchers.

(3) Enhance infrastructure for research and education by linking with scientists and programs to bring added value and enhance impacts of research activities. These can include “Advances in networking and cyber infrastructure give researchers novel ways and new opportunities for collaboration, for conducting research and education, and sharing their work.”

(4) Broaden dissemination to enhance scientific and technological understanding … [including] working with science centers on new materials research and education exhibits; assisting journalists with their stories on technical topics; and developing new art forms for communicating materials research to wider audiences; creating materials research related websites enhanced by engaging animations and movies to educate non-scientists and the public at large.

(5) Provide benefits to society by communicating to the public the excitement, benefits, and long term impacts of materials research and enhance public appreciation of the relevance of advanced materials research to the future and society.
[Excerpted from NSF-08062]

Meeting the Criterion with rigor
A 2008 “Dear Colleague” letter from the Director of the NSF Division of Materials Research, Dr. Zakya H. Kafafi, makes clear that the NSF expects that a broader impacts plan is to be conceived with intellectual rigor, referencing prior work that justifies its approach; it must involve people with appropriate expertise, and include measurable goals and means of assessment:

In light of NSF’s commitment to the broader impacts criterion, the proposer(s) should carefully consider ways to incorporate rigorous, meaningful and innovative broader impacts activities (e.g., broadening participation) that integrate with the research being proposed. It is expected that project activities related to broader impacts will be of the same caliber as those addressing the intellectual merit criterion. Contributions to broader impacts should be based on good scholarship, and be designed to achieve clearly stated goals and metrics, while possessing the appropriate expertise and resources available for implementation. [NSF 08-062]

The case for partnerships with science museums
Few research scientists have had the time or the necessary experience to become scholars in the field of informal science education or to effectively design and implement an education outreach plan with measurable impacts.

Some researchers have questioned the wisdom of tying BIC activities to research funding, arguing that research, teaching, and graduate student mentoring in a university is more than a full time job; and that scientists don’t necessarily have the skills or resources to design implement a rigorous outreach plan to broader audiences. And NSF is quite explicit the BIC components must be explicitly designed and integrated into each research proposal and not merely “synergistic,” i.e., separately-funded pre-existing or complementary activities that occur whether or not the grant-proposal is funded [Kafafi 2008].

It is easy to see why the PI of a proposed university-based research center would be interested in seeking a collaboration with a professional informal science education organization, such as a science museum, to develop a sound approach to addressing the broader impacts criterion.

When it works well
Here are three brief examples from that show how well these collaborations can work:

At the Museum of Science, we have had the fortunate opportunity to work with two NSF Nanoscale Science and Engineering Center directors who understand that a five to ten percent allocation of overall funding to initiatives that fall under the “broader impacts” portfolio for their center demonstrates to NSF a commitment that goes “beyond what is normally expected” for teaching and mentoring graduate students, and beyond a mere alignment with other parallel “synergistic” outreach activities. Five percent of a $5 -10 million, 5-year award can provide $50,000 -$100,000 per year to a science museum sub-awardee, enough to fund one to two full-time educators producing a plethora of creative informal science education programs, small exhibits, media, and special events, while still reserving funding for the on-campus activities of the Center’s education and outreach coordinator. Using this strategy, these partnering research centers have received very high marks each year from their NSF review panels. Bob Westervelt provided this excerpt from an NSF site visit panel:

The Center demonstrated a very strong commitment to its educational and outreach mission, and should be commended for introducing a truly outstanding program in collaboration with the Boston Museum of Science. This program serves as a focal point of the Center’s educational outreach activities by coupling the scientific expertise of the technical investigators at MIT and Harvard with the energetic and enthusiastic communicators of science and technology information at the Boston Museum. This program has the ability to expose a vast cross-section of people throughout the Boston area to the excitement of science and technology. The Center should also be applauded in their initial attempts to disseminate this information to other Museums throughout the country, which will further enhance national impact.

Similar commendations from NSF went to the MRSEC at Penn State, which provided a sub-award to The Franklin Institute in Philadelphia to develop hands-on nano demo kits with training and national dissemination to multiple science museums, as well as summer camp and high school initiatives. MRSEC Director Tom Mallouk provided this excerpt from a site visit review panel’s report:

Educational activities proposed within this MRSEC are of outstanding quality. It is clear that the MRSEC team considers the integration of research and education as one of the highest priorities and requires each MRSEC member to contribute. The activities are innovative and non-traditional and include partnership with the Franklin Institute science museum in Philadelphia, strong ties with new magnet school Science Leadership Academy, a summer science camp “Action Potential Science Experience”, K-12 teacher training, a strong REU program, and a special program to involve graduate students in outreach activities.

NSF program officers were also quite pleased about being able to post on the Foundation’s website news about two traveling exhibits on nanotechnology that were produced by the Sciencenter of Ithaca, NY, in association with Cornell University’s NSF Nanobiotechnology Center; both exhibitions began their national tours at Epcot Center in Florida, reaching a very large audience in a relatively short time.

Reports of NSF’s enthusiasm for the success of these and other highly effective science museum – research center collaborations has begun to spread throughout the broader research community, and an increasing number of science museums are experiencing calls from researchers making inquiries about possible collaborations to include in their upcoming proposals. The next section is about how science museums can best prepare themselves for receiving these inquiries.

Next, in Section Two: Building a Partnership will also explore the attributes of healthy partnerships and the stewardship practices that maintain them.

back | next: Building a Partnership…