Credibility and Use of Scientific and Technical Information in Science Policy Making: An Analysis of the Information Bases of the National Research Co

Credibility and Use of Scientific and Technical Information in Science Policy Making: An Analysis of the Information Bases of the National Research Co Credibility and Use of Scientific and Technical Information in Science Policy Making: An Analysis of the Information Bases of the National Research Co Project Summary Often researchers are disappointed by the limited extent to which peer reviewed STEM research seems to contribute directly to high level public policy decision-making. However, does the perception of the limited use of formal scientific and technical information (STI) accord with empirical reality? What types of information compete with STI for inclusion in science policy-making, the realm in which one might intuitively expect greatest receptivity? How does the choice of various types of information relate to the use and impacts of science policy reports and recommendations? While there is a prodigious literature on the use of formal information in decision-making, our focus is on the use of STI in science, technology and innovation (S&T) policy, a domain in which there is virtually no empirical literature. Our study will examine the use and impacts of STI, comparing its use to the use of alternative types and sources of information such as expert testimony, gray literature, raw data, personal experiences and anecdotes. Our focus is on a single, but arguably quite important, S&T policy domain: National Research Council (NRC) reports. This is an especially important target institution for analysis because NRC committees have extensive information access and resources, as well as decision-makers who are well equipped to deal with a variety of information types, including STI. To understand the information ingredients of high-level S&T policymaking and advice, we propose three avenues of investigation: (1) a quantifiable bibliometric analysis of a random, stratified sample of 70 NRC reports conducted from 2005-2012 supplemented with committee information (e.g. composition and frequency of meeting etc.) (2) semi-structured interviews of staff and committee members from 10 of the 70 randomly sampled NRC reports, and (3) real-time observational analysis of two NRC panels from inception to completion. We hypothesize that a variety of factors affect the amount, proportion and characteristics of STI use, including such factors as (a) committee attributes (e.g. discipline, information preferences, relevant experiences) of the chair, members, and staff, (b) the origins of the committee request, (c) group dynamics involved in the committee and (d) aspects of the policy environment related to the study topic. Since some of these variables are not easily captured by bibliographic techniques alone, we will rely on the semi-structured interviews for this information. Our participant-observer approaches will add additional internal validity to our bibliometric analysis. As a result of this approach, we expect to have a multi-perspectival view that will inform or analysis of the relation of information use to impacts. Intellectual Merit. The research advances knowledge of information use in public policymaking, particularly with respect to science, technology and innovation policy. Since there is essentially no systematic research on the topic (excepting some useful case studies) we will develop perhaps the first database on this obviously important topic. Since the research is theory driven, the approach we employ to examine NRC reports will be applicable to virtually any forum for STI-based decision-making. Broader Impacts. Often, STEM researchers have a strong desire to ensure that their best work informs public decisions and they are disappointed that it so rarely does. By the same token, public policy-makers often complain that the STI produced by STEM researchers is less useful for them, in both form and content, than they would wish. A major goal of this study is to bring together these divergent perspectives on the production and use of STI. Using our results we propose developing a workshop that will be presented at the national meetings of the AAAS. The findings will present lessons to researchers about how their work can have greater utility and to users about how to evaluate policyrelevant information.