Widespread use of National Academies consensus reports by the American public

Academic and Miscellaneous Report Use.

Scholarly users engaged in research, teaching, and student studying accounted for half of report use (48% of classified US comments). This share is higher than the 33% of comments associated with a school or university domain, because many downloaders provided a Gmail or ISP email address (2). Research accounted for 20% of report use, students studying for classes accounted for 15%, teachers using reports to prepare courses accounted for 10%, and school administration, for example, curriculum development, accounted for 2.7%. Subcomponents of these categories, each accounting for less than 1% of comments, included museum informal education (in teaching), debate and competition (students preparing for competitions like the National Ocean Sciences Bowl), and, in research, the following: literature review, historical research, writing books, and research governance (institutional review boards, institutional animal care and use committees, and laboratory safety or animal care). Fig. 1 plots the full results.

The pattern of report reading in academia largely correlates with the overall pattern, because scholarly use accounts for such a large share of overall usage. The correlation with the overall distribution of downloads across reports was high—teachers, 0.85; research, 0.80; and students, 0.77. Teachers most often downloaded the NGSS report (14% of teacher comments) and accounted for 36% of the comments associated with that report. Added to this were teaching administrators, developing curriculum for schools or districts, for example. They accounted for 2.7% of all comments but 17% of comments on the NGSS report. In total, teachers, teaching administrators, teaching coaches (professional development in schools), and student teachers account for 72% of the NGSS report users. Other reports downloaded by teachers included How People Learn or concerned the implementation of NGSS and STEM teaching. Given the strong emphasis in the Future of Nursing report on university education, it is not surprising that 61% of this report’s comments report use by students, often studying for BSN degrees, and 11% of student comments were associated with this report. The NGSS report is the fourth most common student download, because people studying for education degrees use it. The downloading pattern for research purposes was slightly less correlated with the overall pattern, because the Nursing and NGSS reports were the first and third most downloaded reports. The second most downloaded report for research purposes was The Health Effects of Cannabis, accounting for only 0.8% of research downloads. No single report accounted for more than 1% of research downloads, suggesting that research interest is spread evenly across reports. Thus, the Herfindahl index for concentration across reports is very low in the research category; see SI Appendix, Table S3.

Nineteen thousand comments did not describe report use and were classified as “other.” Almost 6,400 of these comments expressed gratitude. This type of comment was more common in the early years when free access was novel. No particular report was favored by those expressing gratitude. Another 2,800 comments provided opinions on issues but no indication of how the report would be used, that is, were rants. Ranters’ concerns included gun violence, evolution and creationism, genetically engineered crops, and cannabis. Seven thousand seven hundred comments involved misreading of the prompt by people who reported how long they expect to use the report (hours, days, months, etc.) or the device they read the report on (laptop, iPad, etc.). The distribution of these comments across reports was highly correlated with the overall distribution (0.87 and 0.96), indicating misreadings were random with respect to report contents.

Academic use of NASEM reports is expected, and we are not surprised to learn that teachers were very interested in science standards promulgated in NASEM reports. Some misinterpretation of the prompt was also inevitable and of little interest. The mystery is, Why are scientifically grounded reports, largely commissioned by federal agencies to meet their needs, of any interest to the general public? The taxonomy we develop below answers this question in detail, uncovering the wide variety of uses citizens make of high-quality scientific information. The 52% of US comments outside the scholarly and other categories fell into three broad areas: everyday information activity (17% of comments), governance work (18%), and self-directed learning, both professional (6%) and personal (10%); see Table 4. The activities are heterogeneous and cannot be encompassed by one theoretical framework. Of necessity, therefore, we are eclectic in our interpretation drawing on ideas of everyday information use, new public governance, and self-directed learning (13–16).

Everyday Information Activity.

The first broad category—information activity—involves the report as opposed to life activities focused on work or leisure goals. Information activity categories were required for comments that do not mention, the more interesting to us, life activities. Table 5 organizes the information activity categories using a taxonomy of everyday information use developed by Hektor (13). For example, a handful of people were downloading for their boss or, less often, a neighbor or friend. About twice as many people were uncertain how they intended to use the report, marking them as people browsing for information that they believe might prove useful, partly depending on the report’s contents. This small group of people, 0.20%, provided insight only into their first stage of information use, seeking out the report.

Table 5.

Taxonomy of everyday information use and associated comment categories

Open in a new tab

Somewhat more people volunteered that they were gathering information from the report. Report review included editors fact-checking other documents that referenced the report as well as those taking a somewhat skeptical stance to the report and reviewing its recommendations. The category “for information,” accounting for 1% of comments, refers to those gathering background information without stating a purpose. Referencing or citation checking refers to people gathering reference material or using the report for general reference. After NGSS and nursing, downloads here included handling chemical hazards in laboratories, care of laboratory animals, roundabouts, and dietary reference intakes. Monitoring, meaning intentionally turning to a familiar source, seems to encompass these uses. Monitoring accounted for 5% of comments.

Hektor applied the term “unfold” to the activity of directing attention toward information in order to take part in its contents. We applied this concept to those who said they were going to read the report (2%) and those whose purpose was to learn about the report’s topic (9%). Note that we did not use these information categories if a comment also mentioned a life activity. In such cases, we classified the comment into that life activity. In other words, these categories are residual, used when there was no alternative. Comments in these categories were distributed across reports randomly, indicated by a high correlation between the distribution of comments across reports in these categories and the overall distribution; see SI Appendix, Table S3. This suggests the categories convey minimal insight.

Exchanging information is the next, more involved information activity. In the NASEM data, four categories expressed the concept of interacting with others about the report. Sharing or discussing the report, most often with colleagues, accounted for 1.4% of use. Particular to this category was the fourth most shared report, the 2018 examination of sexual harassment of women in academia. Six percent of users of this report intended to share it. Preparing for task forces, advisory committees, or working groups represented 0.6% of report use. The sexual harassment report was the fourth most common in this category. Sharing reports and committee work are two life activities that are also part of governance, discussed below. Preparing for workshops and conferences accounted for another 0.5% of comments. Finally, journal and book clubs read NASEM reports; over 1,300 comments referred to this activity, representing both work and leisure activity. Book clubs read How People Learn, as well as many reports on STEM teaching.

Hektor’s instructing activity encompasses both professional development (3%) and public outreach (0.5%). Professional development involved, among other things, instructional coaches in school, certification examinations, and interprofessional education for health care providers. Most outreach was STEM outreach, high school programs, or community engagement work, and the most downloaded report was Communicating Science Effectively. Hektor’s final information activity is publish. In this activity, we find those using the report to prepare their own white paper, talk, or conference presentation (1%). These people most often downloaded health care reports. Also in “publish” were authors of books. About 6,400 of those conducting research mentioned it was for a book. Nonfiction authors were interested in a broad range of reports, most often concerned with incarceration, dying, NGSS, and the poor state of health in this country. Authors of fiction, most often science fiction, as well as a few visual artists appeared in the NASEM comments 700 times. They were most interested in understanding severe space weather events, forensic science, and reflecting sunlight to cool the planet.

Two final categories seemed to be information activities that did not fit into Hektor’s framework because they were not everyday use. Authors of NASEM reports, members of report committees, and NASEM staff who worked on reports accounted for about 6,900 comments. Librarians adding reports to their libraries accounted for 4,000 comments. Their third most downloaded report was Preparing the Workforce for Digital Curation.

Use in Governance.

Governance accounted for 17% of comments. Theoretically, the concept of new public governance directs our attention to the network of government, nonprofit, and civic organizations engaged in the collaborative solving of societal problems (14). For societal problems addressed in NASEM reports, the comments provide a window into the activities of this network of actors across a range of substantive issues. However, there is significant variation in how activities were described, with similar activities likely described differently by different people. At one extreme, some comments say just “government work.” At the other, a person provided the name of the nonprofit they worked for and/or the interorganizational committee they were preparing to present to and described the purpose of the committee’s strategic planning activity. Therefore, categories cannot be mutually exclusive or have clear boundaries. Having said that, report downloaders work in consulting (0.5%), government agencies (0.7%), and nonprofits (0.2%) and work on issues in transportation (Transportation Research Board reports) (1.7%), public health (1%), and environment (0.4%). They were planning and developing strategies, designing and managing programs, evaluating, developing guidelines and regulations, accrediting, helping veterans with disability claims, and working with legislators to develop policy, all of which accounted for 5% of the comments. Some categories discussed earlier as information activities also are governance activities—professional development, committee work, sharing with colleagues, preparing for meetings, and writing reports or presentations.

Self-Directed Learning.

Comments about applying knowledge in professional or personal life were notable for the sense of self-directed learning (15) they conveyed. The picture that emerges is of adults outside of any formal structure, even professional development courses, seeking to improve their job skills, to improve the quality of the clinical or pastoral care they provide, to improve the diet of their animals, or to improve the grounding of the universes they imagine in their science fiction novels. Others in the media sought to provide pointers to these self-education resources to the interested public.

Work-related use, with little to no further specification, accounted for 3% of commenting and was correlated with overall commenting (0.74). Often, however, additional information was provided. Doctors and nurses intending to apply the knowledge gained to their clinical work left 1.7% of comments and were most concerned with dying, cannabis, and improving diagnosis. A purpose associated with a firm, such as “for my business” or “market research,” amounted to 0.7% of comments (although 173,000 comments were associated with a .com domain). Beyond cannabis and NGSS, there was interest in Best Care at Lower Cost, genetically engineered crops, and cost–benefit of nonemergency medical transportation. Journalists writing about the report in an article or others mentioning it in websites and newsletters left 0.6% of comments. They were most concerned with genetically engineered crops, immigration, cannabis, and incarceration. Lawyers left 0.4% of comments when downloading NASEM reports such as Assessing Eyewitness Identification, Reference Manual on Scientific Evidence, and Strengthening Forensic Science. Farmers interested in reports such as Nutrient Requirements of Beef Cattle accounted for 0.1% of comments. Finally, pastors and chaplains, often working in hospitals, accounted for about 400 comments, downloading, for example, Dying in America and The Growth of Incarceration. Including the fiction writers mentioned earlier, in total, 6% of comments related to these professional uses.

Personal use accounted for 10% of comments, 8% being undifferentiated personal use concerned with cannabis, dying, genetically engineered crops, evolution versus creationism, and reducing gun violence. When people provided more detail, they were often helping family members (0.7%), most often helping children with math. Others were veterans working on a VA disability claim (0.4%) and using some of NASEM’s 20 reports on Agent Orange’s effects or other reports on the health effects of burn pits or high noise levels. Serious leisure activity accounted for about 4,300 comments. Serious leisure is a concept articulated by Stebbins (16) that describes unpaid activities engaged in by adults over a long time and often involving self-directed learning. Volunteer work, blogging, ham radio, and amateur astronomy enthusiasts used NASEM reports on, for example, frequency allocation, lost crops of the Incas, or astronomy, to pursue their interests. Improving skills is another well-known motivator for adult learning (15) that accounted for about 6,300 comments. The improving skills category holds comments like “personal professional development” that appeared to be self-directed learning not associated with a formal course (which would be classified as professional development). Retired people left at least 2,300 comments, sometimes divulging their age, often expressing a personal interest in keeping up with areas related to their former profession. Most downloaded reports concerned dying, cognitive aging, NGSS, and preventing dementia. Job transitions are another prompt to self-directed learning recognized in the literature (15). Over 4,400 comments mentioned preparation for job interviews, exploring career changes, or becoming familiar with a new job area. People dealing with illness and interested in the NASEM reports on chronic fatigue syndrome, pain, and cannabis left over 2,600 comments.