Grey Literature Vs White Papers for Systematic Reviews

You’ve probably heard about grey/gray literature in systematic reviews. If you’re looking for the difference between grey literature and white papers, then you’re in the right place. In this article, we will define grey literature vs white papers and discuss their suitability in systematic reviews and meta-analyses.

What is grey literature?

According to Jean Adams, Programme Leader and Professor (MRC Epidemiology Unit) at the University of Cambridge, and his colleagues, grey literature can be defined as documents not controlled by publishing companies. Since you’re probably going to use data from primary research in your systematic review, the expanded definition of grey literature could be a set of primary research articles not controlled by publishing companies.

Arsenio Paez, a clinical specialty lead of the IDEAL Collaboration at the University of Oxford, provided the following examples of grey literature: academic papers (including theses and dissertations), research and committee reports, government reports, conference papers, and ongoing research, among others. The Table below provides a comprehensive list of grey literature types, according to Richard Adams:

What is the purpose of grey literature in a systematic review?

The primary purpose of including grey literature in your systematic review is to address publication bias. This kind of bias arises when authors are propelled to publish only significant and positive findings. Grey literature, such as doctoral theses do not undergo the same process as published literature, therefore can publish both negative and positive findings, as well as significant and non-significant findings. This provides a balanced view of the evidence.

Publication bias can be detected when there is inconsistent findings between published and unpublished (grey) studies.

In another post, I emphasized the importance of searching Google Scholar as a source of grey literature. This is because grey literature, not being controlled by publishing companies, cannot be found in commercial databases such as EMBASE, Web of Science, and Scopus. Apart from Google Scholar, you can find grey literature using the following sources, as recommended by Mahood and colleagues: “online databases (e.g., ProQuest), Web search engines and websites, university and institutional repositories, library catalogues, as well as contacting subject specialists, handsearching and consulting reference lists of relevant documents.”

What is a white paper?

Before defining a white paper, let’s acknowledge the fact that many experts consider white papers as grey literature. Consider for example Kansas State University and UC Davis Library positions, whereby they explicitly indicate that white papers are one of the types of grey literature in academia. We take a similar position that white papers are categorized under grey literature. Let’s define them!

According to Arianna Psichas and colleagues from Costello Medical in Cambridge, UK, white papers are frequently published in medical literature. The researchers discovered that, unlike the conventional definition of white papers as policy documents presenting the government’s stance on complex issues, the white papers published in medical literature serve various purposes, including raising awareness, promoting alignment, providing education, and fostering collaboration.

How frequently are white papers used in systematic reviews?

So far, I have not come across an expert advising on the use of white papers in systematic reviews. Instead, what I have seen is a systematic review within a white paper. Consider the example of a white paper published by Barbara Pieper, who at the time was a Professor/Nurse Practitioner at the College of Nursing, Wayne State University, Detroit, and colleagues. What they did was publish a white paper to communicate their position on pressure ulcer pain management, and within this white paper, they conducted a rigorous systematic review to support their position.

To address the lack of knowledge about the use of white papers in systematic reviews, we conducted a search for systematic reviews that utilized white papers. I searched for the following terms on PubMed: (“systematic review” OR meta-analysis) AND “white paper.” A total of 31 results were found. After reviewing all of them, we did not come across any systematic reviews that explicitly mentioned the use of white papers. However, we did find over ten articles that were identified as white papers and utilized systematic review and meta-analysis methodologies to support their perspectives.

Grey Literature Vs White Papers: Recommendations for Systematic Reviews

Whenever possible, use grey literature in your systematic review or meta-analysis. On the other hand, you may need to be a bit cautious if you decide to include white papers in your systematic review. This is because so far, we couldn’t locate a single systematic review that has ever used a white paper. Our finding is consistent with Richard Adams, who after reviewing ten scientific manuscripts found that none listed white papers as an example of grey literature.

Also, white papers are generally used in communicating positions on complex issues. That means they mostly do not contain primary research data that may be eligible for inclusion in a systematic review. However, if you find a white paper publishing primary data and meeting quality guidelines and published in a reputable journal or source (search databases like EMBASE, Web of Science, and Scopus), you can definitely include it in your systematic review without hesitation.

Challenges of Using Grey Literature in a Systematic Review

Remember our position is that white papers are a type of grey literature. Therefore, challenges of using grey literature will encompass the challenges of using white papers. The challenges of using grey literature, as outlined by Arsenio Paez, are as follows:

i. Time-consuming to search in multiple sources. There are also few resources on grey literature search methodologies.

ii. Apart from theses and dissertation and conference abstracts, other types of grey literature may lack the methodological quality required to be included in a rigorous systematic review.

The challenges above may be tempting to abandon the idea of including grey literature in your review. But it may not be a wise decision. Paying the price is worth it because your review would have efficiently mitigated publication bias.

Conclusion

In conclusion, grey literature refers to documents that have not undergone the traditional journal publication process and are not controlled by publishing companies. It includes white papers, although they are less commonly used in systematic reviews due to the challenges associated with searching for them. Even when easily found, white papers are less likely to contain primary data.

References

1. Searching and synthesising ‘grey literature’ and ‘grey information’ in public health: critical reflections on three case studies. Link: https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-016-0337-y

2. Gray literature: An important resource in systematic reviews. Link: https://onlinelibrary.wiley.com/doi/10.1111/jebm.12266

3. Shades of Grey: Guidelines for Working with the Grey Literature in Systematic Reviews for Management and Organizational Studies. Link: https://onlinelibrary.wiley.com/doi/full/10.1111/ijmr.12102

4. Publication bias: what is it? How do we measure it? How do we avoid it? Link: https://www.tandfonline.com/doi/abs/10.2147/OAJCT.S34419

5. Searching for grey literature for systematic reviews: challenges and benefits. Link: https://onlinelibrary.wiley.com/doi/abs/10.1002/jrsm.1106

6. Pressure Ulcer Pain: A Systematic Literature Review and National Pressure Ulcer Advisory Panel White Paper. Link: https://www.medline.com/media/mkt/pdf/research/Wound-Skin-Care/pressure-ulcer-pain.pdf