Thing 4: Transparency

Think about a re-user of the research compendium and the specific information they would need in order to re-execute the analysis to generate expected results. Consider the questions below for each key aspect of research transparency.

Access transparency #

Access transparency requires that re-users of the research compendium are provided specific information about the location of the materials, protocols for accessing them, and their origin (i.e., provenance).

• Are explicit instructions provided on where and how to request and access materials not included in the compendium? If they cannot be made publicly available due to licensing or sensitive data restrictions, is this detail included?
• For research that involves secondary analysis of extant data or uses existing code, are original sources cited with a persistent identifier or stable URL that resolves to the specific data files used in the analysis?

Production transparency #

Production transparency is evident when details about the study design and research methods (e.g., sampling frameworks, response rates, validity testing) are clearly articulated.

• Are key aspects of the research design and analytic methods disclosed to allow for understanding and evaluation of the research process?
• Is information on how data were selected, captured, transformed, and analyzed provided in adequate detail to understand the choices made and steps taken by the researcher?

Analytic Transparency #

• Are the technical specifications of the computing environment, including operating system, software, and packages requirements (and their versions), provided?
• Is the process used to transform raw or original data into the analysis dataset made evident in the script files?
• Do script files contain non-executable comments that explain the purpose of codeblocks to enhance human readability of the analytical workflow?

Many scholars have been advancing the cause of transparency by highlighting its positive impact on scientific rigor and integrity while also describing the adverse effects of research that fails to disclose the processes that underpinned the results. Read the articles below to further understand the impetus for increasing transparency as a means to promote research reproducibility.

• Elman, C., Kapiszewski, D., & Lupia, A. (2018). Transparent social inquiry: Implications for social science. Annual Review of Political Science, 21, 29–47. https://doi.org/10.1146/annurev-polisci-091515-025429
  The authors of this article explain the value and practice of research transparency in terms of transparency’s three primary elements–data access, production transparency, and analytic transparency.
• Stark, P. B. (2018). Before reproducibility must come preproducibility. Nature, 557(7707), 613–613. https://doi.org/10.1038/d41586-018-05256-0
  Philip Stark introduces the term “preproducibility” to describe the desired condition in which adequate information is provided to enable individuals to repeat an experiment or analysis.

Transparent research requires actionable strategies for exposing the knowledge-making processes that are often omitted from the published research article. Learn about some of these proposed strategies from the resources below.

• Miguel, E., Camerer, C., Casey, K., Cohen, J., Esterling, K. M., Gerber, A., Glennerster, R., Green, D. P., Humphreys, M., Imbens, G., Laitin, D., Madon, T., Nelson, L., Nosek, B. A., Petersen, M., Sedlmayr, R., Simmons, J. P., Simonsohn, U., & Van der Laan, M. (2014). Promoting transparency in social science research. Science, 343(6166), 30–31. https://doi.org/10.1126/science.1245317
  This article makes the case for transparency in social science research and describes three specific practices–disclosure, registration and analysis plans, and open data and materials–that can increase transparency in scientific reporting.
• Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., Contestabile, M., Dafoe, A., Eich, E., Freese, J., Glennerster, R., Goroff, D., Green, D. P., Hesse, B., Humphreys, M., … Yarkoni, T. (2015). Promoting an open research culture. Science, 348(6242), 1422–1425. https://doi.org/10.1126/science.aab2374
  This article introduces the Transparency and Openness Promotion (TOP) guidelines, which looks to scholarly journals to promote the principles of transparency, openness, and reproducibility by issuing policies that require authors to engage in transparent research practices.
• EQUATOR Network. (n.d.). Enhancing the QUAlity and Transparency Of health Research. https://www.equator-network.org/
  The EQUATOR Network offers a searchable online database of almost 500 reporting guidelines for various types of research studies along with guidance on how to choose from among the guidelines. -See also Thing 5: Documentation to learn how transparency is made evident by producing tangible documentation that records essential details of the research workflow and Thing 7: Provenance to understand the importance of tracking the origin, chain of custody, and ownership of research materials.

These resources go into more depth about research transparency by presenting disciplinary case studies, tools, and initiatives that promote transparency and specific situations in which transparency was crucial for addressing scientific challenges.

• Jacobs, A. M., Büthe, T., Arjona, A., Arriola, L. R., Bellin, E., Bennett, A., Björkman, L., Bleich, E., Elkins, Z., Fairfield, T., Gaikwad, N., Greitens, S. C., Hawkesworth, M., Herrera, V., Herrera, Y. M., Johnson, K. S., Karakoç, E., Koivu, K., Kreuzer, M., … Yashar, D. J. (2021). The Qualitative Transparency Deliberations: Insights and implications. Perspectives on Politics, 19(1), 171–208. https://doi.org/10.1017/S1537592720001164
  While the focus has been on quantitative research, transparency standards do indeed apply for qualitative research. This article discusses how transparency is operationalized in qualitative research.
• Kidwell, M. C., Lazarević, L. B., Baranski, E., Hardwicke, T. E., Piechowski, S., Falkenberg, L.-S., Kennett, C., Slowik, A., Sonnleitner, C., Hess-Holden, C., Errington, T. M., Fiedler, S., & Nosek, B. A. (2016). Badges to acknowledge open practices: A simple, low-cost, effective method for increasing transparency. PLOS Biology, 14(5), e1002456. https://doi.org/10.1371/journal.pbio.1002456
  This article explains how applying badges to transparent research can incentivize scientists to make their research more transparent.
• The Center for Effective Global Action. (2020, June 23). Why COVID-19 makes research transparency more important than ever. Medium. https://medium.com/center-for-effective-global-action/why-covid-19-makes-research-transparency-more-important-than-ever-84071f3586be
  This article uses the COVID-19 pandemic as a case study for the importance of research transparency.
• Carsey, T. M. (2014). Making DA-RT a reality. PS: Political Science & Politics, 47(01), 72–77. https://doi.org/10.1017/S1049096513001753
  Read about how a group of political scientists advocated for research transparency in their discipline by developing the Data Access and Research Transparency (DA-RT) statement, which has been incorporated into the ethics guidelines of the American Political Science Association, and adopted by a large number of political science journals.