Resources for Practicing Open Science with Qualitative Research in Education
Overview
This list of resources consists of resources for researchers, editors, and reviewers interested in practicing open science principles, particularly in education research. This list is not exhaustive but meant as a starting point for individuals wanting to learn more about doing open science work specifically for qualitative research.
This list was compiled by the following contributors: Rachel Renbarger, Sondra Stegenga, Thomas, Sebastian Karcher, and Crystal Steltenpohl. This resource list grew out of a hackathon at the Virtual Unconference on Open Scholarship Practices in Education Research.
Resources for Practicing Open Science with Qualitative Research in Education
Why Open Science in Qualitative Research?
Open science is a well-known topic in quantitative research. In recent years, conversations on the need for open science practices in qualitative research and calls for attention to this issue are emerging. Many original conversations in open science were focused on reproducibility. However, in qualitative research, methodological principles differ from quantitative research due to the intent, theory, and premises of the research and data collection. This means that in many instances reproducibility may not be a focus or in fact might be contraindicated (see Twitter discussions on the topic).
Potential benefits of open science practices in qualitative research include improving transparency and rigor of methods, demystifying the analytic process, along with increasing collaboration and secondary analysis of qualitative data which overall can increase the trustworthiness of qualitative research (i.e., confirmability, dependability, credibility, transferability). Although we see these potential benefits, we also acknowledge cautions that need to be addressed (e.g., informed consent, confidentiality with sensitive data, avoiding compromising the rigor and methods of strong qualitative research by framing the research through a quantitative lens, etc.). Hence we hope this document provides resources and initiates scholarly discourse on this critical topic, especially within the field of education.
Purpose: This list of resources consists of resources for researchers, editors, and reviewers interested in practicing open science principles, particularly in education research. This list is not exhaustive but meant as a starting point for individuals wanting to learn more about doing open science work specifically for qualitative research. For more general information about open science research, please visit https://www.cos.io/.
Note: This resource list grew out of a hackathon at the Virtual Unconference on Open Scholarship Practices in Education Research. This resource list is shared under a Creative Commons- By Attribution 4.0 International license which requires attribution if used or shared. Questions or suggested edits can be sent to rachelrenbarger (at) gmail.com.
The suggested citation is:
Renbarger, R., Stegenga, S., Lösch, T., Karcher, S., & Steltenpohl, C. (2021). Resources for practicing open science with qualitative research in education. https://www.oercommons.org/courseware/lesson/80058/overview
General Guidance. These are resources that provide more context around qualitative open science research.
- Lorenz, T. K., & Holland, K. J. (2020). Response to Sakaluk (2020): Let’s get serious about including qualitative researchers in the open science conversation. Archives of Sexual Behavior, 49(8), 2761-2763. doi: 10.1007/s10508-020-01851-3
- Steinhardt, I. (2020). Learning open science by doing open science: A reflection of a qualitative research project-based seminar. Education for Information, 36(3), 263-279. DOI: 10.3233/EFI-190308
- Thread on Open Science in Qualitative (Experts, Resources across all Areas, etc) https://twitter.com/hansijzerman/status/1215634331454124038?s=20
- Experts on Open & Reproducible Qualitative Research https://twitter.com/OrbenAmy/status/1358824139360976904
- Views on comparing open science in qualitative and quantitative research https://twitter.com/CrystalNSYoung/status/1364210539484876812?s=20
Transparency & Rigor. Open qualitative research does not have to focus on hypothesis testing or reproducibility but emphasizes increasing information around the research process before, during, and after study completion.
- General Reporting.
- Qualitative reporting standards (General - not open-science focused):
- COREQ checklist: https://www.equator-network.org/reporting-guidelines/coreq/
- SRQR checklist: https://www.equator-network.org/reporting-guidelines/srqr/
- Aguinis, H., & Solarino, A. M. (2019). Transparency and replicability in qualitative research: The case of interviews with elite informants. Strategic Management Journal, 40, 1291–1315. https://doi.org/10.1002/smj.3015
- Kapiszewski, D., & Karcher, S. (2019). Transparency in practice in qualitative research. PS: Political Science & Politics, 1-7. doi:10.1017/S1049096520000955
- Moravcsik, A. (2014). Transparency: The revolution in qualitative research. PS: Political Science & Politics, 47(1), 48–53. https://doi.org/10.1017/S1049096513001789
- Qualitative Transparency Deliberations (QTD).https://www.qualtd.net/
- Pratt, M. G., Kaplan, S., & Whittington, R. (2019). Editorial essay: The tumult over transparency: Decoupling transparency from replication in establishing trustworthy qualitative research. Administrative Science Quarterly, 65(1), 1–19. https://doi.org/10.1177/0001839219887663
- Researcher Positionality
- Jacobson, D., & Mustafa, N. (2019). Social identity map: A reflexivity tool for practicing explicit positionality in critical qualitative research. International Journal of Qualitative Methods, 19, 1-12. https://doi.org/10.1177/1609406919870075
- Roberts, S. O., Bareket-Shavit, C., Dollins, F. A., Goldie, P. D., & Mortenson, E. (2020). Racial inequality in psychological research: Trends of the past and recommendations for the future. Perspectives on Psychological Science, 15(6), 1295–1309. https://doi.org/10.1177/1745691620927709
- Preregistration and Registered Reports. Qualitative researchers can be more transparent by outlining and submitting the study design and analysis plans before beginning the study (e.g., sampling plan, interview protocol, reflexivity statements). This is called preregistration. The term registered reports means that preregistration is completed for a specific journal to undergo peer review for later publication in the journal.
- Preregistration.
- Workshop Video on Preregistering Qualitative Research by Tamarinde Haven (https://www.youtube.com/watch?v=BNf9totTA_4)
- Blog on Preregistering Qualitative Work https://www.cos.io/blog/qualitative-preregistration
- Qualitative Preregistration Template https://osf.io/j7ghv/
- Haven, T. L., Errington, T. M., Gleditsch, K. S., van Grootel, L., Jacobs, A. M., Kern, F. G., ... & Mokkink, L. B. (2020). Preregistering qualitative research: A Delphi study. International Journal of Qualitative Methods. https://doi.org/10.1177/1609406920976417
- L. Haven, T., & Van Grootel, D. L. (2019). Preregistering qualitative research. Accountability in Research, 26(3), 229-244. https://doi.org/10.1080/08989621.2019.1580147
- Kern, F. G., & Gleditsch, K. S. (2017). Exploring pre-registration and pre-analysis plans for qualitative inference. Preprint. https://www.researchgate.net/publication/319141144_Exploring_Pre-registration_and_Pre-analysis_Plans_for_Qualitative_Inference
- Registered Reports.
- List of journals that accept Registered Reports https://katiedrax.shinyapps.io/cos_registered_reports/
- Journals that accept qualitative Registered Reports (excerpt of this list): AERA Open, Computer Science Education, Emerging Adulthood, Language Learning, PLOS ONE
Open Materials. A potential open science practice includes providing the research tools used during the study to individuals outside of the research study. Materials may include aspects of data and analysis, such as codebooks, memos, or software code/syntax. Sharing may be restricted or completely open depending on important considerations such as participant consent and IRB requirements.
- General Guidance.
- Elman, C., & Kapiszewski, D. (2014). Data access and research transparency in the qualitative tradition. PS: Political Science & Politics, 47(1), 43–47. https://doi.org/10.1017/S1049096513001777
- Methods Threads (Cautions, Ideas) - Open Access and Qualitative Research
- Methods thread on reproducibility and cautions in qualitative research https://twitter.com/jacasiegel/status/1358831397918695428?s=20
- Analysis.
- Antonio, M. G., Schick-Makaroff, K., Doiron, J. M., Sheilds, L., White, L., & Molzahn, A. (2020). Qualitative data management and analysis within a data repository. Western Journal of Nursing Research, 42(8), 640-648. https://doi.org/10.1177/0193945919881706
- Piñeiro, R., & Rosenblatt, F. (2016). Pre-analysis plans for qualitative research. Journal of Political Science, 36(3), 785-796. http://www.redalyc.org/pdf/324/32449207009.pdf
- Schindler, C., Veja, C., Hocker, J., Kminek, H., & Meier, M. (2020). Collaborative open analysis in a qualitative research environment. Education for Information. Preprint.
- Tools.
- R qual packages- data mining; text analysis; sentiment analysis; Welcome to RQDA Project
- Qualitative Research Analysis Using qcoder (R package) and Taguette (desktop application) https://zenodo.org/record/2673016#.YCVabGhKg2y
- Qualitative Data
- Data in qualitative research can be shared legally, even when not anonymized.
- When considering a place to share data, two considerations could be relevant:
- Anonymous data can be shared openly but personal data and especially sensitive data needs to be protected (e.g. by restricted access). Some of the repositories offer a restricted access (e.g. ICPSR), while others do not (e.g., OSF, Zenodo).
- Ideally, data should be shared FAIR, that is, findable, accessible, interoperable and reusable (see https://www.go-fair.org/fair-principles/). Some of the repositories explicitly comply with these norms.
- Sharing and Managing Data
- Managing Qualitative Social Science Data: An Interactive Online Course (with Diana Kapiszewski and Sebastian Karcher, commissioned by the Social Science Resource Council). https://managing-qualitative-data.org/
- Antonio, M. G., Schick-Makaroff, K., Doiron, J. M., Sheilds, L., White, L., & Molzahn, A. (2020). Qualitative data management and analysis within a data repository. Western Journal of Nursing Research, 42(8), 640-648. https://doi.org/10.1177/0193945919881706
- Bishop, L. (2005). “Protecting respondents and enabling data sharing: Reply to Parry and Mauthner.” Sociology, 39(2): 333–36. https://doi.org/10.1177/0038038505050542.
- Bishop, L. (2009). “Ethical sharing and reuse of qualitative data.” Australian Journal of Social Issues, 44(3): 255–72.
- Chauvette, A., Schick-Makaroff, K., & Molzahn, A. E. (2019). Open data in qualitative research. International Journal of Qualitative Methods. https://doi.org/10.1177/1609406918823863
- DuBois, M. (2020.) “Research participant views regarding qualitative data sharing.” Ethics & Human Research, 42(2): 13–27. https://doi.org/10.1002/eahr.500044.
- Jarman, B. (2020). Open data and sensitive interviews: Reflecting on ethics, consent, and reproducibility. Presentation. Retrieved from https://core.ac.uk/download/pdf/362178177.pdf
- Korkiakangas, T. K. (2014). Challenges in archiving and sharing video data: Considering moral, pragmatic, and substantial arguments. Journal of Research Practice, 10(1), 1-18. Retrieved from http://jrp.icaap.org/index.php/jrp/article/view/454/350
- Mannheimer, S., Pienta, A., Kirilova, D., Elman, C., and Wutich, A. (2019). Qualitative data sharing: Data repositories and academic libraries as key partners in addressing challenges. American Behavioral Scientist, 63(5), 643-664. https://doi.org/10.1177/0002764218784991
- Mozersky, J., Walsh, H., Parsons, M., McIntosh, T., Baldwin, K., & DuBois, J. M. (2020). Are we ready to share qualitative research data? Knowledge and preparedness among qualitative researchers, IRB Members, and data repository curators. IASSIST Quarterly, 43(4), 952. https://doi.org/10.29173/iq952
- Roulston, K. Archiving qualitative data. (Blog). https://qualpage.com/2019/05/09/archiving-qualitative-data/
- Tamminen, K. A., Bohaker, H., Bundon, A., Gastaldo, D., Gladstone, B. M., Krmpotich, C., ... & Smith, B. (2020). Building and supporting the use of digital research infrastructure among qualitative researchers. White Paper. Retrieved from https://engagedri.ca/wp-content/uploads/2020/12/Tamminen-et-al-NDRIO-White-Paper-Qualitative-Research.pdf
- VandeVusse, A., Mueller, J., & Karcher, S. (2021, March 12). Qualitative Data Sharing: Participant Understanding, Motivation, and Consent. https://doi.org/10.31235/osf.io/ghkjz
- Data Repositories
- Lester, J. (2017, December 11). Episode 3: The Qualitative Data Repository & Dr. Sebastian Karcher [Audio podcast episode]. In Qualitative Conversations. AERA Qualitative Special Interest Group. https://podcasts.apple.com/us/podcast/episode-3-qualitative-data-repository-dr-sebastian/id1324213425?i=1000413594733
- International
- List of repositories: Home | re3data.org - find common discipline/topic repository
- Open Science Framework: https://osf.io
- Zenodo: https://zenodo.org/
- United States
- Qualitative Data Repository: https://qdr.syr.edu/
- ICPSR: https://www.icpsr.umich.edu/ **Has assistance for anonymizing data
- UK
- UK Data Service www.ukdataservice.ac.uk
- Germany:
- German Network for Educational Research Data: https://www.forschungsdaten-bildung.de/index.php?la=en
- Research Data Centre for Education: https://www.fdz-bildung.de/home?la=en
- Qualiservice: https://www.qualiservice.org/de/
- Research Data Centre for Higher Education Research and Science Studies: https://www.fdz.dzhw.eu/en
Open Access. To make research available to a wide audience, it can be published in a way that no fee or subscription is required to access scientific papers or books. Such a publication format is in contrast to a more traditional format, where interested readers need to pay for research products. With open access, only an internet connection is required to access scientific information.
- There are varying types of open access levels.
- Preprints. The term preprint refers to scientific publications that are published before the peer-review process is finished and before papers are published in a scientific journal.
- Chiarelli, A., Johnson, R., Pinfield, S., & Richens, E. (2019). Accelerating scholarly communication: The transformative role of preprints. Zenodo. https://doi.org/10.5281/ZENODO.3357727
Ethics. Ethical considerations must be made at every step of the research process. Researchers must consider participants’ rights and perceptions alongside requirements put into place by the institutional review board that approved the study. Many of the aforementioned articles discuss ethics alongside data sharing, transparency, and other open science practices, and will not be repeated in this section.
- Bishop, L. (2012). “Using archived qualitative data for teaching: Practical and ethical considerations.” International Journal of Social Research Methodology, 15(4): 341–50. https://doi.org/10.1080/13645579.2012.688335.
- Bishop, L. (2014.) “Re-using qualitative data: A little evidence, on-going issues and modest reflections.” Studia Socjologiczne, no. 3: 167.
- Jacobs, A., Büthe, T., Arjona, A., Arriola, L., Bellin, E., Bennett, A., . . . Yashar, D. (2021). The Qualitative Transparency Deliberations: Insights and Implications. Perspectives on Politics, 19(1), 171-208. doi:10.1017/S1537592720001164
- Kapiszewski, D., & Wood, E. (2021). Ethics, epistemology, and openness in research with human participants. Perspectives on Politics, 1-17. doi:10.1017/S1537592720004703
- Kirilova, D., & Kapiszewski, D. (2021.) "Optimizing openness in human participants research: Harmonizing standards for consent agreements and data management plans to empower the reuse of sensitive scientific data." Qualitative Data Repository. https://doi.org/10.5064/F6RQA7AQ. QDR Main Collection.
- Yardley, S. J., K. M. Watts, J. Pearson, and J. C. Richardson. (2014.) “Ethical issues in the reuse of qualitative data: Perspectives from literature, practice, and participants.” Qualitative Health Research, 24(1): 102–13. https://doi.org/10.1177/1049732313518373.