Updating search results...

Reproducibility

Agreement of research results repeated. Reproducibility, replicability, repeatability, robustness, generalizability, organization, documentation, automation, dissemination, guidance, definitions, and more.

183 affiliated resources

Search Resources

View
Selected filters:
Meeting the Requirements of Funders Around Open Science: Open Resources and Processes for Education
Unrestricted Use
CC BY
Rating
0.0 stars

Expectations by funders for transparent and reproducible methods are on the rise. This session covers expectations for preregistration, data sharing, and open access results of three key funders of education research including the Institute of Education Sciences, the National Science Foundation, and Arnold Ventures. Presenters cover practical resources for meeting these requirements such as the Registry for Efficacy and Effectiveness Studies (REES), the Open Science Framework (OSF), and EdArXiv. Presenters: Jessaca Spybrook, Western Michigan University Bryan Cook, University of Virginia David Mellor, Center for Open Science

Subject:
Applied Science
Computer Science
Information Science
Material Type:
Lecture
Provider:
Center for Open Science
Author:
Center for Open Science
Date Added:
08/07/2020
Meta-assessment of bias in science
Unrestricted Use
CC BY
Rating
0.0 stars

Numerous biases are believed to affect the scientific literature, but their actual prevalence across disciplines is unknown. To gain a comprehensive picture of the potential imprint of bias in science, we probed for the most commonly postulated bias-related patterns and risk factors, in a large random sample of meta-analyses taken from all disciplines. The magnitude of these biases varied widely across fields and was overall relatively small. However, we consistently observed a significant risk of small, early, and highly cited studies to overestimate effects and of studies not published in peer-reviewed journals to underestimate them. We also found at least partial confirmation of previous evidence suggesting that US studies and early studies might report more extreme effects, although these effects were smaller and more heterogeneously distributed across meta-analyses and disciplines. Authors publishing at high rates and receiving many citations were, overall, not at greater risk of bias. However, effect sizes were likely to be overestimated by early-career researchers, those working in small or long-distance collaborations, and those responsible for scientific misconduct, supporting hypotheses that connect bias to situational factors, lack of mutual control, and individual integrity. Some of these patterns and risk factors might have modestly increased in intensity over time, particularly in the social sciences. Our findings suggest that, besides one being routinely cautious that published small, highly-cited, and earlier studies may yield inflated results, the feasibility and costs of interventions to attenuate biases in the literature might need to be discussed on a discipline-specific and topic-specific basis.

Subject:
Applied Science
Biology
Health, Medicine and Nursing
Life Science
Physical Science
Social Science
Material Type:
Reading
Provider:
Proceedings of the National Academy of Sciences
Author:
Daniele Fanelli
John P. A. Ioannidis
Rodrigo Costas
Date Added:
08/07/2020
NIGMS Clearinghouse for Training Modules to Enhance Data Reproducibility
Read the Fine Print
Rating
0.0 stars

In January 2014, NIH launched a series of initiatives to enhance rigor and reproducibility in research. As a part of this initiative, NIGMS, along with nine other NIH institutes and centers, issued a funding opportunity announcement (FOA) RFA-GM-15-006 to develop, pilot, and disseminate training modules to enhance data reproducibility. This FOA was reissued in 2018 (RFA-GM-18-002).For the benefit of the scientific community, we will post the products of grants funded by these FOAs on this website as they become available. In addition, we are sharing here other relevant training modules developed, including courses developed from administrative supplements to NIGMS predoctoral T32 grants.

Subject:
Applied Science
Life Science
Physical Science
Social Science
Material Type:
Lecture
Provider:
NIH
Author:
National Institutes of Health
Date Added:
08/07/2020
OSF101
Unrestricted Use
CC BY
Rating
0.0 stars

This webinar walks you through the basics of creating an OSF project, structuring it to fit your research needs, adding collaborators, and tying your favorite online tools into your project structure. OSF is a free, open source web application built by the Center for Open Science, a non-profit dedicated to improving the alignment between scientific values and scientific practices. OSF is part collaboration tool, part version control software, and part data archive. It is designed to connect to popular tools researchers already use, like Dropbox, Box, Github, and Mendeley, to streamline workflows and increase efficiency.

Subject:
Applied Science
Computer Science
Information Science
Material Type:
Lecture
Provider:
Center for Open Science
Author:
Center for Open Science
Date Added:
08/07/2020
OSF Collections: supporting research discoverability and reuse
Unrestricted Use
CC BY
Rating
0.0 stars

The OSF Collections repository platform supports the discoverability and reuse of research by enabling the aggregation of related projects across OSF. With OSF Collections, any funder, journal, society, or research community can show their commitment to scientific integrity by aggregating the open outputs from their disciplines, grantees, journal articles, or more. Learn how research collections can foster new norms for sharing, collaboration, and reproducibility.

We also provide a demo of how OSF Collections aggregates and hosts your research by discipline, funded outcomes, project type, journal issue, and more.

Visit cos.io/collections to learn more.

Subject:
Education
Material Type:
Lesson
Provider:
Center for Open Science
Date Added:
03/31/2021
OSF In The Lab: Organizing related projects  with Links, Forks, and Templates
Unrestricted Use
CC BY
Rating
0.0 stars

Files for this webinar are available at: https://osf.io/ewhvq/ This webinar focuses on how to use the Open Science Framework (OSF) to tie together and organize multiple projects. We look at example structures appropriate for organizing classroom projects, a line of research, or a whole lab's activity. We discuss the OSF's capabilities for using projects as templates, linking projects, and forking projects as well as some considerations for using each of those capabilities when designing a structure for your own project. The OSF is a free, open source web application built to help researchers manage their workflows. The OSF is part collaboration tool, part version control software, and part data archive. The OSF connects to popular tools researchers already use, like Dropbox, Box, Github and Mendeley, to streamline workflows and increase efficiency.

Subject:
Applied Science
Computer Science
Information Science
Material Type:
Lecture
Provider:
Center for Open Science
Author:
Center for Open Science
Date Added:
08/07/2020
OSF in the Classroom
Unrestricted Use
CC BY
Rating
0.0 stars

This webinar will introduce how to use the Open Science Framework (OSF; https://osf.io) in a Classroom. The OSF is a free, open source web application built to help researchers manage their workflows. The OSF is part collaboration tool, part version control software, and part data archive. The OSF connects to popular tools researchers already use, like Dropbox, Box, Github and Mendeley, to streamline workflows and increase efficiency. This webinar will discuss how to introduce reproducible research practices to students, show ways of tracking student activity, and introduce the use of Templates and Forks on the OSF to allow students to easily make new class projects. The OSF is the flagship product of the Center for Open Science, a non-profit technology start-up dedicated to improving the alignment between scientific values and scientific practices. Learn more at cos.io and osf.io, or email contact@cos.io.

Subject:
Applied Science
Computer Science
Information Science
Material Type:
Lecture
Provider:
Center for Open Science
Author:
Center for Open Science
Date Added:
08/07/2020
On the reproducibility of science: unique identification of research resources in the biomedical literature
Unrestricted Use
CC BY
Rating
0.0 stars

Scientific reproducibility has been at the forefront of many news stories and there exist numerous initiatives to help address this problem. We posit that a contributor is simply a lack of specificity that is required to enable adequate research reproducibility. In particular, the inability to uniquely identify research resources, such as antibodies and model organisms, makes it difficult or impossible to reproduce experiments even where the science is otherwise sound. In order to better understand the magnitude of this problem, we designed an experiment to ascertain the “identifiability” of research resources in the biomedical literature. We evaluated recent journal articles in the fields of Neuroscience, Developmental Biology, Immunology, Cell and Molecular Biology and General Biology, selected randomly based on a diversity of impact factors for the journals, publishers, and experimental method reporting guidelines. We attempted to uniquely identify model organisms (mouse, rat, zebrafish, worm, fly and yeast), antibodies, knockdown reagents (morpholinos or RNAi), constructs, and cell lines. Specific criteria were developed to determine if a resource was uniquely identifiable, and included examining relevant repositories (such as model organism databases, and the Antibody Registry), as well as vendor sites. The results of this experiment show that 54% of resources are not uniquely identifiable in publications, regardless of domain, journal impact factor, or reporting requirements. For example, in many cases the organism strain in which the experiment was performed or antibody that was used could not be identified. Our results show that identifiability is a serious problem for reproducibility. Based on these results, we provide recommendations to authors, reviewers, journal editors, vendors, and publishers. Scientific efficiency and reproducibility depend upon a research-wide improvement of this substantial problem in science today.

Subject:
Biology
Life Science
Social Science
Material Type:
Reading
Provider:
PeerJ
Author:
Gregory M. LaRocca
Holly Paddock
Laura Ponting
Matthew H. Brush
Melissa A. Haendel
Nicole A. Vasilevsky
Shreejoy J. Tripathy
Date Added:
08/07/2020
OpenRefine for Social Science Data
Unrestricted Use
CC BY
Rating
0.0 stars

Lesson on OpenRefine for social scientists. A part of the data workflow is preparing the data for analysis. Some of this involves data cleaning, where errors in the data are identifed and corrected or formatting made consistent. This step must be taken with the same care and attention to reproducibility as the analysis. OpenRefine (formerly Google Refine) is a powerful free and open source tool for working with messy data: cleaning it and transforming it from one format into another. This lesson will teach you to use OpenRefine to effectively clean and format data and automatically track any changes that you make. Many people comment that this tool saves them literally months of work trying to make these edits by hand.

Subject:
Applied Science
Information Science
Mathematics
Measurement and Data
Social Science
Material Type:
Module
Provider:
The Carpentries
Author:
Erin Becker
François Michonneau
Geoff LaFlair
Karen Word
Lachlan Deer
Peter Smyth
Tracy Teal
Date Added:
08/07/2020
Open + Reproducible Research Workshop
Unrestricted Use
CC BY
Rating
0.0 stars

Topics covered:

Understanding reproducible research
Setting up a reproducible project
Understanding power
Preregistering your study
Keeping track of things
Containing bias
Sharing your work

Subject:
Applied Science
Information Science
Material Type:
Module
Author:
April Clyburne-Sherin
Courtney Soderberg
Date Added:
08/07/2020
The Open Research Lifecycle | Center for Open Science
Unrestricted Use
CC BY
Rating
0.0 stars

Open science reduces waste and accelerates the discovery of knowledge, solutions, and cures for the world's most pressing needs. Shifting research culture toward greater openness, transparency, and reproducibility is challenging, but there are incremental steps at every stage of the research lifecycle that can improve rigor and reduce waste. Visit cos.io to learn more.

Subject:
Education
Material Type:
Lecture
Provider:
Center for Open Science
Date Added:
03/18/2021
Open Science Manual
Conditional Remix & Share Permitted
CC BY-NC
Rating
0.0 stars

About This Document: This manual was assembled and is being updated by Professor Benjamin Le (@benjaminle), who is on the faculty in the Department of Psychology at Haverford College. The primary goal of this text is to provide guidance to his senior thesis students on how to conduct research in his lab by working within general principles that promote research transparency using the specific open science practices described here. While it is aimed at undergraduate psychology students, hopefully it will be of use to other faculty/researchers/students who are interested in adopting open science practices in their labs.

Subject:
Psychology
Social Science
Material Type:
Reading
Provider:
Benjamin Le
Author:
Benjamin Le
Date Added:
05/01/2018
An Open Science Primer for Social Scientists
Unrestricted Use
CC BY
Rating
0.0 stars

“Open Science” has become a buzzword in academic circles. However, exactly what it means, why you should care about it, and – most importantly – how it can be put into practice is often not very clear to researchers. In this session of the SSDL, we will provide a brief tour d'horizon of Open Science in which we touch on all of these issues and by which we hope to equip you with a basic understanding of Open Science and a practical tool kit to help you make your research more open to other researchers and the larger interested public. Throughout the presentation, we will focus on giving you an overview of tools and services that can help you open up your research workflow and your publications, all the way from enhancing the reproducibility of your research and making it more collaborative to finding outlets which make the results of your work accessible to everyone. Absolutely no prior experience with open science is required to participate in this talk which should lead into an open conversation among us as a community about the best practices we can and should follow for a more open social science.

Subject:
Social Science
Material Type:
Lesson
Author:
Eike Mark Rinke
Date Added:
06/21/2017
Open Science Toolbox
Unrestricted Use
CC BY
Rating
0.0 stars

There is a vast body of helpful tools that can be used in order to foster Open Science practices. For reasons of clarity, this toolbox aims at providing only a selection of links to these resources and tools. Our goal is to give a short overview on possibilities of how to enhance your Open Science practices without consuming too much of your time.

Subject:
Applied Science
Life Science
Physical Science
Social Science
Material Type:
Reading
Provider:
Uni Muenchen
Author:
Lutz Heil
Date Added:
07/10/2019
The Open Science Training Handbook
Read the Fine Print
Some Rights Reserved
Rating
0.0 stars

Open Science, the movement to make scientific products and processes accessible to and reusable by all, is about culture and knowledge as much as it is about technologies and services. Convincing researchers of the benefits of changing their practices, and equipping them with the skills and knowledge needed to do so, is hence an important task.This book offers guidance and resources for Open Science instructors and trainers, as well as anyone interested in improving levels of transparency and participation in research practices. Supporting and connecting an emerging Open Science community that wishes to pass on its knowledge, the handbook suggests training activities that can be adapted to various settings and target audiences. The book equips trainers with methods, instructions, exemplary training outlines and inspiration for their own Open Science trainings. It provides Open Science advocates across the globe with practical know-how to deliver Open Science principles to researchers and support staff. What works, what doesn’t? How can you make the most of limited resources? Here you will find a wealth of resources to help you build your own training events.

Subject:
Applied Science
Life Science
Physical Science
Social Science
Material Type:
Reading
Provider:
FOSTER Open Science
Author:
FOSTER Open Science
Date Added:
06/18/2020
Open Science: What, Why, and How
Unrestricted Use
CC BY
Rating
0.0 stars

Open Science is a collection of actions designed to make scientific processes more transparent and results more accessible. Its goal is to build a more replicable and robust science; it does so using new technologies, altering incentives, and changing attitudes. The current movement towards open science was spurred, in part, by a recent “series of unfortunate events” within psychology and other sciences. These events include the large number of studies that have failed to replicate and the prevalence of common research and publication procedures that could explain why. Many journals and funding agencies now encourage, require, or reward some open science practices, including pre-registration, providing full materials, posting data, distinguishing between exploratory and confirmatory analyses, and running replication studies. Individuals can practice and encourage open science in their many roles as researchers, authors, reviewers, editors, teachers, and members of hiring, tenure, promotion, and awards committees. A plethora of resources are available to help scientists, and science, achieve these goals.

Subject:
Applied Science
Life Science
Physical Science
Social Science
Material Type:
Reading
Author:
Bobbie Spellman
Elizabeth Gilbert
Katherine Corker
Date Added:
07/02/2018
Open Science in Latin America
Unrestricted Use
CC BY
Rating
0.0 stars

Note: This webinar was presented in Spanish. The slides presented during this webinar can be found here:https://osf.io/6qnse/ The slides presented during this seminar can be found here: https://osf.io/6qnse/ Este seminario web se centrará en el estado de la ciencia abierta en América Latina, desde los esfuerzos de los investigadores individuales para abrir sus flujos de trabajo, herramientas para ayudar a los investigadores a ser abiertos y nuevas redes e iniciativas prometedoras en ciencia abierta. Ricardo Hartley (@ametodico) es profesor de metodología de la investigación de la Universidad Central de Chile, investigador en biología de la reproducción y en comunicación - valoración del conocimiento. Organizador de las OpenCon Santiago 2016 y 2017 y embajador COS. Erin McKiernan es profesora del Departamento de Física, Programa de Física Biomédica de la Universidad Nacional Autónoma de México. También es la fundadora del Why Open Research? proyecto, un sitio educativo para que los investigadores aprendan cómo compartir su trabajo, financiado en parte por la Fundación Shuttleworth. Fernan Federici Noe es profesor asistente e investigador de la Universidad Católica de Chile y fellow internacional del OpenPlant Synthetic Biology Center, University of Cambridge. Fernan es miembro del Global For Open Science Hardware (GOSH) y TECNOx (www.tecnox.org).

Subject:
Applied Science
Computer Science
Information Science
Material Type:
Lecture
Provider:
Center for Open Science
Author:
Center for Open Science
Date Added:
08/07/2020
Open science challenges, benefits and tips in early career and beyond
Unrestricted Use
CC BY
Rating
0.0 stars

The movement towards open science is a consequence of seemingly pervasive failures to replicate previous research. This transition comes with great benefits but also significant challenges that are likely to affect those who carry out the research, usually early career researchers (ECRs). Here, we describe key benefits, including reputational gains, increased chances of publication, and a broader increase in the reliability of research. The increased chances of publication are supported by exploratory analyses indicating null findings are substantially more likely to be published via open registered reports in comparison to more conventional methods. These benefits are balanced by challenges that we have encountered and that involve increased costs in terms of flexibility, time, and issues with the current incentive structure, all of which seem to affect ECRs acutely. Although there are major obstacles to the early adoption of open science, overall open science practices should benefit both the ECR and improve the quality of research. We review 3 benefits and 3 challenges and provide suggestions from the perspective of ECRs for moving towards open science practices, which we believe scientists and institutions at all levels would do well to consider.

Subject:
Biology
Life Science
Material Type:
Reading
Provider:
PLOS Biology
Author:
Christopher Allen
David M. A. Mehler
Date Added:
08/07/2020