Applied Open Science Talk GSA 2020
- Subject:
- Life Science
- Social Science
- Material Type:
- Lecture Notes
- Author:
- Jennifer Lodi-smith
- Yunhwan Lee
- Date Added:
- 08/03/2021
Applied Open Science Talk GSA 2020
Although there is clear potential to improve science and innovation systems through big data and open science, barriers still remain with respect to data sharing efforts. How can the available massive and diverse data collections be used and shared more efficiently to boost global research and innovation and improve care? What actions are needed to facilitate open access to research data generated with public funding?
The OECD is bringing together policy makers, funding agencies and researchers to tackle the issue of open access to data, focused around developing good practice and principles on data governance. Four case studies highlight best practice and identify barriers to progress.
Following an OECD-hosted consultation with the Ontario Brain Institute (OBI), the United Kingdom Medical Research Council (MRC), and the US Alzheimer’s Association, two concrete examples of global data sharing have been created. The first, focused on providing a wealth of open-source biomedical data for the community (deep data), builds upon GAAIN, the Global Alzheimer’s Association Interactive Network, and links eleven international partners through a federated network of data resources. The capability of this network is being extended significantly through connections with the French National Alzheimer’s Database (BNA), the European Medicines Informatics Framework (EMIF), and the Canadian based Longitudinal Online Research and Imaging System (LORIS). The second focused on linking big data approaches at the population level (broad data), is a complementary collaboration between the Canadian Consortium on Neurodegeneration in Ageing and the Dementias Platform UK to share and analyse large-scale complex population-wide datasets from up to 2 million individuals, including imaging, genomics and health data.
As a result, these collaborations will enable the aggregation of an unprecedented volume of individual and population-level data, offering an open science solution to help research to more efficiently tackle Alzheimer’s disease and related disorders.
In response to concerns about the replicability of published research, some disciplines have used open science practices to try to enhance the credibility of published findings. Gerontology has been slow to embrace these changes. We argue that open science is important for aging research, both to reduce questionable research practices that may also be prevalent in the field (such as too many reported significant age differences in the literature, underpowered studies, hypothesizing after the results are known, and lack of belief updating when findings do not support theories), as well as to make research in the field more transparent overall. To ensure the credibility of gerontology research moving forward, we suggest concrete ways to incorporate open science into gerontology research: for example, by using available preregistration templates adaptable to a variety of study designs typical for aging research (even secondary analyses of existing data). Larger sample sizes may be achieved by many-lab collaborations. Though using open science practices may make some aspects of gerontology research more challenging, we believe that gerontology needs open science to ensure credibility now and in the future.
SPECIAL ISSUE: PREREGISTERED STUDIES OF PERSONALITY DEVELOPMENT AND AGING USING EXISTING DATA (Volume 76, Issue 1, January 2021)
Preregistration is the process of specifying project details, such as hypotheses, data collection procedures, and analytical decisions, prior to conducting a study. It is designed to make a clearer distinction between data-driven, exploratory work and a-priori, confirmatory work. Both modes of research are valuable, but are easy to unintentionally conflate. See the Preregistration Revolution for more background and recommendations.
For research that uses existing datasets, there is an increased risk of analysts being biased by preliminary trends in the dataset. However, that risk can be balanced by proper blinding to any summary statistics in the dataset and the use of hold out datasets (where the "training" and "validation" datasets are kept separate from each other). See this page for specific recommendations about "split samples" or "hold out" datasets. Finally, if those procedures are not followed, disclosure of possible biases can inform the researcher and her audience about the proper role any results should have (i.e. the results should be deemed mostly exploratory and ideal for additional confirmation).
This project contains a template for creating your preregistration, designed specifically for research using existing data. In the future, this template will be integrated into the OSF.
Preregistration is the process of specifying project details, such as hypotheses, data collection procedures, and analytical decisions, prior to conducting a study. It is designed to make a clearer distinction between data-driven, exploratory work and a-priori, confirmatory work. Both modes of research are valuable, but are easy to unintentionally conflate. See the Preregistration Revolution for more background and recommendations.
For research that uses existing datasets, there is an increased risk of analysts being biased by preliminary trends in the dataset. However, that risk can be balanced by proper blinding to any summary statistics in the dataset and the use of hold out datasets (where the "training" and "validation" datasets are kept separate from each other). See this page for specific recommendations about "split samples" or "hold out" datasets. Finally, if those procedures are not followed, disclosure of possible biases can inform the researcher and her audience about the proper role any results should have (i.e. the results should be deemed mostly exploratory and ideal for additional confirmation).
This project contains a template for creating your preregistration, designed specifically for research using existing data. In the future, this template will be integrated into the OSF.
Special Issue: Innovation in Aging
The past decade has seen rapid growth in conversations around and progress towards fostering a more transparent, open, and cumulative science. Best practices are being codified and established across fields relevant to gerontology from cancer science to psychological science. Many of the areas currently under development are of particular relevance to gerontologists such as best practices in balancing open science with participant confidentiality or best practices for preregistering archival, longitudinal data analysis. The present panel showcases one of the particular strengths of the open science movement - the contribution that early career researchers are making to these ongoing conversations on best practices. Early career researchers have the opportunity to blend their expertise with technology, their knowledge of their disciplines, and their vision for the future in shaping these conversations. In this panel, three early career researchers share their insights. Pfund presents an introduction to preregistration and the value of preregistration from the perspective of “growing up” within the open science movement. Seaman discusses efforts in and tools for transparency and reproducibility in neuroimaging of aging research. Ludwig introduces the idea of registered reports as a particularly useful form of publication for researchers who use longitudinal methods and/or those who work with hard-to-access samples. The symposium will include time for the audience to engage the panel in questions and discussion about current efforts in and future directions for transparent, open, and cumulative science efforts in gerontology.