This case study describes the educational use of an open dataset collected as part of a thousand mile research walk. The content connects to many hot topics including quantified self, privacy, biosensing, mobility and the digital divide, so has an immediate interest to students. It includes inter-linkable qualitative and quantitative data, in a variety of specialist and general formats, so offers a variety of technical challenges including visualisation and data mining as well. Finally, it is raw data with all the glitches, gaps and problems attached to this.

The case study draws on experience in two educational settings: the first with a group of computer science and interaction design masters students in class-based discussions run by the first author; the second a computer science bachelor's project supervised by the second author.

Ongoing technological developments have made it easier than ever before for scientists to share their data, materials, and analysis code. Sharing data and analysis code makes it easier for other researchers to re-use or check published research. These benefits will only emerge if researchers can reproduce the analysis reported in published articles, and if data is annotated well enough so that it is clear what all variables mean. Because most researchers have not been trained in computational reproducibility, it is important to evaluate current practices to identify practices that can be improved. We examined data and code sharing, as well as computational reproducibility of the main results, without contacting the original authors, for Registered Reports published in the psychological literature between 2014 and 2018. Of the 62 articles that met our inclusion criteria, data was available for 40 articles, and analysis scripts for 37 articles. For the 35 articles that shared both data and code and performed analyses in SPSS, R, Python, MATLAB, or JASP, we could run the scripts for 31 articles, and reproduce the main results for 20 articles. Although the articles that shared both data and code (35 out of 62, or 56%) and articles that could be computationally reproduced (20 out of 35, or 57%) was relatively high compared to other studies, there is clear room for improvement. We provide practical recommendations based on our observations, and link to examples of good research practices in the papers we reproduced.

This workshop demonstrates how using R can advance open science practices in education. We focus on R and RStudio because it is an increasingly widely-used programming language and software environment for data analysis with a large supportive community. We present: a) general strategies for using R to analyze educational data and b) accessing and using data on the Open Science Framework (OSF) with R via the osfr package. This session is for those both new to R and those with R experience looking to learn more about strategies and workflows that can help to make it possible to analyze data in a more transparent, reliable, and trustworthy way. Access the workshop slides and supplemental information at https://osf.io/vtcak/​.

Resources:

1) Download R: https://www.r-project.org/​
2) Download RStudio (a tool that makes R easier to use): https://rstudio.com/products/rstudio/...​
3) R for Data Science (a free, digital book about how to do data science with R): https://r4ds.had.co.nz/​
4) Tidyverse R packages for data science: https://www.tidyverse.org/​
5) RMarkdown from RStudio (including info about R Notebooks): https://rmarkdown.rstudio.com/​
6) Data Science in Education Using R: https://datascienceineducation.com/​

The webinar features Dr. Joshua Rosenberg from the University of Tennessee, Knoxville and Dr. Cynthia D’Angelo from the University of Illinois at Urbana-Champaign discussing best practices examples for using R. They will present: a) general strategies for using R to analyze educational data and b) accessing and using data on the Open Science Framework (OSF) with R via the osfr package. This session is for those both new to R and those with R experience looking to learn more about strategies and workflows that can help to make it possible to analyze data in a more transparent, reliable, and trustworthy way.

Beginning January 2014, Psychological Science gave authors the opportunity to signal open data and materials if they qualified for badges that accompanied published articles. Before badges, less than 3% of Psychological Science articles reported open data. After badges, 23% reported open data, with an accelerating trend; 39% reported open data in the first half of 2015, an increase of more than an order of magnitude from baseline. There was no change over time in the low rates of data sharing among comparison journals. Moreover, reporting openness does not guarantee openness. When badges were earned, reportedly available data were more likely to be actually available, correct, usable, and complete than when badges were not earned. Open materials also increased to a weaker degree, and there was more variability among comparison journals. Badges are simple, effective signals to promote open practices and improve preservation of data and materials by using independent repositories.

This case study describes a practical exercise developed for students in the School of Geography and Environmental Science at Monash University. The exercise is based around simple bioclimatic modelling techniques and designed for first-year university students of biogeography, ecology and climatology. It incorporates aspects of past, present and future climates and their impact on species distributions, particularly in Victoria, but could be easily modified to suit any part of Australia. The practical exercise has three main parts: the first is on animal distributions under current and future climates; the second concerns plant distributions in the past and present; and the third part looks at how rare and endangered species may respond to future climate change in alpine environments.

Dashboards for data visualisation, such as R Shiny and Tableau, allow an interactive exploration of data by means of drop-down lists and checkboxes, with no coding for the user. The apps can be useful for both the data analyst and the public.

DASHlink is a virtual laboratory for scientists and engineers to disseminate results and collaborate on research problems in health management technologies for aeronautics systems. Managed by the Integrated Vehicle Health Management project within NASA's Aviation Safety program, the Web site is designed to be a resource for anyone interested in data mining, IVHM, aeronautics and NASA.

Original data has become more accessible thanks to cultural and technological advances. On the internet, we can find innumerable data sets from sources such as scientific journals and repositories, local and national governments, and non-governmental organisations. Often, these data may be presented in novel ways, by creating new tables or plots, or by integrating additional data. Free, open-source software has become a great companion for open data. This open scholarship project offers free workshops and coding meet-ups (hackathons) to learn and practise data presentation, across the UK. It is made possible by a fellowship of the Software Sustainability Institute.

By encouraging and requiring that authors share their data in order to publish articles, scholarly journals have become an important actor in the movement to improve the openness of data and the reproducibility of research. But how many social science journals encourage or mandate that authors share the data supporting their research findings? How does the share of journal data policies vary by discipline? What influences these journals’ decisions to adopt such policies and instructions? And what do those policies and instructions look like? We discuss the results of our analysis of the instructions and policies of 291 highly-ranked journals publishing social science research, where we studied the contents of journal data policies and instructions across 14 variables, such as when and how authors are asked to share their data, and what role journal ranking and age play in the existence and quality of data policies and instructions. We also compare our results to the results of other studies that have analyzed the policies of social science journals, although differences in the journals chosen and how each study defines what constitutes a data policy limit this comparison.We conclude that a little more than half of the journals in our study have data policies. A greater share of the economics journals have data policies and mandate sharing, followed by political science/international relations and psychology journals. Finally, we use our findings to make several recommendations: Policies should include the terms “data,� “dataset� or more specific terms that make it clear what to make available; policies should include the benefits of data sharing; journals, publishers, and associations need to collaborate more to clarify data policies; and policies should explicitly ask for qualitative data.

Background. Attribution to the original contributor upon reuse of published data is important both as a reward for data creators and to document the provenance of research findings. Previous studies have found that papers with publicly available datasets receive a higher number of citations than similar studies without available data. However, few previous analyses have had the statistical power to control for the many variables known to predict citation rate, which has led to uncertain estimates of the “citation benefit”. Furthermore, little is known about patterns in data reuse over time and across datasets. Method and Results. Here, we look at citation rates while controlling for many known citation predictors and investigate the variability of data reuse. In a multivariate regression on 10,555 studies that created gene expression microarray data, we found that studies that made data available in a public repository received 9% (95% confidence interval: 5% to 13%) more citations than similar studies for which the data was not made available. Date of publication, journal impact factor, open access status, number of authors, first and last author publication history, corresponding author country, institution citation history, and study topic were included as covariates. The citation benefit varied with date of dataset deposition: a citation benefit was most clear for papers published in 2004 and 2005, at about 30%. Authors published most papers using their own datasets within two years of their first publication on the dataset, whereas data reuse papers published by third-party investigators continued to accumulate for at least six years. To study patterns of data reuse directly, we compiled 9,724 instances of third party data reuse via mention of GEO or ArrayExpress accession numbers in the full text of papers. The level of third-party data use was high: for 100 datasets deposited in year 0, we estimated that 40 papers in PubMed reused a dataset by year 2, 100 by year 4, and more than 150 data reuse papers had been published by year 5. Data reuse was distributed across a broad base of datasets: a very conservative estimate found that 20% of the datasets deposited between 2003 and 2007 had been reused at least once by third parties. Conclusion. After accounting for other factors affecting citation rate, we find a robust citation benefit from open data, although a smaller one than previously reported. We conclude there is a direct effect of third-party data reuse that persists for years beyond the time when researchers have published most of the papers reusing their own data. Other factors that may also contribute to the citation benefit are considered. We further conclude that, at least for gene expression microarray data, a substantial fraction of archived datasets are reused, and that the intensity of dataset reuse has been steadily increasing since 2003.

A number of publishers and funders, including PLOS, have recently adopted policies requiring researchers to share the data underlying their results and publications. Such policies help increase the reproducibility of the published literature, as well as make a larger body of data available for reuse and re-analysis. In this study, we evaluate the extent to which authors have complied with this policy by analyzing Data Availability Statements from 47,593 papers published in PLOS ONE between March 2014 (when the policy went into effect) and May 2016. Our analysis shows that compliance with the policy has increased, with a significant decline over time in papers that did not include a Data Availability Statement. However, only about 20% of statements indicate that data are deposited in a repository, which the PLOS policy states is the preferred method. More commonly, authors state that their data are in the paper itself or in the supplemental information, though it is unclear whether these data meet the level of sharing required in the PLOS policy. These findings suggest that additional review of Data Availability Statements or more stringent policies may be needed to increase data sharing.

In this deep dive session, Dr. Willa van Dijk discusses how transparency with data, materials, and code is beneficial for educational research and education researchers. She illustrates these points by sharing experiences with transparency that were crucial to her success. She then shifts gears to provide tips and tricks for planning a new research project with transparency in mind, including attention to potential pitfalls, and also discusses adapting materials from previous projects to share.

Objectives To identify and appraise empirical studies on publication and related biases published since 1998; to assess methods to deal with publication and related biases; and to examine, in a random sample of published systematic reviews, measures taken to prevent, reduce and detect dissemination bias. Data sources The main literature search, in August 2008, covered the Cochrane Methodology Register Database, MEDLINE, EMBASE, AMED and CINAHL. In May 2009, PubMed, PsycINFO and OpenSIGLE were also searched. Reference lists of retrieved studies were also examined. Review methods In Part I, studies were classified as evidence or method studies and data were extracted according to types of dissemination bias or methods for dealing with it. Evidence from empirical studies was summarised narratively. In Part II, 300 systematic reviews were randomly selected from MEDLINE and the methods used to deal with publication and related biases were assessed. Results Studies with significant or positive results were more likely to be published than those with non-significant or negative results, thereby confirming findings from a previous HTA report. There was convincing evidence that outcome reporting bias exists and has an impact on the pooled summary in systematic reviews. Studies with significant results tended to be published earlier than studies with non-significant results, and empirical evidence suggests that published studies tended to report a greater treatment effect than those from the grey literature. Exclusion of non-English-language studies appeared to result in a high risk of bias in some areas of research such as complementary and alternative medicine. In a few cases, publication and related biases had a potentially detrimental impact on patients or resource use. Publication bias can be prevented before a literature review (e.g. by prospective registration of trials), or detected during a literature review (e.g. by locating unpublished studies, funnel plot and related tests, sensitivity analysis modelling), or its impact can be minimised after a literature review (e.g. by confirmatory large-scale trials, updating the systematic review). The interpretation of funnel plot and related statistical tests, often used to assess publication bias, was often too simplistic and likely misleading. More sophisticated modelling methods have not been widely used. Compared with systematic reviews published in 1996, recent reviews of health-care interventions were more likely to locate and include non-English-language studies and grey literature or unpublished studies, and to test for publication bias. Conclusions Dissemination of research findings is likely to be a biased process, although the actual impact of such bias depends on specific circumstances. The prospective registration of clinical trials and the endorsement of reporting guidelines may reduce research dissemination bias in clinical research. In systematic reviews, measures can be taken to minimise the impact of dissemination bias by systematically searching for and including relevant studies that are difficult to access. Statistical methods can be useful for sensitivity analyses. Further research is needed to develop methods for qualitatively assessing the risk of publication bias in systematic reviews, and to evaluate the effect of prospective registration of studies, open access policy and improved publication guidelines.

This is the lab manual for the Fay Lab at the University of Massachusetts Dartmouth School for Marine Science and Technology. The focus of our work centers around developing interdisciplinary modeling approaches to extend the scope of applications for fisheries and ecosystem-based management. More about our group’s research activity can be found on our website.

This lab manual resource is intended to provide an overview for lab members and others about how we do our work, and our expectations for our team. It is also a space to document institutional knowledge and for important information about procedures and available resources.

The content for this book was developed as part of our group’s participation in the Openscapes Champions program. We are extremely grateful to and acknowledge Dr. Julia Stewart Lowndes’ role in helping shape how our lab both works and how we articulate our identity. 🙏 Thanks also to Ileana Fenwick and Stefanie Butland for their work moving the lab-manual from bookdown to Quarto.

Figshare is a repository where users can make all of their research outputs available in a citable, shareable and discoverable manner. Figshare allows users to upload any file format to be previewed in the browser so that any research output from posters and presentations to datasets and code, can be disseminated in a way that the current scholarly publishing model does not allow.

Where does our money go? Who are the beneficiaries of public funding? Which projects get funded? When and how do publicly funded projects deliver concrete results? Are they effective enough? The Scuola di OpenCoesione (ASOC) is an educational challenge and a MOOC (Massive Online Open Course), designed for Italian high-school students.
It has three main objectives. First, to find out how public money is spent in a given local area or neighbourhood; second, to follow the projects and investigate how they are progressing and what challenges are they facing and third, to involve local communities in monitoring the effectiveness of public investment.

This report covers funder data-sharing policies/practices, and provides recommendations to funders and others as they consider their own policies. It was commissioned by Robert Wood Johnson Foundation in 2017. If any comments or questions, please contact Stephanie Wykstra (stephanie.wykstra@gmail.com).

The last ten years have witnessed increasing awareness of questionable research practices (QRPs) in the life sciences, including p-hacking, HARKing, lack of replication, publication bias, low statistical power and lack of data sharing (see Figure 1). Concerns about such behaviours have been raised repeatedly for over half a century but the incentive structure of academia has not changed to address them. Despite the complex motivations that drive academia, many QRPs stem from the simple fact that the incentives which offer success to individual scientists conflict with what is best for science. On the one hand are a set of gold standards that centuries of the scientific method have proven to be crucial for discovery: rigour, reproducibility, and transparency. On the other hand are a set of opposing principles born out of the academic career model: the drive to produce novel and striking results, the importance of confirming prior expectations, and the need to protect research interests from competitors. Within a culture that pressures scientists to produce rather than discover, the outcome is a biased and impoverished science in which most published results are either unconfirmed genuine discoveries or unchallenged fallacies. This observation implies no moral judgement of scientists, who are as much victims of this system as they are perpetrators.