ICS@PSU: Institute for CyberScience

News Archive

Published on May 22, 2017 by Matt Swayne

Data sharing can offer help in science’s reproducibility crisis

UNIVERSITY PARK, Pa. — Criticism that researchers in the psychological and brain sciences are failing to reproduce studies — a key step in the scientific method — may have more to do with the complexity of managing data, rather than an attempt to hide methods and results, according to researchers. However, without greater focus on reproducibility, scientists will likely continue to face questions about the reliability of their research.

“What we researchers try to do is provide the science-consuming public with genuine insights about brain and behavior,” said Rick Gilmore, associate professor of psychology, Penn State and Institute for CyberScience affiliate. “We want to say things that are robust and true. Without reproducibility it’s hard to say that convincingly.”

Gilmore said there are positive trends in creating more open and more reproducible environments in psychological and brain science. For example, he said, there are more researchers sharing both data and the computer software used to analyze it. Researchers are also continuing to raise awareness about reproducibility issues with one another.

“Many scientists are beginning to embrace a more open approach to science that includes developing better software tools and databases that make it easier to share information,” said Gilmore.

Technology is also improving, which is also helping the effort, according to the researchers, who released their findings in a recent issue of the Annals of the New York Academy of Sciences. Developers are creating new web-based management tools and software to help researchers work with and openly share data with one another. There are also new data depositories with increased storage space.

According to Gilmore, cognitive neuroscience is a computationally intensive field that produces data files in a variety of sizes and formats. Data can include files from EEG, fMRI, MRI, CT and PET scans; video or audio recordings and surveys; or computer-based tasks.

However, there are relatively few organized initiatives to encourage sharing of these different file types, nor is sharing widespread.

“Right now, data sharing is still largely unfunded and unrewarded and is only rarely required,” said Gilmore, who also is founding co-director of the Databrary digital data library. “It’s something that isn’t a universal requirement for federal grant funding, for example.”

The researchers suggest that requiring data sharing for funding might be one way to motivate researchers. Publishers of scientific journals also could mandate the accessibility of data as a requirement to be published in the journal; in fact, some journals are beginning to require this.

According to Gilmore, while research papers are often looked at as the finished product of the scientific process, the data behind research papers are just as important.

“In addition to publishing scientific papers, behavioral and brain scientists need to be more open about the detailed procedures underlying their studies, more freely share the statistical programs that they use in analyzing data,” said Gilmore. “And researchers should share the data itself as openly as possible.”

He added that better reproducibility of studies can help researchers discover more reliable and useful information about perception, thinking, memory, action and other products of the mind and brain.

“We think that investments in the future of cognitive neuroscience infrastructure will generate big payoffs,” the researchers said. “Fostering the widespread adoption of open, transparent and reproducible research practices coupled with innovations in technology that enable the large-scale analysis of our particular store of ‘big data’ will accelerate the discovery of generalizable, robust and meaningful findings about the nature and origin of human cognition.”

Gilmore worked with Michele T. Diaz, associate professor of psychology and linguistics, and Brad A. Wyble, associate professor of psychology, both of Penn State; and Tal Yarkoni, assistant professor of psychology and director, Psychoinformatics lab, University of Texas.

This article was originally posted on Penn State News and modified to clarify Gilmore’s connection with ICS.

Penn State Institute for CyberScience

Help us build the future. Contact ICS at Penn State.

  • email: ics@psu.edu
  • phone: ICS-ACI Support (i-ASK Center): 814-865-4275 | General Inquiries: 814-867-1467

October 2017

SuMoTuWeThFrSa
1234567
891011121314
15161718192021
22232425262728
293031 
  • Oct 17, 2017 - Oct 18, 201710:00 am - 3:00 pm

    Materials Day 2017: Hot Topics in Materials

    This October, Materials Day 2017 will dig deep into a number…

    More Info
  • Oct 4, 2017 - Oct 6, 201712:00 pm

    COMPASS Science Communication Seminars

    COMPASS and the Institutes of Energy and the Environment (IE…

    More Info
  • Oct 27, 20173:00 pm - 4:30 pm

    ICS-ACI Training Series – Intro to HPC

    Do you need high-performance computing for your research pro…

    More Info
  • Oct 20, 20173:00 pm - 4:30 pm

    ICS-ACI Training Series – Intro to HPC

    Do you need high-performance computing for your research pro…

    More Info
  • Oct 5, 20171:30 pm - 3:00 pm

    CyberScience Seminar: Using Machine Learning to Forecast Student Outcomes

    For a variety of reasons, some college students are likely …

    More Info
  • Oct 4, 20174:00 pm - 6:00 pm

    COMPASS Science Communication Plenary

    COMPASS and the Institutes of Energy and the Environment (I…

    More Info
  • Oct 3, 2017 - Oct 4, 201711:00 am - 4:30 pm

    XSEDE HPC Workshop: MPI

    This workshop is intended to give C and Fortran programmers …

    More Info
  • Oct 12, 201712:00 pm - 1:00 pm

    Social Data Analytics Workshop Series: Data Structures and Subsetting in R

    In this workshop, we will finally move beyond the basic func…

    More Info
  • Oct 13, 20174:00 pm - 5:00 pm

    Stochastic Computations Working Group: “Parameter estimation using linear response statistics: Theory and numerical scheme”

    Join ICS Associate John Harlim and Xiantao Li for their bi-w…

    More Info