Reproducibility

College expertise on the subjects of reproducibility, transparency and reduction of bias and waste in research.

Reproducibility means that research data and code are made available so that others are able to reach the same results as are claimed in scientific outputs. Closely related is the concept of replicability, the act of repeating a scientific methodology to reach similar conclusions. These concepts are core elements of empirical research.

UK Reproducibility Network (UKRN)

The UKRN is a peer-led consortium which aims to ensure the UK retains its place as a centre for world-leading research by promoting a research culture that prioritises rigour and transparency. It does this by investigating factors that contribute to robust research, promoting training activites, sharing best practice and collaborating with external stakeholders.

The UKRN is led by a Supervisory Board comprising Marcus Munafò (Bristol), Emily Farran (Surrey), Eike Mark Rinke (Leeds), Etienne Roesch (Reading), Harvinder Virk (Leicester) and Malcolm MacLeod (Edinburgh).

The UKRN local network lead for the University of Edinburgh is Sarah Stanton and the institutional leads are Crispin Jordan and William Cawthorn.

Professor Malcolm MacLeod has contributed to publications about the UKRN:

The UK Reproducibility Network: A progress report – article in Journal of Neuroscience Methods (2023)

From grassroots to global: A blueprint for building a reproducibility network – article in PLoS Biology (2021)

Research Culture and Reproducibility – article in Trends in Cognitive Sciences (2020)

Edinburgh ReproducibiliTea

The aim of the ReproducibiliTea journal club is to help early career researchers build a network of people interested in open and reproducible research. The Edinburgh branch runs monthly seminar series on topics related to open research, reproducibility, research culture, and integrity – these meetings are open to all.

CMVM publications on the subject of reproducibility

Names in bold denote University of Edinburgh authors

2020-

Michael J. Thrippleton, co-author with Ben R. Dickie and others (2023)

A community-endorsed open-source lexicon for contrast agent–based perfusion MRI: A consensus guidelines report from the ISMRM Open Science Initiative for Perfusion Imaging (OSIPI) - article in Magnetic Resonance in Medicine

Emma Wilson, Fiona J. Ramage, Kimberley E. Wever, Emily S. Sena, Malcolm R. Macleod and Gillian L. Currie (2023)

Designing, conducting, and reporting reproducible animal experiments – article in Journal of Endocrinology

Cyril Pernet, co-author with Elise Bannier and others (2021)

The Open Brain Consent: Informing research participants and obtaining consent to share brain imaging data - editorial in Human Brain Mapping

Cyril R. Pernet, Ramon Martinez-Cancino, Dung Truong, Scott Makeig and Arnaud Delorme (2021)

From BIDS-Formatted EEG Data to Sensor-Space Group Results: A Fully Reproducible Workflow With EEGLAB and LIMO EEG – article in Frontiers in Neuroscience

Gaia Brezzo, Gillian Currie, Jill Fowler, Karen Horsburgh, Malcolm Macleod, Emily Sena and Stefan Szymkowiak, co-authors with Aisling McFall and others (2020)

UK consensus on pre-clinical vascular cognitive impairment functional outcomes assessment: questionnaire and workshop proceedings - article in Journal of Cerebral Blood Flow & Metabolism 

Cyril Pernet and others (2020)

Issues and recommendations from the OHBM COBIDAS MEEG committee for reproducible EEG and MEG research – perspective article in Nature Neuroscience

Malcolm Macleod, co-author with Walter J. Koroshetz and others (2020) 

Research Culture: Framework for advancing rigorous research – feature article in eLife magazine

2010-2019

Stewart J. Wiseman, Rozanna Meijboom, Maria del C. Valdés Hernández, Cyril Pernet, Eleni Sakka, Dominic Job, Adam D. Waldman & Joanna M. Wardlaw (2019)

Longitudinal multi-centre brain imaging studies: guidelines and practical tips for accurate and reproducible imaging endpoints and data sharing - article in Trials

Michael J Thrippleton, Yulu Shi, Gordon Blair, Iona Hamilton, Gordon Waiter, Christian Schwarzbauer, Cyril Pernet, Peter JD Andrews, Ian Marshall, Fergus Doubal, and Joanna M Wardlaw (2018) 

Cerebrovascular reactivity measurement in cerebral small vessel disease: rationale and reproducibility of a protocol for MRI acquisition and image processing - article in International Journal of Stroke

Bernhard Voelkl, Lucile Vogt, Emily S. Sena, Hanno Würbel (2018)

Reproducibility of preclinical animal research improves with heterogeneity of study samples – article in PLOS Biology 

Peter-Paul Zwetsloot, Mira Van Der Naald, Emily S Sena, David W Howells, Joanna IntHout, Joris AH De Groot, Steven AJ Chamuleau, Malcolm R MacLeod and Kimberley E Wever (2017)

Standardized mean differences cause funnel plot distortion in publication bias assessments – article in eLife

Malcolm MacLeod and Emily Sena, co-authors with Zsanett Bahor and others (2017)

Risk of bias reporting in the recent animal focal cerebral ischaemia literature - article in Clinical Science

Rustam Salman and Malcolm MacLeod, co-authors with Eivind Berge and others (2017)

Increasing value and reducing waste in stroke research - article in The Lancet Neurology

Gillian L Currie and Emily S Sena, co-authors with Nick A Andrews and others (2016)

Ensuring transparency and minimization of methodologic bias in preclinical pain research: PPRECISE considerations - article in PAIN

Cyril Pernet, Member of Open Science Collaboration (2015)

Estimating the reproducibility of psychological science - article in Science

Cyril Pernet and Jean-Baptiste Poline (2015)

Improving functional magnetic resonance imaging reproducibility - article in GigaScience

If you are a CMVM-affiliated author and would like your publication on reproducibility featured on this page, please get in touch at CMVMopenaccess@ed.ac.uk.