A guide to making your data analysis more reproducible
Karthik Ram • rOpenSci // UC Berkeley
(rough and incomplete)
(and a bunch of other work you do)
1.
2.
3.
4.
5.
6.
KÖMPENDIUM
1x
14x
4x
KÖMPENDIUM
Research compendia
Gentleman and Temple Lang, 2004
...We introduce the concept of a compendium as both a container for the different elements that make up the document and its computations (i.e. text, code, data, ...), and as a means for distributing, managing and updating the collection.
Research compendium principles
Stick with the conventions of your peers
Keep data, methods and outputs separate
Specify your computational environment as clearly as you can
Key components you'll need for sharing a compendium
License
VCS
Metadata
Archive
The R package structure is great way to organize and share a compendium!
Package: glue
Title: Interpreted String Literals
Version: 1.3.0.9000
Authors@R: person("Jim", "Hester", email = "james.f.hester@gmail.com", role = c("aut", "cre"))
Description: An implementation of interpreted string literals, inspired by
Python's Literal String Interpolation <https://www.python.org/dev/peps/pep-0498/> and Docstrings
<https://www.python.org/dev/peps/pep-0257/> and Julia's Triple-Quoted String Literals
<https://docs.julialang.org/en/stable/manual/strings/#triple-quoted-string-literals>.
Depends:
R (>= 3.1)
Imports:
methods
Suggests:
testthat,
(and many more)
License: MIT + file LICENSE
Encoding: UTF-8
LazyData: true
RoxygenNote: 6.0.1
Roxygen: list(markdown = TRUE)
URL: https://github.com/tidyverse/glue
BugReports: https://github.com/tidyverse/glue/issues
VignetteBuilder: knitr
ByteCompile: true
Package DESCRIPTION file
compendium DESCRIPTION file
Packaging your analysis as a compendium gives you access to powerful developer tools
COMPENDIUM
DESCRIPTION
LICENSE
Readme.md
data/
analysis/
Mydata.csv
Report.Rmd
Marwick et al 2017
Small compendia
COMPENDIUM
DESCRIPTION
LICENSE
Readme.md
data/
analysis/
Mydata.csv
Report.Rmd
NAMESPACE
R/
myfunctions.R
my_functions.Rd
man/
Marwick et al 2017
Medium compendia
COMPENDIUM
DESCRIPTION
LICENSE
Readme.md
NAMESPACE
R/
myfunctions.R
analysis/
man/
tests/
.travis.yml
Dockerfile
scripts/
my_report.Rmd
data/
Makefile
datasets.csv
Marwick et al 2017
Large/complex compendia
Data (Small → Medium)
Computing environment
Workflows
1. Data
How does one manage small to medium data in the context of a research compendium?
Small data
Put small data inside packages, especially if you ship a methods package with your analysis
CRAN = < 5 mb.
37% of the 13K packages on CRAN have some form of data.
piggyback
Attach large [data] files to Github repositories
github.com/ropensci/piggyback
Leveraging Github releases to share medium sized files
github.com/ropensci/piggyback
Medium data
github.com/ropensci/arkdb
2. Isolate your computing environment
It's important to isolate the computing environment so that changes in software dependencies don't break your analysis.
doi:10.1038/nbt.3780
Adding a Dockerfile to your compendium
Many ways to write a Dockerfile for your R project
o2r/containerit
jupyter/repo2docker
Binder
Binder is an open source project that is designed to make it really easy to share analyses that are in notebooks.
mybinder.org
Binder
mybinder.org
Git + Docker + RStudio
Setting up Binder
r-2018-12-20
Setting up Binder
install.packages("ggplot2")
Binder
Basic
Premium
Pro
install.r runtime.txt apt.txt
Dockerfile install.r
Dockerfile DESCRIPTION
Slow but easy to setup. Recommended for beginners
Faster launch
Best for compendia
free
free
free
Basic
Premium
Pro
install.r runtime.txt apt.txt
Dockerfile install.r
Dockerfile DESCRIPTION
Slow but easy to setup. Recommended for beginners
Faster launch
Best for compendia
free
free
free
Basic
Premium
Pro
install.r runtime.txt apt.txt
Dockerfile install.r
Dockerfile DESCRIPTION
Slow but easy to setup. Recommended for beginners
Faster launch
Best for compendia
free
free
free
A fast set up binder
DESCRIPTION
Dockerfile
Pull a base image from Rocker (e.g. rocker:binder/latest)
rocker-project.org
{holepunch} package makes this setup easy
github.com/karthik/holepunch
library(holepunch)
write_compendium_description(package = "Your compendium name",
description = "Compendium desc")
write_dockerfile(maintainer = "your_name")
generate_badge()
# Commit and push to GitHub
build_binder()
3. Workflow
Include a workflow to manage relationships between data output and code.
drake
general purpose workflow manager & pipeline toolkit for reproducibility and high-performance computing.
github.com/ropensci/drake
Drake: Data Frames in R for Make
No cumbersome Makefiles
Vast arsenal of parallel computing options
Visualize dependency graph and estimate run times
Convenient organization of output.
Drake: visualize dependency graph
Take home
Leverage the R package structure and support tools/services as much as possible
Take home
Use modern tools to make your compendia more accessible, but don't forget long-term archives and simpler formats
github.com/topics/research-compendium
data
environment
workflow
Near term
Long term
piggyback, data packages
Zenodo and friends
Binder and friends
Dockerfile
Drake
Core R tools, Make
1.
2.
3.
4.
5.
KÖMPENDIUM
1x
14x
4x
KÖMPENDIUM
1x
/ karthik/ rstudio2019
git repo has links to slides and all resources mentioned in the talk
A guide to making your data analysis more reproducible
Karthik Ram • rOpenSci // UC Berkeley
(rough and incomplete)
(and a bunch of other work you do)