Karthik Ram

@_inundata

Reproducible computation is finally being recognized today by many scientific leaders as a central requirement for valid scientific publication.

What will data science look like in 2065?

Lack of reproducibility is quite widespread in applied computational research

Collberg et al 2014​

The extent to which code would actually build with reasonable effort is quite low

Collberg et al 2014​

< 20%

Software is critical for research but we don't value it as scholarship

Prof. Daniel Bolnick

Recently, Dr. Tony Wilson from CUNY Brooklyn tried to recreate my analysis, so that he could figure out how it worked and apply it to his own data ... he couldn’t quite recreate some of my core results.

I dug up my original code, sent it to him, and after a couple of back-and-forth emails we found my error.

So: how many results, negative or positive, that enter the published literature are tainted by a coding mistake as mine was. We just don’t know. Which raises an important question: why don’t we review code (or other custom software) as part of the peer-review process?

When software is not visible, it is often excluded from peer review

 Lack of visibility means that incentives to produce high-quality, widely shared, and collaboratively developed software are lacking

Founded in 2011 by Carl Boettiger, Scott Chamberlain and myself over a Twitter thread

The earliest known image of a coffeehouse dated to 1674

Founded in 2011 by Carl Boettiger, Scott Chamberlain and myself over a Twitter thread

Early motivation was to make data access easier & reproducible

Open Source Software

100+ software packages to support data science. e.g. spatial data, biodiversity informatics & climate change, glue for workflows.

Glue software

Data retrieval (APIs, data storage services, journals)

Data visualization (plot.ly, magick)

Data sharing (figshare, Zenodo)

Reproducibility

Developing a community of research software engineers, and the next generation of data science mentors.

Community

Culture change

Bringing the best parts of academic peer-review to research software
 

Incentivize scientists who engage in reproducible research ($50k fellowships)

Improving Quality of Scientific Software

Even without software pubs, we need to create a culture around peer-reviewing our research software

Pre-submission inquiry

Fit based on our criteria

🤔

Peer-review

evaluate the package for usability, quality, and style based on our guidelines

Acceptance 💯

Packages are badged and added to our system

Open & non-adverserial

No rejections

Makes the process constructive for everyone involved

OSI compatible license

Complete documentation

High test coverage

Readable code

Usability

Software Review

A software review thread

Nov 2017

Feb 2018

“I don’t really see myself writing another serious package without having it go through code review.”

Onboarding

JOSS, MEE

Review for us

ropensci.org/onboarding

Get involved

discuss.ropensci.org

onboarding.ropensci.org

@ropensci

ropensci.org/community

inundata.org/talks/yvr