Insights from planning a research software sustainability institute
Karthik
•
@berkeley.edu
Ram
Karthik Ram
•
@berkeley.edu
Without software, modern research would be impossible
Without research software, modern research would be impossible
Research software (as opposed to simply software) is software developed within academia and used for the purpose of research: generate, process and analyze results.
Hettrick et al 2016
90%
70%
63%
95%
Use
Can't continue without
Research Software
doi.org/10.5281/zenodo.843607
Software is critical for research but we don't value it as scholarship
The reproducibility crisis
is widespread
Baker, 2015. Baker & Dolgin 2017, Aschwanden, C. 2016, Casadevall & Fang 2010
Lack of reproducibility is quite widespread even in applied computational research
Collberg et al 2014
Prof. Daniel Bolnick
Recently, Dr. Tony Wilson from CUNY Brooklyn tried to recreate my analysis, so that he could figure out how it worked and apply it to his own data ... he couldn’t quite recreate some of my core results.
“
So: how many results, negative or positive, that enter the published literature are tainted by a coding mistake as mine was. We just don’t know. Which raises an important question: why don’t we review code (or other custom software) as part of the peer-review process?
“
Training in computational skills is one of the largest unmet needs
Barone et al, 2017
If we want the research community to use software, we need to advocate for sustainable software and provide access to expertise
Hettrick et al, 2016
We don't know how to cite software
Howison & Bullard 2016
Formal citations: 31% - 43%
Informal mentions are the norm, even in high impact journals
Software is frequently inaccessible (15 - 29%)
Lack of visibility means that incentives to produce high-quality, widely shared, and collaboratively developed software are lacking
Software sustainability describes the practices, both technical and non-technical that allow software to continue to operate as expected in the future
Hettrick et al 2016
Software sustainability is strongly linked to reproducibility and transparency
> 18k awards totaling $9.6 billion related to research software.
NSF funding 1996-2016
urssi.us/files/urssi_proposal.pdf
SGCI (Science Gateways)
MolSSI
HEP-S2I2
URSSI
Science & Eng →
Tech & SW →
URSSI mission
To improve the quality, usefulness, and sustainability of research software by improving practices, and increasing diversity of practitioners
First community unconference
Second community unconference
Community survey
1
2
Funded
Jan 2018
Software metrics, citation
Research software incubators
Institute plans by December 2019
3
4
April, Oct 2018
Community unconferences
Software
People
Training
Org practices
Critical, Discoverability, Dev practices, Metrics, Citation
Career paths, promotion, tenure, diversity
SWE carpentry, broad training and mentoring initiatives,
URSSI Summer School
Governance, collaboration
1
Training
Curriculum development, partnering with existing initiatives, assessment, outreach
What we train people for
What we expect of them
There are large gaps in software engineering training for researchers
Could there be a RSE Carpentry?
SWC, DC
Summer schools
Workshops
Traditional scientific meetings
Hackweeks
Training landscape
rOpenSci unconf
dotAstro
Project based learning
Pedagogy
pnas.org/content/115/36/8872
RSC
2
Supporting software development
Incubate projects, provide consulting support, offer startup grants
A Ycombinator for science?
Feb 2019
College Park, MD
Software Incubators
3
Improving credit mechanisms
Credit mechanisms, Improving software citations, advocating for RSE roles
OSI compatible license
Complete documentation
High test coverage
Readable code
Usability
Writing software that is easier to sustain
Journal of Open Source Software
joss.theoj.org
JOSS is a is a free, developer friendly, open access journal for research software packages
What does JOSS look for?
A really simple manuscript
Documentation
Functionality
General Checks
published papers
APC
500
to publish
$0
$3.50
testing, docs, containers, contributor guidelines, license, archive
Software paper
4
Building community
Disseminate best practices, governance etc.
ropensci.org/
software-review
The UK has successfully campaigned for RSE roles
Could the rest of us do something analogous?
Developing a community of research software engineers, and the next generation of research software mentors.
A set of core activities for URSSI
There is a considerable training gap to be filled. We plan to train new developers and improve the diversity of the pipeline
Incubate software projects and help them grow, become sustainable, develop a governance model.
Disseminate expertise in software sustainability and help researcher get credit for their software work
Build a community around research software and make software a first class research output
