Conceptualizing a US Research Software Sustainability Institute (URSSI)

#1743188
urssi.us

URSSI team

Karthik Ram

Sandra Gesing

Jeff Carver

Daniel S. Katz

Nic Weber

UIUC

Notre Dame

University of Alabama

U. Washington

UC Berkeley

Wolfgang Bangerth · Anshu Dubey · Melissa Haendel · Michael A Heroux · Kathryn Huff · Suresh Marru · Kate Mueller · Jarek Nabrzyski · Kyle Niemeyer · Marlon Pierce · Ariel Rokem · Arfon Smith · Tracy Teal · Matthew Turk · Rick Wagner · Michael Zentner

Richard Arthur · Michelle Barker · Philip E. Bourne · Neil Chue Hong · Daniel Crawford · James Howison · Kurt Schwehr  · Jeff Spies · Nancy Wilkins-Diehr

Senior Personnel

Advisory Board

Without software, modern research would be impossible

Without research software, modern research would be impossible

Research software (as opposed to simply software) is software developed within academia and used for the purpose of research: generate, process and analyze results.

 

Hettrick et al 2016

90%

70%

63%

95%

Use

Can't continue without

Research Software

http://doi.org/10.5281/zenodo.843607

The reproducibility crisis

is widespread

Baker, 2015. Baker & Dolgin 2017, Aschwanden, C. 2016, Casadevall & Fang 2010

Medicine

▪ study power and bias

 

Psychology

▪ p-hacking

 

Biomed

▪ lack of access to full datasets and protocols

 

Lack of reproducibility is quite widespread even in applied computational research

Collberg et al 2014​

The extent to which software would actually build with reasonable effort was quite low

Collberg et al 2014​

< 20%

Software is critical for research but we don't value it as scholarship

Prof. Daniel Bolnick

Recently, Dr. Tony Wilson from CUNY Brooklyn tried to recreate my analysis, so that he could figure out how it worked and apply it to his own data ... he couldn’t quite recreate some of my core results.

So: how many results, negative or positive, that enter the published literature are tainted by a coding mistake as mine was. We just don’t know. Which raises an important question: why don’t we review code (or other custom software) as part of the peer-review process?

Training in computational skills is one of the largest unmet needs

Barone et al, 2017

If we want the research community to use software, we need to advocate for sustainable software and provide access to expertise

Hettrick et al, 2016

We don't know how to cite software

Howison & Bullard 2016

Formal citations: 31% - 43% 

Informal mentions are the norm, even in high impact journals

Software is frequently inaccessible (15 - 29%)

 Lack of visibility means that incentives to produce high-quality, widely shared, and collaboratively developed software are lacking

Career paths

Besides credit and training, we don't have institutional support for developers and maintainers to ensure long-term availability of software.

Software sustainability describes the practices, both technical and non-technical that allow software to continue to operate as expected in the future

Hettrick et al 2016

Software sustainability is strongly linked to reproducibility and transparency

> 18k awards totaling $9.6 billion related to research software.

NSF funding 1996-2016

urssi.us/files/urssi_proposal.pdf​

SGCI (Science Gateways)

MolSSI

HEP-S2I2

Science & Engineering

Tech & SW  

SGCI (Science Gateways)

MolSSI

HEP-S2I2

URSSI

Science & Engineering

Tech & SW  

A road map for URSSI

Initial workshop

github.com/si2-urssi/berkeley

April 2018

Get the community together to understand what we have learned and identify issues URSSI will be best suited to address

Workshop goal

1

Training

Curriculum development, partnering with existing initiatives, assessment, outreach

Could there be a RSE Carpentry?

SWC, DC

Summer schools

Workshops

Traditional scientific meetings

Hackweeks

Training landscape

rOpenSci unconf

dotAstro

Project based learning

Pedagogy

arxiv.org/abs/1711.00028

2

Policy

Credit mechanisms, Improving software citations, advocating for RSE roles

OSI compatible license

Complete documentation

High test coverage

Readable code

Usability

Improving software quality

Journal of Open Source Software

joss.theoj.org

Arfon Smith

Data Science Mission Office (DSMO) Head, STSCI

Journal of Open Source Software

joss.theoj.org

A mechanism for research software developers to get credit within the current merit system of science

joss.theoj.org

How do we promote software discovery?

The UK has successfully campaigned for RSE roles

Could we do something analogous?

3

Supporting software development

Incubate projects, provide consulting support, offer startup grants

A Ycombinator for science?

4

Building community

Disseminate best practices, governance etc.

Developing a community of research software engineers, and the next generation of Research Software Engineers

Supporting software Supporting science Supporting community Science Impact
Development support x x
​Incubator x x
Training x x x
Policy x x x
Community x x x x

Initial straw man

 

Help projects grow, become sustainable, develop a governance model

Train new developers and improve the diversity of the pipeline

Influence policies around hiring, tenure, promotion, and career paths

Raise awareness of the critical role played by research software 

Disseminate expertise in software sustainability

Promote the role of software engineers in research groups

Upcoming workshops

  •  Software credit and citation metrics

  • Software incubator (Nic Weber)

  • Final workshop

Contribute to URSSI

github.com/si2-urssi

discuss.urssi.us

urssi.us

contact@urssi.us

inundata.org/talks/si2