Karthik Ram

•  

@berkeley.edu

Without software, modern research would be impossible

Without research software, modern research would be impossible

Research software (as opposed to simply software) is software developed within academia and used for the purpose of research: generate, process and analyze results.

Hettrick et al 2016

90%

70%

63%

95%

Use

Can't continue without

Research Software

doi.org/10.5281/zenodo.843607

Software is critical for research but we don't value it as scholarship

The reproducibility crisis

is widespread

Baker, 2015. Baker & Dolgin 2017, Aschwanden, C. 2016, Casadevall & Fang 2010

Lack of reproducibility is quite widespread even in applied computational research

Collberg et al 2014​

Recently, Dr. Tony Wilson from CUNY Brooklyn tried to recreate my analysis, so that he could figure out how it worked and apply it to his own data ... he couldn’t quite recreate some of my core results.

“

So: how many results, negative or positive, that enter the published literature are tainted by a coding mistake as mine was. We just don’t know. Which raises an important question: why don’t we review code (or other custom software) as part of the peer-review process?

“

Training in computational skills is one of the largest unmet needs

Barone et al, 2017

If we want the research community to use software, we need to advocate for sustainable software and provide access to expertise

Hettrick et al, 2016

We don't know how to cite software

Howison & Bullard 2016

Formal citations: 31% - 43% 

Informal mentions are the norm, even in high impact journals

Software is frequently inaccessible (15 - 29%)

 Lack of visibility means that incentives to produce high-quality, widely shared, and collaboratively developed software are lacking

Software sustainability describes the practices, both technical and non-technical that allow software to continue to operate as expected in the future

Hettrick et al 2016

Software sustainability is strongly linked to reproducibility and transparency

> 18k awards totaling $9.6 billion related to research software.

NSF funding 1996-2016

urssi.us/files/urssi_proposal.pdf​

SGCI (Science Gateways)

URSSI

Tech & SW  →

 URSSI mission

To improve the quality, usefulness, and sustainability of research software by improving practices, and increasing diversity of practitioners

First community unconference

Second community unconference

Community survey

1

2

Funded

Jan 2018

Software metrics, citation

Research software incubators

Institute plans by December 2019

3

4

April, Oct 2018

Community unconferences

Software

People

Training

Org practices

Critical, Discoverability, Dev practices, Metrics, Citation 

Career paths, promotion, tenure, diversity

SWE carpentry, broad training and mentoring initiatives, 
URSSI Summer School
        

Governance, collaboration

1

Training

Curriculum development, partnering with existing initiatives, assessment, outreach

What we train people for

What we expect of them

There are large gaps in software engineering training for researchers

Could there be a RSE Carpentry?

SWC, DC

Summer schools

Traditional scientific meetings

Hackweeks

Training landscape

rOpenSci unconf

Project based learning

pnas.org/content/115/36/8872

RSC

2

Supporting software development

Incubate projects, provide consulting support, offer startup grants

A Ycombinator for science?

Feb 2019

College Park, MD

Software Incubators

3

Improving credit mechanisms

Credit mechanisms, Improving software citations, advocating for RSE roles

OSI compatible license

Complete documentation

High test coverage

Readable code

Usability

Writing software that is easier to sustain

joss.theoj.org

JOSS is a is a free, developer friendly, open access journal for research software packages
 

What does JOSS look for?

A really simple manuscript

Documentation

Functionality

General Checks

published papers

APC 

testing, docs, containers, contributor guidelines, license, archive

4

Building community

Disseminate best practices, governance etc.

ropensci.org/

software-review

The UK has successfully campaigned for RSE roles

Could the rest of us do something analogous?

Developing a community of research software engineers, and the next generation of research software mentors.

A set of core activities for URSSI  

There is a considerable training gap to be filled. We plan to train new developers and improve the diversity of the pipeline

Incubate software projects and help them grow, become sustainable, develop a governance model.
 

Disseminate expertise in software sustainability and help researcher get credit for their software work

Build a community around research software and make software a first class research output

Collaborate with URSSI

github.com/si2-urssi

discuss.urssi.us

urssi.us

bit.ly/c3dis19