How to get credit for your research software

Karthik

 

@berkeley.edu

Ram

Use arrows or touch gestures to navigate slide deck

Software impacts all modern research

Software turns a theoretical model into

quantitative predictions; software controls an  experiment; and software extracts from raw data evidence supporting or rejecting a theory.

(Gael Varoquaux, scikit-learn)

90%

70%

63%

95%

Use

Can't continue without

Research Software

doi.org/10.5281/zenodo.843607

The skills required to be a successful scientific researchers are increasingly indistinguishable from the skills required to be in industry

(Jake VanderPlas, Google)

Software isn't a creditable research activity

 

Howison & Bullard 2016

Formal citations: 31% - 43% 

Informal mentions are the norm, even in high impact journals

Software is frequently inaccessible (15 - 29%)

 Lack of visibility means that incentives to produce high-quality, widely shared, and collaboratively developed software are lacking

We must find a way to
legitimize software as a
form of scholarship
.

(Phil Bourne, University of Virgina)

Software isn't easily citable

Technical

Cultural

Software citations aren't allowed

Software citations aren't indexed

Software isn't peer reviewed

Software can't cite other software

To show we've done our research
To credit the work of others
To enrich the scholarly record
To avoid plagiarism

Why we cite things?

To show we've done our research

 


To enrich the scholarly record
To avoid plagiarism

Why we cite things?

To credit the work of others

How to better recognize software contributions?

1.  Find some way to fit software into current (paper/book-centric) system.

 

2.  Evolve beyond one-dimensional credit model.

Transitive credit

Evolving the model requires action across the whole scholarly ecosystem

Individual change: Authors, reviewers, editors.

Group/community change: Journals, societies.

Ecosystem change: Journals, publishers, indexers.

What if we just wrote papers about software?

Software paper

Gives us something easy to cite 👍


No changes required to existing infrastructure 👌


Publishing in existing journals raises profile of software within a community 🤘

 

Software paper

Writing another paper can be a ton of work 😅

Many journals don’t accept software papers 🤬

For long-lived software packages, static authorship presents major issues 😕

How to better recognize software contributions?

1.  Find some way to fit software into current (paper/book-centric) system.

 

2.  Evolve beyond one-dimensional credit model.

Really hard

Embracing the hack: What if we made it as easy as possible to write and publish a software paper?

Journal of Open Source Software

joss.theoj.org

A mechanism for research software developers to get credit within the current merit system of science

A developer friendly journal* for research software packages.

 

Paper preparation (and submission) for well-documented software should take no more than an hour.

Short – generally under two pages.

 

High-level functionality for a diverse, non-specialist audience.

 

A clear Statement of Need

 

Acknowledgement of any financial support and references to related work

 

Not permitted to contain novel results.

Format

Be as conventional as possible when it comes to integrating with the scholarly ecosystem 

A hosted Git repository

Why is this research software?

Whedon

Whedon can be summoned on any review thread

Compiles PDF on demand

Identifies and labels languages

@whedon assign @x as reviewer
@whedon assign @x as editor
@whedon start review

🤖

For editors

@whedon generate pdf

🤖

@whedon check references

For authors & reviewers

Paper preview service

@whedon set archive
@whedon accept

🤖

@whedon set version

For editors & EICs

Python and R dominate

Most submissions are US/UK centric

Submissions have been steadily growing

The before times

~ 50% of JOSS papers get cited

UMAP: Uniform Manifold Approximation and Projection 2018-09-02 514
corner.py: Scatterplot matrices in Python 2016-06-08 377
Welcome to the Tidyverse 2019-11-21 259
Armadillo: a template-based C++ library for linear algebra 2016-06-10 184
hdbscan: Hierarchical density based clustering 2017-03-21 158

Most cited papers

Users

Contributors

Maintainers

Roles in open source projects

Authors

Reviewers

Editors

Roles in open journals

Be as conventional as possible when it comes to integrating with the scholarly ecosystem

ORCID for logins.
Deposit metadata and issue DOIs with Crossref.
Archive papers (and reviews) with Portico.
Archive reviewed software with Zenodo.
Get indexed by Google Scholar, Scopus 

1

Follow best practices where they exist

2

Open access – papers licensed CC-BY (authors retain copyright).

Enforce the Open Source Definition

Clear ethics, ownerships, business model and governance statements.

Rigorous, transparent peer review process.

Provide a valuable service to authors

Minimize author effort beyond existing software documentation

3

Offer a constructive peer review process by experts.

Improving submissions, not rejecting

Transparent, achievable review criteria.

Free, open access.

Leverage the best parts of open source

Automate all the things! 

4

Carry out reviews in a familiar environment (on GitHub)

Collaborative, welcoming community

Transparent editorial decision making

Leverage open source tools

Journal of Open Source Software

joss.theoj.org