How to get credit for your research software
Karthik
•
@berkeley.edu
Ram
Use arrows or touch gestures to navigate slide deck
Software impacts all modern research
Software turns a theoretical model into
quantitative predictions; software controls an experiment; and software extracts from raw data evidence supporting or rejecting a theory.
(Gael Varoquaux, scikit-learn)
90%
70%
63%
95%
Use
Can't continue without
Research Software
doi.org/10.5281/zenodo.843607
The skills required to be a successful scientific researchers are increasingly indistinguishable from the skills required to be in industry
(Jake VanderPlas, Google)
Software isn't a creditable research activity
Howison & Bullard 2016
Formal citations: 31% - 43%
Informal mentions are the norm, even in high impact journals
Software is frequently inaccessible (15 - 29%)
Lack of visibility means that incentives to produce high-quality, widely shared, and collaboratively developed software are lacking
We must find a way to
legitimize software as a
form of scholarship.
(Phil Bourne, University of Virgina)
Software isn't easily citable
Technical
Cultural
Software citations aren't allowed
Software citations aren't indexed
Software isn't peer reviewed
Software can't cite other software
To show we've done our research
To credit the work of others
To enrich the scholarly record
To avoid plagiarism
Why we cite things?
To show we've done our research
To enrich the scholarly record
To avoid plagiarism
Why we cite things?
To credit the work of others
How to better recognize software contributions?
1. Find some way to fit software into current (paper/book-centric) system.
2. Evolve beyond one-dimensional credit model.
Transitive credit
Evolving the model requires action across the whole scholarly ecosystem
Individual change: Authors, reviewers, editors.
Group/community change: Journals, societies.
Ecosystem change: Journals, publishers, indexers.
What if we just wrote papers about software?
Software paper
Gives us something easy to cite 👍
No changes required to existing infrastructure 👌
Publishing in existing journals raises profile of software within a community 🤘
Software paper
Writing another paper can be a ton of work 😅
Many journals don’t accept software papers 🤬
For long-lived software packages, static authorship presents major issues 😕
How to better recognize software contributions?
1. Find some way to fit software into current (paper/book-centric) system.
2. Evolve beyond one-dimensional credit model.
Really hard
Embracing the hack: What if we made it as easy as possible to write and publish a software paper?
Journal of Open Source Software
joss.theoj.org
A mechanism for research software developers to get credit within the current merit system of science
A developer friendly journal* for research software packages.
Paper preparation (and submission) for well-documented software should take no more than an hour.
Short – generally under two pages.
High-level functionality for a diverse, non-specialist audience.
A clear Statement of Need
Acknowledgement of any financial support and references to related work
Not permitted to contain novel results.
Format
Be as conventional as possible when it comes to integrating with the scholarly ecosystem
A hosted Git repository
Why is this research software?
Whedon
Whedon can be summoned on any review thread
Compiles PDF on demand
Identifies and labels languages
@whedon assign @x as reviewer
@whedon assign @x as editor
@whedon start review
🤖
For editors
@whedon generate pdf
🤖
@whedon check references
For authors & reviewers
Paper preview service
@whedon set archive
@whedon accept
🤖
@whedon set version
For editors & EICs
Python and R dominate
Most submissions are US/UK centric
Submissions have been steadily growing
The before times
~ 50% of JOSS papers get cited
| UMAP: Uniform Manifold Approximation and Projection | 2018-09-02 | 514 |
| corner.py: Scatterplot matrices in Python | 2016-06-08 | 377 |
| Welcome to the Tidyverse | 2019-11-21 | 259 |
| Armadillo: a template-based C++ library for linear algebra | 2016-06-10 | 184 |
| hdbscan: Hierarchical density based clustering | 2017-03-21 | 158 |
Most cited papers
Users
Contributors
Maintainers
Roles in open source projects
Authors
Reviewers
Editors
Roles in open journals
Be as conventional as possible when it comes to integrating with the scholarly ecosystem
ORCID for logins.
Deposit metadata and issue DOIs with Crossref.
Archive papers (and reviews) with Portico.
Archive reviewed software with Zenodo.
Get indexed by Google Scholar, Scopus
1
Follow best practices where they exist
2
Open access – papers licensed CC-BY (authors retain copyright).
Enforce the Open Source Definition
Clear ethics, ownerships, business model and governance statements.
Rigorous, transparent peer review process.
Provide a valuable service to authors
Minimize author effort beyond existing software documentation
3
Offer a constructive peer review process by experts.
Improving submissions, not rejecting
Transparent, achievable review criteria.
Free, open access.
Leverage the best parts of open source
Automate all the things!
4
Carry out reviews in a familiar environment (on GitHub)
Collaborative, welcoming community
Transparent editorial decision making
Leverage open source tools
Journal of Open Source Software
joss.theoj.org
