How to get credit for your research software
Karthik
•
@berkeley.edu
Ram
Software impacts all modern research
Software turns a theoretical model into
quantitative predictions; software controls an experiment; and software extracts from raw data evidence supporting or rejecting a theory.
(Gael Varoquaux, scikit-learn)
90%
70%
63%
95%
Use
Can't continue without
Research Software
doi.org/10.5281/zenodo.843607
The skills required to be a successful scientific researchers are increasingly indistinguishable from the skills required to be in industry
(Jake VanderPlas, Google)
Software isn't a creditable research activity
We don't know how to cite software
Howison & Bullard 2016
Formal citations: 31% - 43%
Informal mentions are the norm, even in high impact journals
Software is frequently inaccessible (15 - 29%)
Lack of visibility means that incentives to produce high-quality, widely shared, and collaboratively developed software are lacking
We must find a way to
legitimize software as a
form of scholarship.
(Phil Bourne, University of Virgina)
Software isn't easily citable
Technical
Cultural
Software citations aren't allowed
Software citations aren't indexed
Software isn't peer reviewed
Software can't cite other software
To show we've done our research
To credit the work of others
To enrich the scholarly record
To avoid plagiarism
Why we cite things?
To show we've done our research
To enrich the scholarly record
To avoid plagiarism
Why we cite things?
To credit the work of others
How to better recognize software contributions?
1. Find some way to fit software into current (paper/book-centric) system.
2. Evolve beyond one-dimensional credit model.
Transtive credit
Evolving the model requires action across the whole scholarly ecosystem
Individual change: Authors, reviewers, editors.
Group/community change: Journals, societies.
Ecosystem change: Journals, publishers, indexers.
What if we just wrote papers about software?
Software paper
Gives us something easy to cite 👍
No changes required to existing infrastructure 👌
Publishing in existing journals raises profile of software within a community 🤘
Software paper
Writing another paper can be a ton of work 😅
Many journals don’t accept software papers 🤬
For long-lived software packages, static authorship presents major issues 😕
How to better recognize software contributions?
1. Find some way to fit software into current (paper/book-centric) system.
2. Evolve beyond one-dimensional credit model.
Really hard
Embracing the hack: What if we made it as easy as possible to write and publish a software paper?
Journal of Open Source Software
joss.theoj.org
A mechanism for research software developers to get credit within the current merit system of science
A developer friendly journal* for research software packages.
Paper preparation (and submission) for well-documented software should take no more than an hour.
Short – generally under two pages.
High-level functionality for a diverse, non-specialist audience.
A clear Statement of Need
Acknowledgement of any financial support and references to related work
Not permitted to contain novel results.
Format
Be as conventional as possible when it comes to integrating with the scholarly ecosystem
🤖
@whedon assign @x as reviewer
@whedon assign @x as editor
@whedon start review
🤖
@whedon generate pdf
@whedon check references
🤖
@whedon set archive
@whedon accept
🤖
https://peerj.com/articles/cs-147/
Users
Contributors
Maintaners
Authors
Reviewers
Editors
Be as conventional as possible when it comes to integrating with the scholarly ecosystem
ORCID for logins.
Deposit metadata and issue DOIs with Crossref.
Archive papers (and reviews) with Portico.
Archive reviewed software with Zenodo.
Get indexed by Google Scholar, Scopus
1
Follow best practices where they exist
2
Open access – papers licensed CC-BY (authors retain copyright).
Enforce the Open Source Definition
Clear ethics, ownerships, business model and governance statements.
Rigorous, transparent peer review process.
Provide a valuable service to authors
Minimize author effort beyond existing software documentation
3
Offer a constructive peer review process by experts.
Improving submissions, not rejecting
Transparent, achievable review criteria.
Free, open access.
Leverage the best parts of open source
Automate all the things!
4
Carry out reviews in a familiar environment (on GitHub)
Collaborative, welcoming community
Transparent editorial decision making
Leverage open source tools
Journal of Open Source Software
joss.theoj.org