Markdown and the future of collaborative manuscript writing

by Karthik Ram. Average Reading Time: almost 2 minutes.

When I first started using markdown a couple of years ago, I expected its popularity to be somewhat short lived and mostly in a blogging/note taking context. The greatest appeal of markdown is the fact the learning curve is non-existent, unparsed documents are easily readable (Latex on the other hand is not), and content can easily be parsed to a variety of formats with minimal effort. Little did I envision that it might one day revolutionize the world of collaborative academic writing.

I believe a few factors have made this possible:

  1. In the last few years, Github has skyrocketed in popularity among academics as a way to collaborate on statistical analyses. Github is extremely markdown friendly (and has its own flavored version) allowing people to effortlessly document code.

  2. Although document generation tools have existed in R (currently one of the most widely used statistical software tools in the academic community) for quite some time, recent efforts such Yihui’s knitr package have made is much easier for people to weave in results and figures both into traditional document formats such as Latex but also into markdown. The clutter-free, readability factor of markdown makes it easy to write and edit text alongside results and has a lower barrier to entry compared to Latex. Combine this with the free, cross-platform document generator Pandoc, one could easily embed citations and journal styles to programatically generate a final document in any desired format.

  3. Github’s powerful issue tracker provide a quick and easy way to solicit feedback from collaborators, track milestones, and more importantly leverage Git’s version control capabilities (no more Word document clutter).

Although only a handful of people are currently writing manuscripts in markdown, I’m really excited at the prospect of making this my primary workflow for all future (especially collaborative) manuscripts. All the results and figures can be generated by knitr, citations embedded using Pandoc, and the final document converted on the fly into one of many formats (latex, word, rtf, markdown) while the entire workflow (code, analyses, manuscript) remains synched with all collaborators via Github.

I’m planning to write a series of detailed posts describing my workflow that involves Github + Knitr + Pandoc. Stay tuned.

7 comments on ‘Markdown and the future of collaborative manuscript writing’

  1. Inundata says:

    New blog post: Using markdown for collaborative manuscript writing.

  2. New blog post: Using markdown for collaborative manuscript writing.

  3. rOpenSci says:

    New blog post: Using markdown for collaborative manuscript writing.

  4. I look forward to reading. I’m particularly keen to see how far you can push R Markdown to create more complicated academic documents (e.g., citations, tables of contents, table formatting, table/figure numbering, etc.).

    • Karthik Ram says:

      Hey Jeromy,
      The available tools aren’t perfect but we’re getting there. My colleague Carl has written knitcitations to handle citations in markdown but there are also other projects underway this summer to weave all these different aspects of writing academic papers in markdown (I’ll describe them in detail when I do get around to writing those posts).

  5. I want markdown for collaborative manuscript writing http://t.co/6GyeVT0r

  6. [...] great solutions to both these issues. So here’s a quick how-to (following up from my earlier post) for switching your writing workflow away from Word. There is a small learning curve but the payoff [...]

Leave a Reply