Using Coveralls with Clojure

Coveralls is a service that keep track of your tests coverage for you. It can notifies you when your coverage decreases under a custom threshold, and their bot comments on pull requests to report their tests coverage. Like Travis-CI, it allows you to add a badge to your readme with an up-to-date tests coverage percentage.

If you already test your GitHub projects with a CI server like Travis, it’s very easy to add Coveralls to your workflow. Unfortunately, they have a library for Ruby, a couple user-provided libraries for other languages such as PHP, Java and Python, but nothing for Clojure. Fortunately, they provide an API for unsupported languages like Clojure. Here is how to use it.

Cloverage

Cloverage is a Clojure library and a Leiningen plugin to get form-level tests coverage of your projects. It’s dead easy to use and generates HTML reports for you as well as some other formats.

Add their plugin to your project.clj:

(defproject your-project "0.1.0"
  ; ...
  :plugins [[lein-cloverage "1.0.2"]])

Use lein cloverage to run your tests and generate various reports. You can specify which Cloverage version you want to use with CLOVERAGE_VERSION. The latest stable one is 1.0.3, but we’ll need to use 1.0.4-SNAPSHOT for this article:

CLOVERAGE_VERSION=1.0.4-SNAPSHOT lein cloverage

Coveralls

If you don’t already use it, signup on Coveralls using your GitHub account. Then, click on “Add Repo”, find your repo in the list and activate it. That’s all you have to do on their website for now.

Travis CI

You now need to custom your builds on Travis to use Coveralls. We’ll do it in a Bash script, so add this to your .travis.yml:

after_script:
  - bash -ex test/coveralls.sh

This tells Travis to run test/coveralls.sh after each build. We use Bash options -e and -x to respectively stop the script at the first failure and print each command.

Now edit test/coveralls.sh and add the following content:

COVERALLS_URL='https://coveralls.io/api/v1/jobs'
CLOVERAGE_VERSION='1.0.4-SNAPSHOT' lein2 cloverage -o cov --coveralls
curl -F 'json_file=@cov/coveralls.json' "$COVERALLS_URL"

It’ll run Cloverage and output reports in cov (use another name if you prefer), using a report for Coveralls. This generates a coveralls.json file which can then be sent to their API. That’s what we do on the next line, using cURL.

That’s all! You can now push on GitHub, Travis will run your tests and send their tests coverage to Coveralls.

Caveats

Coveralls doesn’t support partial-line tests coverage, so we’re cheating a little bit here using hits count. A line that it never covered has 0 hits, one that is partially covered has one hit, and a fully covered one has two hits. There’s an open issue regarding this.