January 3, 2021

Who's in charge here: Versions from SCM

I originally wrote entry for the Roomkey tech blog in 2016.  That site has been shut down due to the Coronavirus-triggered demise of Roomkey in 2020.  I'm reproducing it here, un-edited, for archival purposes.

This article examines some of the issues involved in managing versions and describes an alternative to the default management offered by leiningen that we use at Room Key.  It assumes familiarity with Clojure, leiningen and git.

Some History

In 2010 Room Key[1] started developing a web service using Clojure 1.2.0 and Leiningen 1.x.  By early 2011, we had decided that the version of our artifacts, particularly the primary web site application, needed to be something more flexible than commited text within project.clj.  In February, we wrote the first leingingen support code that slurped the version from git and supplied it to the project as it was being evaluated (and it was indeed evaluated back in the Leiningen 1.x days).  It wasn't much code -only a couple of dozen lines (and the author was allergic to line lengths beyond about 40 characters).  But it introduced a way of thinking that seems very natural and obvious.  Since 2011, we have expanded on the original work in several steps and today, in mid-2016 the evolution of that code is lein-v, a leiningen plugin that solves a lot of hairy issues surrounding versions.

Our Problem

A version identifies software artifacts as they evolve over time.  As such, versioning is a critical component of change management.

We find that leiningen's approach of having the version stored in the commited project.clj source requires either ambiguous versions (where many commits share the same version)[2] or unweildy commit practices (changing the version in project.clj on every commit).

We believe:

  1. Unique (and reproducible/committed) source should produce unique versions
  2. Versioning should be painless in the simplest cases, but complex cases should still be manageable
  3. Versions should reflect an ordering of source code where possible[3]
  4. Versioning information should live in the SCM repo -the source of source code truth
  5. Version information is metadata and should not be stored within with the data it describes

A Solution

Lein-v uses git metadata to build a unique, reproducible and meaningful version for every commit.  Along the way, it adds useful metadata to your project and artifacts (jar and war files) to tie them back to a specific commit.  Consequently, it helps ensure that you never release an irreproduceable artifact.

Read git to generate a version

Instead of reading a static string in project.clj, lein-v reads git tag and commit data to construct a version that is unique to the current commit.  It then injects this git-derived version into all subsequent leiningen tasks.  The version is constructed from:

  • Coded tags of the form v<version> (generated by the release process)
  • SHA of the current commit
  • Commit distance from most recent version tag to the current commit

The commit distance provides an ordering of commits and the SHA uniquely identifies commits.  Together, they can be used to produce versions that don't rely on manual editing of project.clj.

Update versions logically during release

By leveraging leiningen's built-in support for extending its release process, lein-v can close the loop on version management by updating the version stored in git tags.  Here is an example process that we use at Room Key:

    :release-tasks [["vcs" "assert-committed"]
                    ["v" "update"] ;; compute new version & tag it
                    ["vcs" "push"]
                    ["deploy"]]

The lein-v update task computes a new version from the current version and a command line bump parameter such as :major or :alpha[4].  The new version is injected into the standard leiningen processing, which allows the leingingen vcs task to work as expected.

A note on version structure

Since the dawn of software management (in all its various guises), version structure has been a contentious subject.  Lein-v attempts to remain neutral on the subject of what a version should look like.  But because leiningen ultimately relies on maven to resolve dependencies, lein-v leans towards maven's (loose) view of versions.  Version structure is abstracted into two protocols (SCMHosted and Releasable), and implementations for maven 3 and semver 2.0.0 are provided.  It's not hard to write your own implementation and select it instead of the default MavenVersion.

Availability

Lein-v is available on Clojars and source is available on GitHub.  Pull requests are welcome and issues can be raised against our GitHub project as well.

Notes

then known as Hotelicopter. ↩︎

SNAPSHOT versions are a prime example of ambiguous versions, and we do not use them at Room Key. ↩︎

branches in the source code repo make a total ordering impossible. ↩︎

The actual bump parameters supported depend on the specific versioning format you use.  The default maven format supports all the built-in leingingen bump levels (:major :minor :patch :alpha :beta and :rc) as well as :snapshot and :release to explicitly manage SNAPSHOT releases. ↩︎