After creating a few Scala projects, we noticed that we repeated a good chunk of SBT configuration across projects. This caused issues, the kind that occur any time there’s repeated code. In some cases, developers were resolving version and dependency incompatibilities over and over. In others, improvements made in one project propagated to others slowly – if at all. As for the configurations themselves, boilerplate noise obscured the parts of a project’s configuration that were unique to the project itself.

So what could be done to DRY up our SBT config? The solution we settled upon was a custom AutoPlugin to define common configuration and to pull in commonly-used plugins, in some ways acting like a parent POM in Maven-land.

The Plugin Project

Project structure

The configuration for the parent plugin was split across four files:

├── build.sbt
├── project
│   └── plugins.sbt
├── src
│   └── main
│       └── scala
│           └── com.sharethrough.sbt
│               └── BaseSettingsPlugin.scala
└── version.sbt

src/main/scala/com.sharethrough.sbt/BaseSettingsPlugin.scala

Skeleton

This is the main class defining the plugin and how it augments the build. A skeleton for an AutoPlugin would be something like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
package com.sharethrough.sbt

import sbt._
import Keys._

object BaseSettingsPlugin extends AutoPlugin {
  object autoImport {
    // settingKeys and commands to be automatically imported
  }

  // Make our own settings and commands readily available for the reset of the file
  import autoImport._

  // allow the plug-in to be included automatically
  override def trigger: PluginTrigger = allRequirements

  /* ... the functionality you want the plug-in to provide ...*/
}

Basic project settings

In the case of a plugin for common settings like this most of the changes would be from overriding the projectSettings member to define default values for setting keys like organization or scalacOptions.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
  override def projectSettings: Seq[Setting[_]] = Seq(
    organization := "com.sharethrough",

    scalacOptions ++= Seq(
      "-encoding", "UTF-8",
      "-deprecation",
      "-feature",
      "-language:higherKinds",
      "-Ywarn-dead-code",
      "-Xlint"
    ),

    ... etc ...
  }

Mixing in plugins which are not AutoPlugins

We’ll see how we can depend on other AutoPlugins when we look at some of the other files, but if you want to depend on other plugins then they need to be mixed in here. For example, we use sbt-dependency-graph. As of this writing sbt-dependency-graph isn’t available as an AutoPlugin, so we need to import its settings and prepend those to our own.

1
2
3
4
5
6
7
8
import net.virtualvoid.sbt.graph.Plugin.graphSettings

object BaseSettingsPlugin extends AutoPlugin {

  override def projectSettings: Seq[Setting[_]] = graphSettings ++ Seq(
    ... etc ...
  )
}

Common library versions

One major pain point we experienced was the struggle of locking down a consistent set of dependencies that we knew worked together (if you have dependencies built on Scalaz, and did the migration from v7.0.x to v7.1.x, then you know the pain I speak of). So we go one step further: to enable this dependency lockdown, we define a new settingKey called libraryVersions which is just a Map[Symbol, String], and populate it with the versions we want to use:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
  object autoImport {
    val libraryVersions = settingKey[Map[Symbol, String]]("Common versions to be used for dependencies")
  }

  ... etc ...

  override def projectSettings: Seq[Setting[_]] = graphSettings ++ Seq(
    ... etc ...
    libraryVersions := Map(
      'akka       -> "2.3.0",
      'argonaut   -> "6.1",
      'aws        -> "1.9.21",
      'dispatch   -> "0.11.3",
      'librato    -> "4.0.1.9",
      'logback    -> "1.1.3",
      'mockito    -> "1.10.19",
      'nscalaTime -> "1.8.0",
      'redis      -> "3.0",
      'scalacheck -> "1.12.5",
      'scalatest  -> "2.2.5",
      'scalaz     -> "7.1.2",
      'scalike    -> "2.2.7",
      'slf4j      -> "1.7.12",
      'spark      -> "1.4.0"
    ),
        ... etc ...
  }

Putting it all together

A sample of a full file is:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
package com.sharethrough.sbt

import sbt._
import Keys._
import net.virtualvoid.sbt.graph.Plugin.graphSettings

object BaseSettingsPlugin extends AutoPlugin {
  object autoImport {
    val libraryVersions = settingKey[Map[Symbol, String]]("Common versions to be used for dependencies")
  }

  import autoImport._

  override def trigger: PluginTrigger = allRequirements

  val repositoryRoot = "http://artifacts.example.com/sharethrough"

  override def projectSettings: Seq[Setting[_]] = graphSettings ++ Seq(
    organization := "com.sharethrough",

    scalacOptions ++= Seq(
      "-encoding", "UTF-8",
      "-deprecation",
      "-feature",
      "-language:higherKinds",
      "-Ywarn-dead-code",
      "-Xlint"
    ),

    // Update the SBT prompt with two features:
    // - color to better visually distinguish between SBT and scala prompts
    // - current project name to give better context when in repo with sub-projects
    shellPrompt := { s: State =>
      val c = scala.Console
      val blue = c.RESET + c.BLUE + c.BOLD
      val white = c.RESET + c.BOLD

      val projectName = Project.extract(s).currentProject.id

      blue + projectName + white + " \u00BB " + c.RESET
    },

    resolvers ++= Seq(
      "sharethrough-releases" at repositoryRoot + "/libs-releases-local",
      "sharethrough-snapshots" at repositoryRoot + "/libs-snapshots-local",
      "Scalaz Bintray Repo" at "http://dl.bintray.com/scalaz/releases"
    ),

    libraryVersions := Map(
      'akka       -> "2.3.14",
      'argonaut   -> "6.1",
      'aws        -> "1.9.21",
      'dispatch   -> "0.11.3",
      'librato    -> "4.0.1.9",
      'logback    -> "1.1.3",
      'mockito    -> "1.10.19",
      'nscalaTime -> "1.8.0",
      'redis      -> "3.0",
      'scalacheck -> "1.12.5",
      'scalatest  -> "2.2.5",
      'scalaz     -> "7.1.2",
      'scalike    -> "2.2.7",
      'slf4j      -> "1.7.12",
      'spark      -> "1.4.0"
    ),

    updateOptions := updateOptions.value.withCachedResolution(true),

    // remove any non-nop implementation of SLF4J during tests
    (dependencyClasspath in Test) ~= { cp =>
      cp.filterNot(_.data.name.contains("slf4j-log4j12"))
        .filterNot(_.data.name.contains("logback"))
    },

    credentials += ... // loaded as usual by your system

    // Select repository to publish to based on whether the current project is a
    // SNAPSHOT or release version.
    publishTo <<= (version, sbtPlugin) { (v: String, isPlugin: Boolean) =>
      val root = "http://artifacts.example.com/sharethrough"
      val layout = if (isPlugin) Resolver.ivyStylePatterns else Resolver.mavenStylePatterns
      val status = if (v.trim.endsWith("SNAPSHOT")) "snapshot" else "release"
      val repository = s"libs-${status}s-local"

      Some(Resolver.url(repository, new URL(s"$root/$repository/"))(layout))
    }
  )
}

project/plugins.sbt

Like most SBT projects, you’d list the plugins you want to use to modify the build here. One small twist is that since this plugin defines common settings, we’d like to use the plugin’s settings for the plugin itself. We gratuitously borrow some code from the sbt-release plugin, which lets us do so before adding the other plugins as normal.

1
2
3
4
5
6
7
// This project is its own plugin :)
unmanagedSourceDirectories in Compile += baseDirectory.value.getParentFile / "src" / "main" / "scala"

addSbtPlugin("com.timushev.sbt"  %  "sbt-updates"           % "0.1.9")
addSbtPlugin("com.github.gseitz" %  "sbt-release"           % "1.0.1")
addSbtPlugin("org.scalastyle"    %% "scalastyle-sbt-plugin" % "0.7.0")
addSbtPlugin("net.virtual-void"  %  "sbt-dependency-graph"  % "0.7.5")

build.sbt

Since the project is pulling itself in as a plugin the build file itself is much more focused, providing:

the project name
the flag indicating project as a plugin
the AutoPlugins we want to aggregate and provide to users of this plugin
the publishing style (needed in our case, might not be universal)

For item 3, this differs from the plugins listed in project/plugins.sbt, in that those modify the build of the plugin, while this list represents the plugins made available to the projects which load this plugin.

When we would publish using the Maven layout, we found that our repository had issuess, layout so we disable that here and in the SbtBaseSettings.sbt publishTo project settings. YMMV whether this will be necessary for your own projects.

1
2
3
4
5
6
7
8
9
10
11
12
lazy val `sbt-base-settings` = project in file(".")

name := "sbt-base-settings"

sbtPlugin := true
publishMavenStyle := false

addSbtPlugin("com.timushev.sbt"  %  "sbt-updates"           % "0.1.9")
addSbtPlugin("com.github.gseitz" %  "sbt-release"           % "1.0.1")
addSbtPlugin("org.scoverage"     %  "sbt-scoverage"         % "1.2.0")
addSbtPlugin("org.scalastyle"    %% "scalastyle-sbt-plugin" % "0.7.0")
addSbtPlugin("net.virtual-void"  %  "sbt-dependency-graph"  % "0.7.5")

version.sbt

Just a single line to define the project’s version. Required by the sbt-release plugin so that it can increment the version appropriately during the release process.

1
version in ThisBuild := "1.1.8-SNAPSHOT"

Usage

After the plugin has been built by our CI server and published to our artifact repository, it’s available to projects that it. Since we’re hosting it in our private artifact repository we do need to have a couple extra steps to add the resolver and credentials. In a child project’s project/plugins.sbt we lead off with:

1
2
3
resolvers += Resolver.url("libs-releases-local", url("http://artifacts.example.com/sharethrough/libs-releases-local")(Resolver.ivyStylePatterns)

credentials += ... // loaded as usual by your system

Once that’s done then the plugin can be loaded like any other

1
addSbtPlugin("com.sharethrough" %% "sbt-base-settings" % "1.1.8-SNAPSHOT")

For most settings, that’s all you need. In order to get something useful out of the libraryVersions setting you’d need to slightly change the way that you’d specify libraryDependencies. Typically you’d append a sequence of those dependencies using something akin to

1
2
3
4
5
6
libraryDependencies ++= Seq(
  "com.github.nscala-time" %% "nscala-time" % "2.0.0",
  "com.jsuereth"           %% "scala-arm"   % "1.4",
  "com.typesafe"           %  "config"      % "1.2.1",
  "org.apache.spark"       %% "spark-core"  % "1.4.0" % "provided"
)

And instead of the ++= operator you’d use the <++= operator. This is similar to ++=, except that the right-hand side takes a SettingKey and maps over it to produce a Seq. This lets us access the Map[Symbol, String] defined in the SettingKey (aliased to v in this snippet):

1
2
3
4
5
libraryDependencies <++= libraryVersions { v => Seq(
  "com.github.nscala-time" %% "nscala-time" % v('nscalaTime),
  "com.typesafe"           %  "config"      % "1.2.1",
  "org.apache.spark"       %% "spark-core"  % v('spark) % "provided"
)}

While you can mix libraryVersions values with versions as plain strings, we prefer moving all versions into the libraryVersions setting for greater consistency. Plus, it’s easier to see when projects augment or override default versions:

1
2
3
4
5
6
7
8
9
libraryVersions ++= Map(
  'config -> "1.2.1"
)

libraryDependencies <++= libraryVersions { v => Seq(
  "com.github.nscala-time" %% "nscala-time" % v('nscalaTime),
  "com.typesafe"           %  "config"      % v('config),
  "org.apache.spark"       %% "spark-core"  % v('spark) % "provided"
)}

Does this sound interesting?