Testing Storehaus with Guice

Director of Engineering, Exchange

Does this sound interesting?

We're Hiring

Testing code that depends on databases has always had twists and turns, but writing tests against large-scale key/value stores has its own set of challenges. If your app writes to Riak, should each developer’s machine have a fully-configured Riak instance? If the code writes to the data store using futures, how would you test it? If the datastore changes to Cassandra or Redis, do the tests need to be re-written?

This last point is important. Kent Beck put it best:

Making workable decisions today and maintaining the flexibility to change your mind in the future is a key to good software development.

Twitter’s Storehaus library helped us build reusable interfaces to our key-value stores, allowing us to write asynchronously while abstracting away the messy internals. If we move from our choice (typically Redis) to some other backend, we have the flexibility to make that change.

Dependency Injection helps us wire together components, so that depending on the current environment (dev, staging, production) the right concrete implementation folds seamlessly into our code. We use Guice – with help from Spiros Tzavellas’ sse-guice library – to inject dependencies into our Scala apps. To demonstrate, we built a demo Play app called Dorota.

A Simple Ticker

Dorota is a news ticker that reads RSS feeds, peels off headlines, and caches the JSON in Redis. The app then serves headlines as a service (HaaS?).

To start, let’s set up our Guice injector and Storehaus. First, you’ll need to add these dependencies:

1
2
3

 "com.google.inject" % "guice" % "3.0",
 "com.tzavellas" % "sse-guice" % "0.7.1",
 "com.twitter" %% "storehaus" % "0.6.0",

Here’s our controller:

1
2
3
4
5
6
7
8
9

@Singleton
class Feed @Inject() (retriever: FeedRetriever, processor: FeedProcessor) extends Controller {
  // ...
  def json = Action {
    // ...
    val feed = processor.headlines(source, randomFeed, retriever)
    // ...
  }
}

processor is parsing RSS feeds, extracting the headlines, converting them to JSON and caching via Storehaus to Redis. source, the friendly name of the site (e.g. Hacker News) becomes the Redis key. The @Inject annotation is Guice’s way of telling the class, “At runtime, you’ll get retriever and processor instances from some other context.”

Dorota specifies that context in ProdModule, via ScalaModule:

1
2
3
4
5
6
7
8

class ProdModule extends ScalaModule {
  def configure() {
    bind[Retriever].to[FeedRetriever]
    bind[RedisClientFactory].toInstance(new RedisClientFactory)
    bind[StorageFactory].toInstance(new RedisStorageFactory)
    bind[FeedProcessor].toInstance(new FeedProcessor)
  }
}

This module handles all our initialization, and the convenient ‘Global.scala’ file handles the direct injection of instances into the context of the running Play app. For more info on DI in Play, see the official docs.

1
2
3
4
5
6

object Global extends GlobalSettings {
  private lazy val injector = Guice.createInjector(new ProdModule)
  override def getControllerInstance[A](clazz: Class[A]) = {
    injector.getInstance(clazz) 
  }
}

Now the controller has access to injected instances.

Where Are The Tests?

You might be wondering what all this has to do with tests. With Guice handling our dependencies, we can use modules to inject our test dependencies.

This happy path spec below (using specs2) binds dependencies to an injectable module that is only utilized in the test run.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

class FeedProcessorSpec extends Specification with Mockito {
  val fakeHeadlines = List(
      "Costa Concordia salvage nears final phase - Reuters",
      "JPMorgan may pay $750M fine for 'whale' losses - USA TODAY",
      "Yellen Is Top Fed Hopeful, Yet Again - Wall Street Journal",
  )

  val fakeRetriever = mock[Retriever]
  val fakeStoreFactory = mock[StorageFactory]
  val fakeStore = mock[MergeableStore[ChannelBuffer, String]]

  fakeStoreFactory.createStore returns fakeStore

  class TestModule extends ScalaModule {
    def configure = {
      bind[Retriever].toInstance(fakeRetriever)
      bind[StorageFactory].toInstance(fakeStoreFactory)
    }
  }

  Dorota.injector = Guice.createInjector(new TestModule)

  val expectedJson = Json.toJson(Json.obj("source" -> "Google News", "headlines" -> fakeHeadlines)).toString

  "A FeedProcessor" should {
    "be able to return finished JSON" in {

      val feedProcessor = new FeedProcessor
      val key = StringToChannelBuffer("Google News")

      fakeStore.get(key) returns Future.value { Some(expectedJson) }
      val result = feedProcessor.headlines("Google News", "http://example.com", fakeRetriever)

      result mustEqual expectedJson
    }
  }
}

The processor asks the Storehaus mergable store for headlines matching “Google News,” and the mock store returns correctly. You could also move the local mocks into a separate code unit, but I like to know what kind of setup is required in my spec, so that I know where I may need to simplify.

I’ve only touched the surface of what Guice has to offer. Hopefully this kind of setup will allow you to explore its power in the context of test-driving Storehaus applications.

Additional Reading

The following resources helped us wire our test infrastructure together: