Having the privilege of working from wherever you want is awesome, as everyone here at vaamo (and especially my fellow remotee Jezen) will probably agree.

By using git and offline-enabling tools like dash/zeal, we can work just fine away from the office, usually even without an Internet connection (who needs SO anyway).

But while travelling, I noticed that sbt will sometimes, especially after issuing clean, try to resolve artifacts in remote repositories that already exist in the local Ivy cache and eventually abort due to the missing Internet connection. This turned out to be a major impediment, as it meant that I needed to be online every time I work on our codebase, even though from my observations, sbt didn’t download anything new.

Exposing My Ignorance

My previous endeavours to make sbt work without an Internet connection failed. I tried every documented or suggested way to keep sbt from trying to resolve already cached artifacts.

The documentation states that this is not the expected behaviour, but I haven’t figured out exactly what I’m doing wrong, or if this is a bug triggered by our pretty complex build definition, or if it’s actually a bug in sbt.

Solution

The last sensible workaround seemed to actually provide sbt with a remote repository.

Using Nexus and Docker, I worked out a solution that will leave me with some small drawbacks regarding CPU and disk space, but most importantly with a working development environment when travelling.

Creating the Docker Container

The following guide was tested using scala 2.11.5 and sbt v0.13.7.

At first, I tried to use Dock (thanks Benjamin), but quickly noticed that repeatedly calling dock nexus will start a new container every time. Still, the marcellodesales/nexus-npm-registry Dockerfile worked pretty well, so I stuck to that.

docker run --detach \
    --publish 127.0.0.1:8081:8081 \
    --name sbt-nexus \
    marcellodesales/nexus-npm-registry

By running this command, we retrieve the Nexus image from Dockerhub, create a new container (named sbt-nexus) and start it in detached mode. Additionally, we forward port 8081 of the container to localhost:8081.

That’s it for the Docker container.

We can now navigate to http://localhost:8081/nexus/ and start setting up the repositories. For reference, the credentials for Nexus are admin:admin123.

You can start and stop this particular container by issuing docker start sbt-nexus and respectively docker stop sbt-nexus.

Setting Up The Repository

We want our Nexus repository to cache/proxy every artifact our projects need. Therefore, we have to configure all remote repositories as proxies. The default settings for a new proxy repository are fine, expect that we don’t want to download the remote index, as it’s a waste of disk space because we won’t ever use it.

Picking the needed repositories to proxy, turned out to be quite tricky. I wasn’t sure where to look for the repositories we use, or which sbt implicitly uses. The command sbt resolvers yields a few repositories, also sbt sbtResolver can help with identifying relevant repositories.

For us, I set up the following repository structure:

  ivy-releases (GROUP)
  |- http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/ (PROXY)
  |- http://repo.typesafe.com/typesafe/ivy-releases/ (PROXY)

  mvn-releases (GROUP)
  |- https://oss.sonatype.org/content/repositories/releases/ (PROXY)
  |- http://repo.typesafe.com/typesafe/maven-releases/ (PROXY)
  |- http://repo.typesafe.com/typesafe/releases/ (PROXY)
  |- Maven Central
  |- $OUR_PRIVATE_REPOSITORY/releases/ (PROXY)

  mvn-snapshots (GROUP)
  |- https://oss.sonatype.org/content/repositories/snapshots/ (PROXY)
  |- $OUR_PRIVATE_REPOSITORY/snapshots/ (PROXY)

  mvn-repositories (GROUP)
  |- mvn-releases (ref to GROUP)
  |- mvn-snapshots (ref to GROUP)

Configuring sbt To Use Our Proxy

Sbt’s documentation states, we need to configure sbt to use our freshly setup local repository instead of every other repository that might be used in our projects.

➜  ~  cat ~/.sbt/repositories
[repositories]
  local
  my-ivy-proxy-releases: http://127.0.0.1:8081/nexus/content/groups/ivy-releases/, [organization]/[module]/(scala_[scalaVersion]/)(sbt_[sbtVersion]/)[revision]/[type]s/[artifact](-[classifier]).[ext]
  my-maven-proxy-releases: http://127.0.0.1:8081/nexus/content/groups/mvn-repositories/

This will suffice for sbt to also (or even preferably) use these repositories for resolving dependencies. The documentation is quite ambigious here, but as Robin Green mentions in this SO comment, they are only added to the list of resolvers. I can confirm this behaviour. 1

Setting sbt.override.build.repos=true will tell sbt to ignore every other repository other than those given in ~/.sbt/repositores. Depending on your sbt workflow, you can set this via JVM_OPTS, JAVA_OPTS, SBT_OPTS or just add it to your sbt command via -Dsbt.override.build.repos=true.

Preparing For The Trip

One thing to keep in mind here is, that our Nexus repository must act as a full proxy for every remote repository we might need to use when we’re offline.

Long story short: Using the Nexus repository is pointless if it is missing even just one artifact, as sbt will fail miserably once it tries to resolve it.

The trick is to exclusively use the nexus repository for your projects, at all times. To achieve this, the launcher script we use has an environment variable called JVM_OPTS, which we can use to globally override all project-specific repositories. I just added the following export to my ~/.zshrc:

export JVM_OPTS="-Dsbt.override.build.repos=true $JVM_OPTS"

Just before you start your journey (or better some time before you have to leave), make sure that the Nexus repository contains all artifacts necessary for you to work offline.

The End

I sincerely hope this guide can help others with the same issue. If someone reading this knows how to debug sbt’s offline behaviour, or even knows what I was doing wrong before, I’d very much appreciate a message via @rradczewski or mail :)

Important Resources

SBT Documentation on Proxy Repositories: The guide where most of this information comes from

Robin Green on Stackoverflow: The comment that brought me to the final solution

marcellodesales/nexus-npm-registry at DockerHub: The nexus container used in this guide

sbt-extras/sbt: The sbt-launcher we use at vaamo

  1. Calling resolvers in sbt will still yield the repositories configured in the project configuration, which is probably a bug in sbt.