The Traveler's Guide for Offline Scala Development
Having the privilege of working from wherever you want is awesome, as everyone here at vaamo (and especially my fellow remotee Jezen) will probably agree.
But while travelling, I noticed that sbt will
sometimes, especially after issuing
clean, try to resolve artifacts in remote
repositories that already exist in the local Ivy cache and eventually abort due
to the missing Internet connection. This turned out to be a major impediment, as
it meant that I needed to be online every time I work on our codebase, even
though from my observations, sbt didn’t download anything new.
Exposing My Ignorance
The documentation states that this is not the expected behaviour, but I haven’t figured out exactly what I’m doing wrong, or if this is a bug triggered by our pretty complex build definition, or if it’s actually a bug in sbt.
The last sensible workaround seemed to actually provide sbt with a remote repository.
Creating the Docker Container
The following guide was tested using scala 2.11.5 and sbt v0.13.7.
At first, I tried to use Dock (thanks
Benjamin), but quickly noticed that repeatedly
dock nexus will start a new container every time. Still, the
marcellodesales/nexus-npm-registry Dockerfile worked pretty well, so I
stuck to that.
docker run --detach \ --publish 127.0.0.1:8081:8081 \ --name sbt-nexus \ marcellodesales/nexus-npm-registry
By running this command, we retrieve the Nexus image from Dockerhub, create a
new container (named
sbt-nexus) and start it in detached mode. Additionally, we
forward port 8081 of the container to
That’s it for the Docker container.
We can now navigate to
http://localhost:8081/nexus/ and start setting
up the repositories. For reference, the credentials for Nexus are
You can start and stop this particular container by issuing
sbt-nexus and respectively
docker stop sbt-nexus.
Setting Up The Repository
We want our Nexus repository to cache/proxy every artifact our projects need. Therefore, we have to configure all remote repositories as proxies. The default settings for a new proxy repository are fine, expect that we don’t want to download the remote index, as it’s a waste of disk space because we won’t ever use it.
Picking the needed repositories to proxy, turned out to be quite tricky. I
wasn’t sure where to look for the repositories we use, or which sbt implicitly
uses. The command
sbt resolvers yields a few repositories, also
sbtResolver can help with identifying relevant repositories.
For us, I set up the following repository structure:
ivy-releases (GROUP) |- http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/ (PROXY) |- http://repo.typesafe.com/typesafe/ivy-releases/ (PROXY) mvn-releases (GROUP) |- https://oss.sonatype.org/content/repositories/releases/ (PROXY) |- http://repo.typesafe.com/typesafe/maven-releases/ (PROXY) |- http://repo.typesafe.com/typesafe/releases/ (PROXY) |- Maven Central |- $OUR_PRIVATE_REPOSITORY/releases/ (PROXY) mvn-snapshots (GROUP) |- https://oss.sonatype.org/content/repositories/snapshots/ (PROXY) |- $OUR_PRIVATE_REPOSITORY/snapshots/ (PROXY) mvn-repositories (GROUP) |- mvn-releases (ref to GROUP) |- mvn-snapshots (ref to GROUP)
Configuring sbt To Use Our Proxy
Sbt’s documentation states, we need to configure sbt to use our freshly setup local repository instead of every other repository that might be used in our projects.
➜ ~ cat ~/.sbt/repositories [repositories] local my-ivy-proxy-releases: http://127.0.0.1:8081/nexus/content/groups/ivy-releases/, [organization]/[module]/(scala_[scalaVersion]/)(sbt_[sbtVersion]/)[revision]/[type]s/[artifact](-[classifier]).[ext] my-maven-proxy-releases: http://127.0.0.1:8081/nexus/content/groups/mvn-repositories/
This will suffice for sbt to also (or even preferably) use these repositories for resolving dependencies. The documentation is quite ambigious here, but as Robin Green mentions in this SO comment, they are only added to the list of resolvers. I can confirm this behaviour. 1
sbt.override.build.repos=true will tell sbt to ignore every other
repository other than those given in
~/.sbt/repositores. Depending on your sbt
workflow, you can set this via
SBT_OPTS or just add
it to your sbt command via
Preparing For The Trip
One thing to keep in mind here is, that our Nexus repository must act as a full proxy for every remote repository we might need to use when we’re offline.
Long story short: Using the Nexus repository is pointless if it is missing even just one artifact, as sbt will fail miserably once it tries to resolve it.
The trick is to exclusively use the nexus repository for your projects, at all times.
To achieve this, the launcher script we use has an environment variable called
which we can use to globally override all project-specific repositories. I just
added the following export to my
export JVM_OPTS="-Dsbt.override.build.repos=true $JVM_OPTS"
Just before you start your journey (or better some time before you have to leave), make sure that the Nexus repository contains all artifacts necessary for you to work offline.
I sincerely hope this guide can help others with the same issue. If someone reading this knows how to debug sbt’s offline behaviour, or even knows what I was doing wrong before, I’d very much appreciate a message via @rradczewski or mail :)
SBT Documentation on Proxy Repositories: The guide where most of this information comes from
Robin Green on Stackoverflow: The comment that brought me to the final solution
marcellodesales/nexus-npm-registry at DockerHub: The nexus container used in this guide
sbt-extras/sbt: The sbt-launcher we use at vaamo
resolversin sbt will still yield the repositories configured in the project configuration, which is probably a bug in sbt. ↩