Reproducible builds, composing different CBT version and various improvements

One large commit, because it is was hard to do these things in isolation or to separate them now.

CBT now knows how to load other versions of itself
 - Support for reproducible builds (!), by providing a CBT git URL and hash to tie build to
 - Support for composing builds using different CBT versions (!)
 - introduce (in compatibility/) Java interfaces all CBT versions need  to stay compatible with, so they can talk to each other. And put extension methods to these interfaces in cbt package object

Class loading
 - add some sanity checks for class loading
 - improve class loader invalidation to fix bugs
 - implement caching in Java land class loaders. In particular to prevent the system class loader to repeatedly generate ClassNotFound exceptions in each sink of the class loader DAG for non JDK classes (meaning major speed up for projects with many classes).
 - getting rid of transient class loader cache unifying into "persistent" one instead (which is still wrong as invalidation eventually needs to invalidate entire sub graphs of the class loading DAG, not single class loaders. Seems like we'll have to abandon the hashmap based approach and tie caching to dependency objects)

Other Caching
 - cache dependencies extracted from xml files, which was one major time killer, but invalidate cache when cbt changed (maven dependency user facing api needs simplification now!)
 - memorize last successful compile time in the file system rather than memory, to guard against unnecessary recompiling even across launches (or when using cbt direct)

Structural improvements
 - Factor out ClassLoaderCache on Java land into its own class.
 - Port MultiClassLoader to Java land, to better compose classloaders in NailgunLauncher.
 - Remove many global constants and variables (in object paths and in NailgunLauncher) and pass them through instead. Needed for composing of builds.
 - move more code from resolver into Lib for less entanglement with classes (needed to compatibility interfaces) and better re-usability
 - remove canBeCached. Everything can be cached now, but we need to be careful about correct invalidation.
 - remove build announcing produced jars. We can add if ever needed.
 - change callNullary to return exit code instead of Unit as preparation for next commit introducing "recursive"

ScalaTest
 - Makes ScalaTest support work (still a bit too inflexible, but mostly works well)

6 files changed, 363 insertions, 178 deletions

diff --git a/nailgun_launcher/CBTUrlClassLoader.java b/nailgun_launcher/CBTUrlClassLoader.java
index 9c41978..b799bc0 100644
--- a/nailgun_launcher/CBTUrlClassLoader.java
+++ b/nailgun_launcher/CBTUrlClassLoader.java
@@ -11,7 +11,7 @@ class CbtURLClassLoader extends java.net.URLClassLoader{
       + "(\n  "
       + Arrays.toString(getURLs())
       + ",\n  "
-      + join("\n  ",getParent().toString().split("\n"))
+      + join("\n  ",(getParent() == null?"":getParent().toString()).split("\n"))
       + "\n)"
     );
   }
@@ -53,7 +53,7 @@ class CbtURLClassLoader extends java.net.URLClassLoader{
     assertExist(urls);
   }
   public CbtURLClassLoader(URL[] urls){
-    super(urls);
+    super(urls, null);
     assertExist(urls);
   }
 }
\ No newline at end of file
diff --git a/nailgun_launcher/ClassLoaderCache2.java b/nailgun_launcher/ClassLoaderCache2.java
new file mode 100644
index 0000000..bf9ca3b
--- /dev/null
+++ b/nailgun_launcher/ClassLoaderCache2.java
@@ -0,0 +1,37 @@
+package cbt;
+
+import java.util.*;
+import java.util.concurrent.ConcurrentHashMap;
+import static java.io.File.pathSeparator;
+import static cbt.Stage0Lib.*;
+
+final class ClassLoaderCache2<T>{
+  ConcurrentHashMap<String,Object> keys;
+  ConcurrentHashMap<Object,T> values;
+
+  public ClassLoaderCache2(
+    ConcurrentHashMap<String,Object> keys,
+    ConcurrentHashMap<Object,T> values
+  ){
+    this.keys = keys;
+    this.values = values;
+  }
+
+  public T get( String key ){
+    return values.get(
+      keys.get( key )
+    );
+  }
+  
+  public Boolean contains( String key ){
+    return keys.containsKey( key );
+  }
+
+  public T put( T value, String key ){
+    LockableKey2 keyObject = new LockableKey2();
+    keys.put( key, keyObject );
+    values.put( keyObject, value );
+    return value;
+  }
+}
+class LockableKey2{}
\ No newline at end of file
diff --git a/nailgun_launcher/EarlyDependencies.java b/nailgun_launcher/EarlyDependencies.java
index f4d446c..7a8d033 100644
--- a/nailgun_launcher/EarlyDependencies.java
+++ b/nailgun_launcher/EarlyDependencies.java
@@ -10,90 +10,133 @@ import static cbt.NailgunLauncher.*;
 class EarlyDependencies{
 
   /** ClassLoader for stage1 */
-  ClassLoader stage1;
+  ClassLoader classLoader;
+  String[] classpathArray;
   /** ClassLoader for zinc */
   ClassLoader zinc;
 
-  String scalaReflect_2_11_8_File = MAVEN_CACHE + "/org/scala-lang/scala-reflect/2.11.8/scala-reflect-2.11.8.jar";
-  String scalaCompiler_2_11_8_File = MAVEN_CACHE + "/org/scala-lang/scala-compiler/2.11.8/scala-compiler-2.11.8.jar";
-  String scalaXml_1_0_5_File = MAVEN_CACHE + "/org/scala-lang/modules/scala-xml_2.11/1.0.5/scala-xml_2.11-1.0.5.jar";
-  String scalaLibrary_2_11_8_File = MAVEN_CACHE + "/org/scala-lang/scala-library/2.11.8/scala-library-2.11.8.jar";
-  String zinc_0_3_9_File = MAVEN_CACHE + "/com/typesafe/zinc/zinc/0.3.9/zinc-0.3.9.jar";
-  String incrementalCompiler_0_13_9_File = MAVEN_CACHE + "/com/typesafe/sbt/incremental-compiler/0.13.9/incremental-compiler-0.13.9.jar";
-  String compilerInterface_0_13_9_File = MAVEN_CACHE + "/com/typesafe/sbt/compiler-interface/0.13.9/compiler-interface-0.13.9-sources.jar";
-  String scalaCompiler_2_10_5_File = MAVEN_CACHE + "/org/scala-lang/scala-compiler/2.10.5/scala-compiler-2.10.5.jar";
-  String sbtInterface_0_13_9_File = MAVEN_CACHE + "/com/typesafe/sbt/sbt-interface/0.13.9/sbt-interface-0.13.9.jar";
-  String scalaReflect_2_10_5_File = MAVEN_CACHE + "/org/scala-lang/scala-reflect/2.10.5/scala-reflect-2.10.5.jar";
-  String scalaLibrary_2_10_5_File = MAVEN_CACHE + "/org/scala-lang/scala-library/2.10.5/scala-library-2.10.5.jar";
-
-  public EarlyDependencies() throws MalformedURLException, IOException, NoSuchAlgorithmException{
-    download(new URL(MAVEN_URL + "/org/scala-lang/scala-reflect/2.11.8/scala-reflect-2.11.8.jar"), Paths.get(scalaReflect_2_11_8_File), "b74530deeba742ab4f3134de0c2da0edc49ca361");
-    download(new URL(MAVEN_URL + "/org/scala-lang/scala-compiler/2.11.8/scala-compiler-2.11.8.jar"), Paths.get(scalaCompiler_2_11_8_File), "fe1285c9f7b58954c5ef6d80b59063569c065e9a");
+  String scalaReflect_2_11_8_File;
+  String scalaCompiler_2_11_8_File;
+  String scalaXml_1_0_5_File;
+  String scalaLibrary_2_11_8_File;
+  String zinc_0_3_9_File;
+  String incrementalCompiler_0_13_9_File;
+  String compilerInterface_0_13_9_File;
+  String scalaCompiler_2_10_5_File;
+  String sbtInterface_0_13_9_File;
+  String scalaReflect_2_10_5_File;
+  String scalaLibrary_2_10_5_File;
+
+  public EarlyDependencies(
+    String mavenCache, String mavenUrl, ClassLoaderCache2<ClassLoader> classLoaderCache, ClassLoader rootClassLoader
+  ) throws Exception {
+    scalaReflect_2_11_8_File = mavenCache + "/org/scala-lang/scala-reflect/2.11.8/scala-reflect-2.11.8.jar";
+    scalaCompiler_2_11_8_File = mavenCache + "/org/scala-lang/scala-compiler/2.11.8/scala-compiler-2.11.8.jar";
+    scalaXml_1_0_5_File = mavenCache + "/org/scala-lang/modules/scala-xml_2.11/1.0.5/scala-xml_2.11-1.0.5.jar";
+    scalaLibrary_2_11_8_File = mavenCache + "/org/scala-lang/scala-library/2.11.8/scala-library-2.11.8.jar";
+    zinc_0_3_9_File = mavenCache + "/com/typesafe/zinc/zinc/0.3.9/zinc-0.3.9.jar";
+    incrementalCompiler_0_13_9_File = mavenCache + "/com/typesafe/sbt/incremental-compiler/0.13.9/incremental-compiler-0.13.9.jar";
+    compilerInterface_0_13_9_File = mavenCache + "/com/typesafe/sbt/compiler-interface/0.13.9/compiler-interface-0.13.9-sources.jar";
+    scalaCompiler_2_10_5_File = mavenCache + "/org/scala-lang/scala-compiler/2.10.5/scala-compiler-2.10.5.jar";
+    sbtInterface_0_13_9_File = mavenCache + "/com/typesafe/sbt/sbt-interface/0.13.9/sbt-interface-0.13.9.jar";
+    scalaReflect_2_10_5_File = mavenCache + "/org/scala-lang/scala-reflect/2.10.5/scala-reflect-2.10.5.jar";
+    scalaLibrary_2_10_5_File = mavenCache + "/org/scala-lang/scala-library/2.10.5/scala-library-2.10.5.jar";
+
+    download(new URL(mavenUrl + "/org/scala-lang/scala-reflect/2.11.8/scala-reflect-2.11.8.jar"), Paths.get(scalaReflect_2_11_8_File), "b74530deeba742ab4f3134de0c2da0edc49ca361");
+    download(new URL(mavenUrl + "/org/scala-lang/scala-compiler/2.11.8/scala-compiler-2.11.8.jar"), Paths.get(scalaCompiler_2_11_8_File), "fe1285c9f7b58954c5ef6d80b59063569c065e9a");
 
     // org.scala-lang:scala-library:2.10.5
-    download(new URL(MAVEN_URL + "/org/scala-lang/scala-library/2.10.5/scala-library-2.10.5.jar"), Paths.get(scalaLibrary_2_10_5_File), "57ac67a6cf6fd591e235c62f8893438e8d10431d");
-    ClassLoader scalaLibrary_2_10_5_ = cachePut(
-      classLoader( scalaLibrary_2_10_5_File ),
-      scalaLibrary_2_10_5_File
-    );
+    download(new URL(mavenUrl + "/org/scala-lang/scala-library/2.10.5/scala-library-2.10.5.jar"), Paths.get(scalaLibrary_2_10_5_File), "57ac67a6cf6fd591e235c62f8893438e8d10431d");
+
+    String[] scalaLibrary_2_10_5_ClasspathArray = new String[]{scalaLibrary_2_10_5_File};
+    String scalaLibrary_2_10_5_Classpath = classpath( scalaLibrary_2_10_5_ClasspathArray );
+    ClassLoader scalaLibrary_2_10_5_ =
+      classLoaderCache.contains( scalaLibrary_2_10_5_Classpath )
+      ? classLoaderCache.get( scalaLibrary_2_10_5_Classpath )
+      : classLoaderCache.put( classLoader( scalaLibrary_2_10_5_File, rootClassLoader ), scalaLibrary_2_10_5_Classpath );
 
     // org.scala-lang:scala-reflect:2.10.5
-    download(new URL(MAVEN_URL + "/org/scala-lang/scala-reflect/2.10.5/scala-reflect-2.10.5.jar"), Paths.get(scalaReflect_2_10_5_File), "7392facb48876c67a89fcb086112b195f5f6bbc3");
-    ClassLoader scalaReflect_2_10_5_ = cachePut(
-      classLoader( scalaReflect_2_10_5_File, scalaLibrary_2_10_5_ ),
-      scalaLibrary_2_10_5_File, scalaReflect_2_10_5_File
-    );
+    download(new URL(mavenUrl + "/org/scala-lang/scala-reflect/2.10.5/scala-reflect-2.10.5.jar"), Paths.get(scalaReflect_2_10_5_File), "7392facb48876c67a89fcb086112b195f5f6bbc3");
+
+    String[] scalaReflect_2_10_5_ClasspathArray = new String[]{scalaLibrary_2_10_5_File, scalaReflect_2_10_5_File};
+    String scalaReflect_2_10_5_Classpath = classpath( scalaReflect_2_10_5_ClasspathArray );
+    ClassLoader scalaReflect_2_10_5_ =
+      classLoaderCache.contains( scalaReflect_2_10_5_Classpath )
+      ? classLoaderCache.get( scalaReflect_2_10_5_Classpath )
+      : classLoaderCache.put( classLoader( scalaReflect_2_10_5_File, scalaLibrary_2_10_5_ ), scalaReflect_2_10_5_Classpath );
 
     // com.typesafe.sbt:sbt-interface:0.13.9
-    download(new URL(MAVEN_URL + "/com/typesafe/sbt/sbt-interface/0.13.9/sbt-interface-0.13.9.jar"), Paths.get(sbtInterface_0_13_9_File), "29848631415402c81b732e919be88f268df37250");
-    ClassLoader sbtInterface_0_13_9_ = cachePut(
-      classLoader( sbtInterface_0_13_9_File, scalaReflect_2_10_5_ ),
-      sbtInterface_0_13_9_File, scalaLibrary_2_10_5_File, scalaReflect_2_10_5_File
-    );
+    download(new URL(mavenUrl + "/com/typesafe/sbt/sbt-interface/0.13.9/sbt-interface-0.13.9.jar"), Paths.get(sbtInterface_0_13_9_File), "29848631415402c81b732e919be88f268df37250");
+
+    String[] sbtInterface_0_13_9_ClasspathArray = new String[]{sbtInterface_0_13_9_File, scalaLibrary_2_10_5_File, scalaReflect_2_10_5_File};
+    String sbtInterface_0_13_9_Classpath = classpath( sbtInterface_0_13_9_ClasspathArray );
+    ClassLoader sbtInterface_0_13_9_ =
+      classLoaderCache.contains( sbtInterface_0_13_9_Classpath )
+      ? classLoaderCache.get( sbtInterface_0_13_9_Classpath )
+      : classLoaderCache.put( classLoader( sbtInterface_0_13_9_File, scalaReflect_2_10_5_ ), sbtInterface_0_13_9_Classpath );
 
     // org.scala-lang:scala-compiler:2.10.5
-    download(new URL(MAVEN_URL + "/org/scala-lang/scala-compiler/2.10.5/scala-compiler-2.10.5.jar"), Paths.get(scalaCompiler_2_10_5_File), "f0f5bb444ca26a6e489af3dd35e24f7e2d2d118e");
-    ClassLoader scalaCompiler_2_10_5_ = cachePut(
-      classLoader( scalaCompiler_2_10_5_File, sbtInterface_0_13_9_ ),
-      sbtInterface_0_13_9_File, scalaCompiler_2_10_5_File, scalaLibrary_2_10_5_File, scalaReflect_2_10_5_File
-    );
+    download(new URL(mavenUrl + "/org/scala-lang/scala-compiler/2.10.5/scala-compiler-2.10.5.jar"), Paths.get(scalaCompiler_2_10_5_File), "f0f5bb444ca26a6e489af3dd35e24f7e2d2d118e");
+
+    String[] scalaCompiler_2_10_5_ClasspathArray = new String[]{sbtInterface_0_13_9_File, scalaCompiler_2_10_5_File, scalaLibrary_2_10_5_File, scalaReflect_2_10_5_File};
+    String scalaCompiler_2_10_5_Classpath = classpath( scalaCompiler_2_10_5_ClasspathArray );
+    ClassLoader scalaCompiler_2_10_5_ =
+      classLoaderCache.contains( scalaCompiler_2_10_5_Classpath )
+      ? classLoaderCache.get( scalaCompiler_2_10_5_Classpath )
+      : classLoaderCache.put( classLoader( scalaCompiler_2_10_5_File, sbtInterface_0_13_9_ ), scalaCompiler_2_10_5_Classpath );
 
     // com.typesafe.sbt:compiler-interface:0.13.9
-    download(new URL(MAVEN_URL + "/com/typesafe/sbt/compiler-interface/0.13.9/compiler-interface-0.13.9-sources.jar"), Paths.get(compilerInterface_0_13_9_File), "2311addbed1182916ad00f83c57c0eeca1af382b");
-    ClassLoader compilerInterface_0_13_9_ = cachePut(
-      classLoader( compilerInterface_0_13_9_File, scalaCompiler_2_10_5_ ),
-      compilerInterface_0_13_9_File, sbtInterface_0_13_9_File, scalaCompiler_2_10_5_File, scalaLibrary_2_10_5_File, scalaReflect_2_10_5_File
-    );
+    download(new URL(mavenUrl + "/com/typesafe/sbt/compiler-interface/0.13.9/compiler-interface-0.13.9-sources.jar"), Paths.get(compilerInterface_0_13_9_File), "2311addbed1182916ad00f83c57c0eeca1af382b");
+
+    String[] compilerInterface_0_13_9_ClasspathArray = new String[]{compilerInterface_0_13_9_File, sbtInterface_0_13_9_File, scalaCompiler_2_10_5_File, scalaLibrary_2_10_5_File, scalaReflect_2_10_5_File};
+    String compilerInterface_0_13_9_Classpath = classpath( compilerInterface_0_13_9_ClasspathArray );
+    ClassLoader compilerInterface_0_13_9_ =
+      classLoaderCache.contains( compilerInterface_0_13_9_Classpath )
+      ? classLoaderCache.get( compilerInterface_0_13_9_Classpath )
+      : classLoaderCache.put( classLoader( compilerInterface_0_13_9_File, scalaCompiler_2_10_5_ ), compilerInterface_0_13_9_Classpath );
 
     // com.typesafe.sbt:incremental-compiler:0.13.9
-    download(new URL(MAVEN_URL + "/com/typesafe/sbt/incremental-compiler/0.13.9/incremental-compiler-0.13.9.jar"), Paths.get(incrementalCompiler_0_13_9_File), "fbbf1cadbed058aa226643e83543c35de43b13f0");
-    ClassLoader incrementalCompiler_0_13_9_ = cachePut(
-      classLoader( incrementalCompiler_0_13_9_File, compilerInterface_0_13_9_ ),
-      compilerInterface_0_13_9_File, incrementalCompiler_0_13_9_File, sbtInterface_0_13_9_File, scalaCompiler_2_10_5_File, scalaLibrary_2_10_5_File, scalaReflect_2_10_5_File
-    );
+    download(new URL(mavenUrl + "/com/typesafe/sbt/incremental-compiler/0.13.9/incremental-compiler-0.13.9.jar"), Paths.get(incrementalCompiler_0_13_9_File), "fbbf1cadbed058aa226643e83543c35de43b13f0");
+
+    String[] incrementalCompiler_0_13_9_ClasspathArray = new String[]{compilerInterface_0_13_9_File, incrementalCompiler_0_13_9_File, sbtInterface_0_13_9_File, scalaCompiler_2_10_5_File, scalaLibrary_2_10_5_File, scalaReflect_2_10_5_File};
+    String incrementalCompiler_0_13_9_Classpath = classpath( incrementalCompiler_0_13_9_ClasspathArray );
+    ClassLoader incrementalCompiler_0_13_9_ =
+      classLoaderCache.contains( incrementalCompiler_0_13_9_Classpath )
+      ? classLoaderCache.get( incrementalCompiler_0_13_9_Classpath )
+      : classLoaderCache.put( classLoader( incrementalCompiler_0_13_9_File, compilerInterface_0_13_9_ ), incrementalCompiler_0_13_9_Classpath );
 
     // com.typesafe.zinc:zinc:0.3.9
-    download(new URL(MAVEN_URL + "/com/typesafe/zinc/zinc/0.3.9/zinc-0.3.9.jar"), Paths.get(zinc_0_3_9_File), "46a4556d1f36739879f4b2cc19a73d12b3036e9a");
-    ClassLoader zinc_0_3_9_ = cachePut(
-      classLoader( zinc_0_3_9_File, incrementalCompiler_0_13_9_ ),
-      compilerInterface_0_13_9_File, incrementalCompiler_0_13_9_File, sbtInterface_0_13_9_File, zinc_0_3_9_File, scalaCompiler_2_10_5_File, scalaLibrary_2_10_5_File, scalaReflect_2_10_5_File
-    );
+    download(new URL(mavenUrl + "/com/typesafe/zinc/zinc/0.3.9/zinc-0.3.9.jar"), Paths.get(zinc_0_3_9_File), "46a4556d1f36739879f4b2cc19a73d12b3036e9a");
+
+    String[] zinc_0_3_9_ClasspathArray = new String[]{compilerInterface_0_13_9_File, incrementalCompiler_0_13_9_File, sbtInterface_0_13_9_File, zinc_0_3_9_File, scalaCompiler_2_10_5_File, scalaLibrary_2_10_5_File, scalaReflect_2_10_5_File};
+    String zinc_0_3_9_Classpath = classpath( zinc_0_3_9_ClasspathArray );
+    ClassLoader zinc_0_3_9_ =
+      classLoaderCache.contains( zinc_0_3_9_Classpath )
+      ? classLoaderCache.get( zinc_0_3_9_Classpath )
+      : classLoaderCache.put( classLoader( zinc_0_3_9_File, incrementalCompiler_0_13_9_ ), zinc_0_3_9_Classpath );
 
     // org.scala-lang:scala-library:2.11.8
-    download(new URL(MAVEN_URL + "/org/scala-lang/scala-library/2.11.8/scala-library-2.11.8.jar"), Paths.get(scalaLibrary_2_11_8_File), "ddd5a8bced249bedd86fb4578a39b9fb71480573");
-    ClassLoader scalaLibrary_2_11_8_ = cachePut(
-      classLoader( scalaLibrary_2_11_8_File ),
-      scalaLibrary_2_11_8_File
-    );
+    download(new URL(mavenUrl + "/org/scala-lang/scala-library/2.11.8/scala-library-2.11.8.jar"), Paths.get(scalaLibrary_2_11_8_File), "ddd5a8bced249bedd86fb4578a39b9fb71480573");
+
+    String[] scalaLibrary_2_11_8_ClasspathArray = new String[]{scalaLibrary_2_11_8_File};
+    String scalaLibrary_2_11_8_Classpath = classpath( scalaLibrary_2_11_8_ClasspathArray );
+    ClassLoader scalaLibrary_2_11_8_ =
+      classLoaderCache.contains( scalaLibrary_2_11_8_Classpath )
+      ? classLoaderCache.get( scalaLibrary_2_11_8_Classpath )
+      : classLoaderCache.put( classLoader( scalaLibrary_2_11_8_File, rootClassLoader ), scalaLibrary_2_11_8_Classpath );
 
     // org.scala-lang.modules:scala-xml_2.11:1.0.5
-    download(new URL(MAVEN_URL + "/org/scala-lang/modules/scala-xml_2.11/1.0.5/scala-xml_2.11-1.0.5.jar"), Paths.get(scalaXml_1_0_5_File), "77ac9be4033768cf03cc04fbd1fc5e5711de2459");
-    ClassLoader scalaXml_1_0_5_ = cachePut(
-      classLoader( scalaXml_1_0_5_File, scalaLibrary_2_11_8_ ),
-      scalaXml_1_0_5_File, scalaLibrary_2_11_8_File
-    );
+    download(new URL(mavenUrl + "/org/scala-lang/modules/scala-xml_2.11/1.0.5/scala-xml_2.11-1.0.5.jar"), Paths.get(scalaXml_1_0_5_File), "77ac9be4033768cf03cc04fbd1fc5e5711de2459");
+
+    String[] scalaXml_1_0_5_ClasspathArray = new String[]{scalaXml_1_0_5_File, scalaLibrary_2_11_8_File};
+    String scalaXml_1_0_5_Classpath = classpath( scalaXml_1_0_5_ClasspathArray );
+    ClassLoader scalaXml_1_0_5_ =
+      classLoaderCache.contains( scalaXml_1_0_5_Classpath )
+      ? classLoaderCache.get( scalaXml_1_0_5_Classpath )
+      : classLoaderCache.put( classLoader( scalaXml_1_0_5_File, scalaLibrary_2_11_8_ ), scalaXml_1_0_5_Classpath );
   
-    stage1 = scalaXml_1_0_5_;
+    classLoader = scalaXml_1_0_5_;
+    classpathArray = scalaXml_1_0_5_ClasspathArray;
 
     zinc = zinc_0_3_9_;
   }
diff --git a/nailgun_launcher/MultiClassLoader2.java b/nailgun_launcher/MultiClassLoader2.java
new file mode 100644
index 0000000..fadd963
--- /dev/null
+++ b/nailgun_launcher/MultiClassLoader2.java
@@ -0,0 +1,29 @@
+package cbt;
+import java.net.*;
+import java.util.*;
+
+public class MultiClassLoader2 extends ClassLoader{
+  public ClassLoader[] parents;
+  public ClassLoader[] parents(){
+    return this.parents;
+  }
+  public MultiClassLoader2(ClassLoader... parents){
+    super(null);
+    this.parents = parents;
+  }
+  public Class findClass(String name) throws ClassNotFoundException{
+    for(ClassLoader parent: parents){
+      try{
+        return parent.loadClass(name);
+      } catch (ClassNotFoundException e) {
+        if(e.getMessage() != name) throw e;
+      }
+    }
+    // FIXME: have a logger in Java land
+    // System.err.println("NOT FOUND: "+name);
+    return null;
+  }
+  public String toString(){
+    return super.toString() + "(" + Arrays.toString(parents) +")";
+  }
+}
diff --git a/nailgun_launcher/NailgunLauncher.java b/nailgun_launcher/NailgunLauncher.java
index 8838543..33ae4c2 100644
--- a/nailgun_launcher/NailgunLauncher.java
+++ b/nailgun_launcher/NailgunLauncher.java
@@ -6,6 +6,7 @@ import java.security.*;
 import java.util.*;
 import java.util.concurrent.ConcurrentHashMap;
 import static cbt.Stage0Lib.*;
+import static java.io.File.pathSeparator;
 
 /**
  * This launcher allows to start the JVM without loading anything else permanently into its
@@ -13,43 +14,40 @@ import static cbt.Stage0Lib.*;
  * dependencies outside the JDK.
  */
 public class NailgunLauncher{
+  /** Persistent cache for caching classloaders for the JVM life time. */
+  private final static ClassLoaderCache2<ClassLoader> classLoaderCache = new ClassLoaderCache2<ClassLoader>(
+    new ConcurrentHashMap<String,Object>(),
+    new ConcurrentHashMap<Object,ClassLoader>()
+  );
 
-  public static String CBT_HOME = System.getenv("CBT_HOME");
-  public static String NAILGUN = System.getenv("NAILGUN");
-  public static String TARGET = System.getenv("TARGET");
-  public static String STAGE1 = CBT_HOME + "/stage1/";
-  public static String MAVEN_CACHE = CBT_HOME + "/cache/maven";
-  public static String MAVEN_URL = "https://repo1.maven.org/maven2";
-
-  /**
-   * Persistent cache for caching classloaders for the JVM life time. Can be used as needed by user
-   * code to improve startup time.
-   */
-  public static ConcurrentHashMap<String, Object> classLoaderCacheKeys = new ConcurrentHashMap<String,Object>();
-  public static ConcurrentHashMap<Object, ClassLoader> classLoaderCacheValues = new ConcurrentHashMap<Object,ClassLoader>();
-
-  public static SecurityManager defaultSecurityManager = System.getSecurityManager();
-
-  public static long lastSuccessfullCompile = 0;
-  static ClassLoader stage1classLoader = null;
-  public static ClassLoader stage2classLoader = null;
+  public final static SecurityManager defaultSecurityManager = System.getSecurityManager();
 
-  public static void main(String[] args) throws ClassNotFoundException,
-                                                NoSuchMethodException,
-                                                IllegalAccessException,
-                                                InvocationTargetException,
-                                                MalformedURLException,
-                                                IOException,
-                                                NoSuchAlgorithmException {
-    //System.err.println("ClassLoader: "+stage1classLoader);
-    //System.err.println("lastSuccessfullCompile: "+lastSuccessfullCompile);
-    //System.err.println("now: "+now);
-
-    _assert(CBT_HOME != null, CBT_HOME);
-    _assert(NAILGUN != null, NAILGUN);
-    _assert(TARGET != null, TARGET);
-    _assert(STAGE1 != null, STAGE1);
+  public static String TARGET = System.getenv("TARGET");
+  private static String NAILGUN = "nailgun_launcher/";
+  private static String STAGE1 = "stage1/";
+  
+  @SuppressWarnings("unchecked")
+  public static Object getBuild( Object context ) throws Exception{
+    BuildStage1Result res = buildStage1(
+      (Boolean) get(context, "cbtHasChanged"),
+      (Long) get(context, "start"),
+      ((File) get(context, "cache")).toString() + "/",
+      ((File) get(context, "cbtHome")).toString(),
+      ((File) get(context, "compatibilityTarget")).toString() + "/",
+      new ClassLoaderCache2<ClassLoader>(
+        (ConcurrentHashMap<String,Object>) get(context, "permanentKeys"),
+        (ConcurrentHashMap<Object,ClassLoader>) get(context, "permanentClassLoaders")
+      )
+    );
+    return
+      res
+        .classLoader
+        .loadClass("cbt.Stage1")
+        .getMethod( "getBuild", Object.class, Boolean.class )
+        .invoke(null, context, res.changed);
+  }
 
+  public static void main( String[] args ) throws Exception {
     Long _start = System.currentTimeMillis();
     if(args[0].equals("check-alive")){
       System.exit(33);
@@ -58,52 +56,108 @@ public class NailgunLauncher{
 
     String[] diff = args[0].split("\\.");
     long start = _start - (Long.parseLong(diff[0]) * 1000L) - Long.parseLong(diff[1]);
-    List<File> stage1SourceFiles = new ArrayList<File>();
-    for( File f: new File(STAGE1).listFiles() ){
-      if( f.isFile() && f.toString().endsWith(".scala") ){
-        stage1SourceFiles.add(f);
+    
+    _assert(System.getenv("CBT_HOME") != null, "environment variable CBT_HOME not defined");
+    String CBT_HOME = System.getenv("CBT_HOME");
+    String cache = CBT_HOME + "/cache/";
+    BuildStage1Result res = buildStage1(
+      false, start, cache, CBT_HOME, CBT_HOME + "/compatibility/" + TARGET, classLoaderCache
+    );
+
+    System.exit(
+      (Integer) res
+        .classLoader
+        .loadClass("cbt.Stage1")
+        .getMethod(
+          "run",
+          String[].class, File.class, File.class, Boolean.class,
+          Long.class, ConcurrentHashMap.class, ConcurrentHashMap.class
+        )
+        .invoke(
+          null,
+          (Object) args, new File(cache), new File(CBT_HOME), res.changed,
+          start, classLoaderCache.keys, classLoaderCache.values
+        )
+    );
+  }
+
+  public static BuildStage1Result buildStage1(
+    Boolean changed, long start, String cache, String cbtHome, String compatibilityTarget, ClassLoaderCache2<ClassLoader> classLoaderCache
+  ) throws Exception {
+    _assert(TARGET != null, "environment variable TARGET not defined");
+    String nailgunTarget = cbtHome + "/" + NAILGUN + TARGET;
+    String stage1Sources = cbtHome + "/" + STAGE1;
+    String stage1Target = stage1Sources + TARGET;
+    File compatibilitySources = new File(cbtHome + "/compatibility");
+    String mavenCache = cache + "maven";
+    String mavenUrl = "https://repo1.maven.org/maven2";
+
+    ClassLoader rootClassLoader = new CbtURLClassLoader( new URL[]{}, ClassLoader.getSystemClassLoader().getParent() ); // wrap for caching
+    EarlyDependencies earlyDeps = new EarlyDependencies(mavenCache, mavenUrl, classLoaderCache, rootClassLoader);
+
+    List<File> compatibilitySourceFiles = new ArrayList<File>();
+    for( File f: compatibilitySources.listFiles() ){
+      if( f.isFile() && (f.toString().endsWith(".scala") || f.toString().endsWith(".java")) ){
+        compatibilitySourceFiles.add(f);
       }
     }
+    changed = compile(changed, start, "", compatibilityTarget, earlyDeps, compatibilitySourceFiles, defaultSecurityManager);
+    
+    ClassLoader compatibilityClassLoader;
+    if( classLoaderCache.contains( compatibilityTarget ) ){
+      compatibilityClassLoader = classLoaderCache.get( compatibilityTarget );
+    } else {
+      compatibilityClassLoader = classLoaderCache.put( classLoader(compatibilityTarget, rootClassLoader), compatibilityTarget );
+    }
 
-    Boolean changed = lastSuccessfullCompile == 0;
-    for( File file: stage1SourceFiles ){
-      if( file.lastModified() > lastSuccessfullCompile ){
-        changed = true;
-        //System.err.println("File change: "+file.lastModified());
-        break;
-      }
+    String[] nailgunClasspathArray = append( earlyDeps.classpathArray, nailgunTarget );
+    String nailgunClasspath = classpath( nailgunClasspathArray );
+    ClassLoader nailgunClassLoader = new CbtURLClassLoader( new URL[]{}, NailgunLauncher.class.getClassLoader() ); // wrap for caching
+    if( !classLoaderCache.contains( nailgunClasspath ) ){
+      nailgunClassLoader = classLoaderCache.put( nailgunClassLoader, nailgunClasspath );
     }
 
-    if(changed){
-      EarlyDependencies earlyDeps = new EarlyDependencies();
-      int exitCode = zinc(earlyDeps, stage1SourceFiles);
-      if( exitCode == 0 ){
-        lastSuccessfullCompile = start;
-      } else {
-        System.exit( exitCode );
-      }
+    String[] stage1ClasspathArray =
+      append( append( nailgunClasspathArray, compatibilityTarget ), stage1Target );
+    String stage1Classpath = classpath( stage1ClasspathArray );
 
-      ClassLoader nailgunClassLoader;
-      if( classLoaderCacheKeys.containsKey( NAILGUN+TARGET ) ){
-        nailgunClassLoader = cacheGet( NAILGUN+TARGET );
-      } else {
-        nailgunClassLoader = cachePut( classLoader(NAILGUN+TARGET, earlyDeps.stage1), NAILGUN+TARGET ); // FIXME: key is wrong here, should be full CP
+    List<File> stage1SourceFiles = new ArrayList<File>();
+    for( File f: new File(stage1Sources).listFiles() ){
+      if( f.isFile() && f.toString().endsWith(".scala") ){
+        stage1SourceFiles.add(f);
       }
-
-      stage1classLoader = classLoader(STAGE1+TARGET, nailgunClassLoader);
-      stage2classLoader = null;
     }
+    changed = compile(changed, start, stage1Classpath, stage1Target, earlyDeps, stage1SourceFiles, defaultSecurityManager);
 
-    try{
-      Integer exitCode =
-        (Integer) stage1classLoader
-          .loadClass("cbt.Stage1")
-          .getMethod("run", String[].class, ClassLoader.class, Boolean.class, Long.class)
-          .invoke( null, (Object) args, stage1classLoader, changed, start);
-      System.exit(exitCode);
-    }catch(Exception e){
-      System.err.println(stage1classLoader);
-      throw e;
+    ClassLoader stage1classLoader;
+    if( !changed && classLoaderCache.contains( stage1Classpath ) ){
+      stage1classLoader = classLoaderCache.get( stage1Classpath );
+    } else {
+      stage1classLoader =
+      classLoaderCache.put(
+        classLoader(
+          stage1Target,
+          new MultiClassLoader2(
+            nailgunClassLoader,
+            compatibilityClassLoader,
+            earlyDeps.classLoader
+          )
+        ),
+        stage1Classpath
+      );
     }
+
+    return new BuildStage1Result(
+      changed,
+      stage1classLoader
+    );
+  }
+}
+class BuildStage1Result{
+  Boolean changed;
+  ClassLoader classLoader;
+  BuildStage1Result( Boolean changed, ClassLoader classLoader ){
+    this.changed = changed;
+    this.classLoader = classLoader;
   }
 }
diff --git a/nailgun_launcher/Stage0Lib.java b/nailgun_launcher/Stage0Lib.java
index d6f33e1..bd18748 100644
--- a/nailgun_launcher/Stage0Lib.java
+++ b/nailgun_launcher/Stage0Lib.java
@@ -8,8 +8,9 @@ import java.security.*;
 import java.util.*;
 import javax.xml.bind.annotation.adapters.HexBinaryAdapter;
 import static java.io.File.pathSeparator;
-import static cbt.Stage0Lib.*;
 import static cbt.NailgunLauncher.*;
+import java.nio.file.*;
+import java.nio.file.attribute.FileTime;
 
 public class Stage0Lib{
   public static void _assert(Boolean condition, Object msg){
@@ -18,7 +19,7 @@ public class Stage0Lib{
     }
   }
 
-  public static int runMain(String cls, String[] args, ClassLoader cl) throws ClassNotFoundException, NoSuchMethodException, IllegalAccessException, InvocationTargetException{
+  public static int runMain(String cls, String[] args, ClassLoader cl, SecurityManager defaultSecurityManager) throws Exception{
     try{
       System.setSecurityManager( new TrapSecurityManager() );
       cl.loadClass(cls)
@@ -32,63 +33,78 @@ public class Stage0Lib{
       }
       throw exception;
     } finally {
-      System.setSecurityManager(NailgunLauncher.defaultSecurityManager);
+      System.setSecurityManager(defaultSecurityManager);
     }
   }
 
-  public static int zinc( EarlyDependencies earlyDeps, List<File> sourceFiles ) throws ClassNotFoundException, NoSuchMethodException, IllegalAccessException, InvocationTargetException{
-    String cp = NAILGUN+TARGET + pathSeparator + earlyDeps.scalaXml_1_0_5_File + pathSeparator + earlyDeps.scalaLibrary_2_11_8_File;
-    List<String> zincArgs = new ArrayList<String>(
-      Arrays.asList(
-        new String[]{ 
-          "-scala-compiler", earlyDeps.scalaCompiler_2_11_8_File,
-          "-scala-library", earlyDeps.scalaLibrary_2_11_8_File,
-          "-scala-extra", earlyDeps.scalaReflect_2_11_8_File,
-          "-sbt-interface", earlyDeps.sbtInterface_0_13_9_File,
-          "-compiler-interface", earlyDeps.compilerInterface_0_13_9_File,
-          "-cp", cp,
-          "-d", STAGE1+TARGET
-        }
-      )
-    );
+  public static Object get(Object object, String method) throws Exception{
+    return object.getClass().getMethod( method ).invoke(object);
+  }
+
+  public static String classpath( String... files ){
+    Arrays.sort(files);
+    return join( pathSeparator, files );
+  }
 
-    for( File f: sourceFiles ){
-      zincArgs.add(f.toString());
+  public static Boolean compile(
+    Boolean changed, Long start, String classpath, String target,
+    EarlyDependencies earlyDeps, List<File> sourceFiles, SecurityManager defaultSecurityManager
+  ) throws Exception{
+    File statusFile = new File( new File(target) + ".last-success" );
+    Long lastSuccessfullCompile = statusFile.lastModified();
+    for( File file: sourceFiles ){
+      if( file.lastModified() > lastSuccessfullCompile ){
+        changed = true;
+        break;
+      }
     }
+    if(changed){
+      List<String> zincArgs = new ArrayList<String>(
+        Arrays.asList(
+          new String[]{ 
+            "-scala-compiler", earlyDeps.scalaCompiler_2_11_8_File,
+            "-scala-library", earlyDeps.scalaLibrary_2_11_8_File,
+            "-scala-extra", earlyDeps.scalaReflect_2_11_8_File,
+            "-sbt-interface", earlyDeps.sbtInterface_0_13_9_File,
+            "-compiler-interface", earlyDeps.compilerInterface_0_13_9_File,
+            "-cp", classpath,
+            "-d", target
+          }
+        )
+      );
 
-    PrintStream oldOut = System.out;
-    try{
-      System.setOut(System.err);
-      return runMain( "com.typesafe.zinc.Main", zincArgs.toArray(new String[zincArgs.size()]), earlyDeps.zinc );
-    } finally {
-      System.setOut(oldOut);
+      for( File f: sourceFiles ){
+        zincArgs.add(f.toString());
+      }
+
+      PrintStream oldOut = System.out;
+      try{
+        System.setOut(System.err);
+        int exitCode = runMain( "com.typesafe.zinc.Main", zincArgs.toArray(new String[zincArgs.size()]), earlyDeps.zinc, defaultSecurityManager );
+        if( exitCode == 0 ){
+          Files.write( statusFile.toPath(), "".getBytes());
+          Files.setLastModifiedTime( statusFile.toPath(), FileTime.fromMillis(start) );
+        } else {
+          System.exit( exitCode );
+        }
+      } finally {
+        System.setOut(oldOut);
+      }
     }
+    return changed;
   }
 
-  public static ClassLoader classLoader( String file ) throws MalformedURLException{
+  public static ClassLoader classLoader( String file ) throws Exception{
     return new CbtURLClassLoader(
       new URL[]{ new URL("file:"+file) }
     );
   }
-  public static ClassLoader classLoader( String file, ClassLoader parent ) throws MalformedURLException{
+  public static ClassLoader classLoader( String file, ClassLoader parent ) throws Exception{
     return new CbtURLClassLoader(
       new URL[]{ new URL("file:"+file) }, parent
     );
   }
-  public static ClassLoader cacheGet( String key ){
-    return classLoaderCacheValues.get(
-      classLoaderCacheKeys.get( key )
-    );
-  }
-  public static ClassLoader cachePut( ClassLoader classLoader, String... jars ){
-    String key = join( pathSeparator, jars );
-    Object keyObject = new Object();
-    classLoaderCacheKeys.put( key, keyObject );
-    classLoaderCacheValues.put( keyObject, classLoader );
-    return classLoader;
-  }
-
-  public static void download(URL urlString, Path target, String sha1) throws IOException, NoSuchAlgorithmException {
+  public static void download(URL urlString, Path target, String sha1) throws Exception {
     final Path unverified = Paths.get(target+".unverified");
     if(!Files.exists(target)) {
       new File(target.toString()).getParentFile().mkdirs();
@@ -107,7 +123,7 @@ public class Stage0Lib{
     }
   }
 
-  public static String sha1(byte[] bytes) throws NoSuchAlgorithmException {
+  public static String sha1(byte[] bytes) throws Exception {
     final MessageDigest sha1 = MessageDigest.getInstance("SHA1");
     sha1.update(bytes, 0, bytes.length);
     return (new HexBinaryAdapter()).marshal(sha1.digest());
@@ -120,4 +136,10 @@ public class Stage0Lib{
     }
     return result;
   }
+
+  public static String[] append( String[] array, String item ){
+    String[] copy = Arrays.copyOf(array, array.length + 1);
+    copy[array.length] = item;
+    return copy;
+  }
 }
\ No newline at end of file