From 70cb911f54c8b049b23d77e28048fd4cdb217454 Mon Sep 17 00:00:00 2001 From: Grzegorz Kossakowski Date: Wed, 11 Feb 2015 18:39:44 +0100 Subject: Introduce sbt build It mimics the ant build as closely as necessary to compare the produced artifacts between the two builds, to build confidence in the transition: - all projects are built into the same directories in build/quick - include legacy projects: forkjoin, asm, actors - TODO: Include forkjoin & asm into library & compiler jars, respectively. We don't carry over ant's built-in bootstrapping process; this will be scripted outside of the build, publishing layers locally. NOTE: the root project cannot be named `scala`: it shadows the `scala` package in e.g., `projectConsole`. --- .gitignore | 4 ++ build.sbt | 144 +++++++++++++++++++++++++++++++++++++++++++++++ project/build.properties | 1 + 3 files changed, 149 insertions(+) create mode 100644 build.sbt create mode 100644 project/build.properties diff --git a/.gitignore b/.gitignore index 20d700dd12..e3bc794f08 100644 --- a/.gitignore +++ b/.gitignore @@ -48,3 +48,7 @@ # Standard symbolic link to build/quick/bin /qbin + +# Sbt's target directories +**/target/ +/build-sbt/ diff --git a/build.sbt b/build.sbt new file mode 100644 index 0000000000..740d823435 --- /dev/null +++ b/build.sbt @@ -0,0 +1,144 @@ +/* + * The new, sbt-based build definition for Scala. + * + * What you see below is very much work-in-progress. Basics like compiling and packaging jars + * (into right location) work. Everything else is missing: + * building docs, placing shell scripts in right locations (so you can run compiler easily), + * running partest test, compiling and running JUnit test, and many, many other things. + * + * You'll notice that this build definition is much more complicated than your typical sbt build. + * The main reason is that we are not benefiting from sbt's conventions when it comes project + * layout. For that reason we have to configure a lot more explicitly. I've tried explain in + * comments the less obvious settings. + * + * This nicely leads me to explaning goal and non-goals of this build definition. Goals are: + * + * - to be easy to tweak it in case a bug or small inconsistency is found + * - to mimic Ant's behavior as closely as possible + * - to be super explicit about any departure from standard sbt settings + * - to achieve functional parity with Ant build as quickly as possible + * - to be readable and not necessarily succint + * - to provide the nicest development experience for people hacking on Scala + * + * Non-goals are: + * + * - to have the shortest sbt build definition possible; we'll beat Ant definition + * easily and that will thrill us already + * - to remove irregularities from our build process right away + * - to modularize the Scala compiler or library further + * + * It boils down to simple rules: + * + * - project laytout is set in stone for now + * - if you need to work on convincing sbt to follow non-standard layout then + * explain everything you did in comments + * - constantly check where Ant build produces class files, artifacts, what kind of other + * files generates and port all of that to here + * + * Note on bootstrapping: + * + * Let's start with reminder what bootstrapping means in our context. It's an answer + * to this question: which version of Scala are using to compile Scala? The fact that + * the question sounds circular suggests trickiness. Indeed, bootstrapping Scala + * compiler is a tricky process. + * + * Ant build used to have involved system of bootstrapping Scala. It would consist of + * three layers: starr, locker and quick. The sbt build for Scala ditches layering + * and strives to be as standard sbt project as possible. This means that we are simply + * building Scala with latest stable release of Scala. + * See this discussion for more details behind this decision: + * https://groups.google.com/d/topic/scala-internals/gp5JsM1E0Fo/discussion + */ + +lazy val commonSettings = Seq[Setting[_]]( + organization := "org.scala-lang", + version := "2.11.6-SNAPSHOT", + scalaVersion := "2.11.5", + // all project will have baseDirectory set to root folder; we shouldn't include + // any source from it in any project + sourcesInBase := false, + // we don't cross build Scala itself + crossPaths := false, + // do not add Scala library jar as a dependency automatically + autoScalaLibrary := false, + // we always assume that Java classes are standalone and do not have any dependency + // on Scala classes + compileOrder := CompileOrder.JavaThenScala, + // we don't want any unmanaged jars; as a reminder: unmanaged jar is a jar stored + // directly on the file system and it's not resolved through Ivy + // Ant's build stored unmanaged jars in `lib/` directory + unmanagedJars in Compile := Seq.empty, + // set baseDirectory to the root folder in all projects + baseDirectory := (baseDirectory in ThisBuild).value, + sourceDirectory in Compile := baseDirectory.value / "src" / name.value, + sourceDirectories in Compile := Seq(sourceDirectory.value), + scalaSource in Compile := (sourceDirectory in Compile).value, + javaSource in Compile := (sourceDirectory in Compile).value, + target := baseDirectory.value / "target" / name.value, + classDirectory in Compile := baseDirectory.value / "build/quick/classes" / name.value, + // given that classDirectory is overriden to be _outside_ of target directory, we have + // to make sure its being cleaned properly + cleanFiles += (classDirectory in Compile).value +) + +lazy val library = project. + settings(commonSettings: _*). + settings( + scalacOptions ++= Seq[String]("-sourcepath", (scalaSource in Compile).value.toString) + ) dependsOn (forkjoin) + +lazy val reflect = project. + settings(commonSettings: _*). + dependsOn(library) + +lazy val compiler = project. + settings(commonSettings: _*). + settings(libraryDependencies += "org.apache.ant" % "ant" % "1.9.4"). + dependsOn(library, reflect, asm) + +lazy val interactive = project. + settings(commonSettings: _*). + dependsOn(compiler) + +lazy val repl = project. + settings(commonSettings: _*). + // TODO: in Ant build def, this version is defined in versions.properties + // figure out whether we also want to externalize jline's version + settings(libraryDependencies += "jline" % "jline" % "2.12"). + dependsOn(compiler) + +lazy val scaladoc = project. + settings(commonSettings: _*). + settings( + libraryDependencies ++= Seq( + "org.scala-lang.modules" %% "scala-xml" % "1.0.3", + "org.scala-lang.modules" %% "scala-parser-combinators" % "1.0.3", + "org.scala-lang.modules" %% "scala-partest" % "1.0.5" + ) + ). + dependsOn(compiler) + +lazy val scalap = project. + settings(commonSettings: _*). + dependsOn(compiler) + +// deprecated Scala Actors project +// TODO: it packages into actors.jar but it should be scala-actors.jar +lazy val actors = project. + settings(commonSettings: _*). + dependsOn(library) + +lazy val forkjoin = project. + settings(commonSettings: _*) + +lazy val asm = project. + settings(commonSettings: _*) + +lazy val root = (project in file(".")). + aggregate(library, forkjoin, reflect, compiler, asm, interactive, repl, + scaladoc, scalap, actors). + // make the root project an aggragate-only + // we disable sbt's built-in Ivy plugin in the root project + // so it doesn't produce any artifact including not building + // an empty jar + disablePlugins(plugins.IvyPlugin) diff --git a/project/build.properties b/project/build.properties new file mode 100644 index 0000000000..748703f770 --- /dev/null +++ b/project/build.properties @@ -0,0 +1 @@ +sbt.version=0.13.7 -- cgit v1.2.3 From 70d041ccc8332fa44d1e8d91b27fab16ba741568 Mon Sep 17 00:00:00 2001 From: Grzegorz Kossakowski Date: Wed, 11 Feb 2015 21:27:40 +0100 Subject: Package jars into build/pack/lib directory. Override default location for jars to be build/pack/lib. This turned out to be a bit more involved than I expected. Check the diff for details. Strip down the -version suffix from jar names. For now we're trying to mimic Ant's build as closely as possible. Ant does not embed version numbers in jar names it creates in build/pack/lib directory. --- build.sbt | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/build.sbt b/build.sbt index 740d823435..aaca6bc35f 100644 --- a/build.sbt +++ b/build.sbt @@ -76,6 +76,17 @@ lazy val commonSettings = Seq[Setting[_]]( javaSource in Compile := (sourceDirectory in Compile).value, target := baseDirectory.value / "target" / name.value, classDirectory in Compile := baseDirectory.value / "build/quick/classes" / name.value, + artifactPath in packageBin in Compile := { + // two lines below are copied over from sbt's sources: + // https://github.com/sbt/sbt/blob/0.13/main/src/main/scala/sbt/Defaults.scala#L628 + //val resolvedScalaVersion = ScalaVersion((scalaVersion in artifactName).value, (scalaBinaryVersion in artifactName).value) + //val resolvedArtifactName = artifactName.value(resolvedScalaVersion, projectID.value, artifact.value) + // if you would like to get a jar with version number embedded in it (as normally sbt does) + // uncomment the other definition of the `resolvedArtifactName` + val resolvedArtifact = artifact.value + val resolvedArtifactName = s"${resolvedArtifact.name}.${resolvedArtifact.extension}" + baseDirectory.value / "build/pack/lib" / resolvedArtifactName + }, // given that classDirectory is overriden to be _outside_ of target directory, we have // to make sure its being cleaned properly cleanFiles += (classDirectory in Compile).value -- cgit v1.2.3 From 03bce72e72c701532ec344b658b94c4dfcfadcdd Mon Sep 17 00:00:00 2001 From: dumpstate Date: Mon, 16 Feb 2015 00:19:23 +0100 Subject: Generate .properties files, pack in jars. Add an sbt task that generates .properties files that are packaged into jars. The properties file contains Scala version in various formats. The task is registered as a resource generator so sbt can properly track generated files and make sure they are included in packaged jars. The implementation of generateVersionPropertiesFile contains a note on arbitrary order of written properties. We could consider using https://github.com/etiennestuder/java-ordered-properties to guarantee a stable order. Or simply ditch using Properties class and generate a String. Our property files are simple enough that nothing more fancy is needed. --- build.sbt | 56 +++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 55 insertions(+), 1 deletion(-) diff --git a/build.sbt b/build.sbt index aaca6bc35f..522a89189f 100644 --- a/build.sbt +++ b/build.sbt @@ -89,7 +89,10 @@ lazy val commonSettings = Seq[Setting[_]]( }, // given that classDirectory is overriden to be _outside_ of target directory, we have // to make sure its being cleaned properly - cleanFiles += (classDirectory in Compile).value + cleanFiles += (classDirectory in Compile).value, + copyrightString := "Copyright 2002-2013, LAMP/EPFL", + resourceGenerators in Compile += generateVersionPropertiesFile.map(file => Seq(file)).taskValue, + generateVersionPropertiesFile := generateVersionPropertiesFileImpl.value ) lazy val library = project. @@ -153,3 +156,54 @@ lazy val root = (project in file(".")). // so it doesn't produce any artifact including not building // an empty jar disablePlugins(plugins.IvyPlugin) + +lazy val copyrightString = SettingKey[String]("Copyright string.") + +lazy val generateVersionPropertiesFile = taskKey[File]("Generating version properties file.") + +lazy val generateVersionPropertiesFileImpl: Def.Initialize[Task[File]] = Def.task { + val propFile = (resourceManaged in Compile).value / s"${name.value}.properties" + val props = new java.util.Properties + + /** + * Regexp that splits version number split into two parts: version and suffix. + * Examples of how the split is performed: + * + * "2.11.5": ("2.11.5", null) + * "2.11.5-acda7a": ("2.11.5", "-acda7a") + * "2.11.5-SNAPSHOT": ("2.11.5", "-SNAPSHOT") + * + */ + val versionSplitted = """([\w+\.]+)(-[\w+\.]+)??""".r + + val versionSplitted(ver, suffixOrNull) = version.value + val osgiSuffix = suffixOrNull match { + case null => "-VFINAL" + case "-SNAPSHOT" => "" + case suffixStr => suffixStr + } + + def executeTool(tool: String) = { + val cmd = + if (System.getProperty("os.name").toLowerCase.contains("windows")) + s"cmd.exe /c tools\\$tool.bat -p" + else s"tools/$tool" + Process(cmd).lines.head + } + + val commitDate = executeTool("get-scala-commit-date") + val commitSha = executeTool("get-scala-commit-sha") + + props.put("version.number", s"${version.value}-$commitDate-$commitSha") + props.put("maven.version.number", s"${version.value}") + props.put("osgi.version.number", s"$ver.v$commitDate$osgiSuffix-$commitSha") + props.put("copyright.string", copyrightString.value) + + // unfortunately, this will write properties in arbitrary order + // this makes it harder to test for stability of generated artifacts + // consider using https://github.com/etiennestuder/java-ordered-properties + // instead of java.util.Properties + IO.write(props, null, propFile) + + propFile +} -- cgit v1.2.3 From 22ab71c826a1a7fae332ffc68a73d2946990c0c1 Mon Sep 17 00:00:00 2001 From: Grzegorz Kossakowski Date: Thu, 19 Feb 2015 12:21:59 +0100 Subject: Change project's base to src/$name Previously, Project.base was set to ./$name and we tweaked manually the `baseDirectory` setting to point at `.`. This was confusing sbt and its plugins. We are setting Project.base to ./src/$name and leave `baseDirectory` intact (so it will point at `./src/$name`. Adapt settings that depend on `baseDirectory` accordingly. Refactor configuration as a subproject into a common method. Given that `baseDirectory` is pointing at `src/$name`, we don't have to worry about `sourcesInBase`. --- build.sbt | 59 ++++++++++++++++++++++++++++++----------------------------- 1 file changed, 30 insertions(+), 29 deletions(-) diff --git a/build.sbt b/build.sbt index 522a89189f..c0e8a4fb0d 100644 --- a/build.sbt +++ b/build.sbt @@ -54,9 +54,6 @@ lazy val commonSettings = Seq[Setting[_]]( organization := "org.scala-lang", version := "2.11.6-SNAPSHOT", scalaVersion := "2.11.5", - // all project will have baseDirectory set to root folder; we shouldn't include - // any source from it in any project - sourcesInBase := false, // we don't cross build Scala itself crossPaths := false, // do not add Scala library jar as a dependency automatically @@ -68,14 +65,12 @@ lazy val commonSettings = Seq[Setting[_]]( // directly on the file system and it's not resolved through Ivy // Ant's build stored unmanaged jars in `lib/` directory unmanagedJars in Compile := Seq.empty, - // set baseDirectory to the root folder in all projects - baseDirectory := (baseDirectory in ThisBuild).value, - sourceDirectory in Compile := baseDirectory.value / "src" / name.value, + sourceDirectory in Compile := baseDirectory.value, sourceDirectories in Compile := Seq(sourceDirectory.value), scalaSource in Compile := (sourceDirectory in Compile).value, javaSource in Compile := (sourceDirectory in Compile).value, - target := baseDirectory.value / "target" / name.value, - classDirectory in Compile := baseDirectory.value / "build/quick/classes" / name.value, + target := (baseDirectory in ThisBuild).value / "target" / name.value, + classDirectory in Compile := (baseDirectory in ThisBuild).value / "build/quick/classes" / name.value, artifactPath in packageBin in Compile := { // two lines below are copied over from sbt's sources: // https://github.com/sbt/sbt/blob/0.13/main/src/main/scala/sbt/Defaults.scala#L628 @@ -85,7 +80,7 @@ lazy val commonSettings = Seq[Setting[_]]( // uncomment the other definition of the `resolvedArtifactName` val resolvedArtifact = artifact.value val resolvedArtifactName = s"${resolvedArtifact.name}.${resolvedArtifact.extension}" - baseDirectory.value / "build/pack/lib" / resolvedArtifactName + (baseDirectory in ThisBuild).value / "build/pack/lib" / resolvedArtifactName }, // given that classDirectory is overriden to be _outside_ of target directory, we have // to make sure its being cleaned properly @@ -95,34 +90,28 @@ lazy val commonSettings = Seq[Setting[_]]( generateVersionPropertiesFile := generateVersionPropertiesFileImpl.value ) -lazy val library = project. - settings(commonSettings: _*). +lazy val library = configureAsSubproject(project). settings( scalacOptions ++= Seq[String]("-sourcepath", (scalaSource in Compile).value.toString) ) dependsOn (forkjoin) -lazy val reflect = project. - settings(commonSettings: _*). +lazy val reflect = configureAsSubproject(project). dependsOn(library) -lazy val compiler = project. - settings(commonSettings: _*). +lazy val compiler = configureAsSubproject(project). settings(libraryDependencies += "org.apache.ant" % "ant" % "1.9.4"). dependsOn(library, reflect, asm) -lazy val interactive = project. - settings(commonSettings: _*). +lazy val interactive = configureAsSubproject(project). dependsOn(compiler) -lazy val repl = project. - settings(commonSettings: _*). +lazy val repl = configureAsSubproject(project). // TODO: in Ant build def, this version is defined in versions.properties // figure out whether we also want to externalize jline's version settings(libraryDependencies += "jline" % "jline" % "2.12"). dependsOn(compiler) -lazy val scaladoc = project. - settings(commonSettings: _*). +lazy val scaladoc = configureAsSubproject(project). settings( libraryDependencies ++= Seq( "org.scala-lang.modules" %% "scala-xml" % "1.0.3", @@ -132,21 +121,17 @@ lazy val scaladoc = project. ). dependsOn(compiler) -lazy val scalap = project. - settings(commonSettings: _*). +lazy val scalap = configureAsSubproject(project). dependsOn(compiler) // deprecated Scala Actors project // TODO: it packages into actors.jar but it should be scala-actors.jar -lazy val actors = project. - settings(commonSettings: _*). +lazy val actors = configureAsSubproject(project). dependsOn(library) -lazy val forkjoin = project. - settings(commonSettings: _*) +lazy val forkjoin = configureAsSubproject(project) -lazy val asm = project. - settings(commonSettings: _*) +lazy val asm = configureAsSubproject(project) lazy val root = (project in file(".")). aggregate(library, forkjoin, reflect, compiler, asm, interactive, repl, @@ -157,6 +142,22 @@ lazy val root = (project in file(".")). // an empty jar disablePlugins(plugins.IvyPlugin) +/** + * Configures passed project as a subproject (e.g. compiler or repl) + * with common settings attached to it. + * + * Typical usage is: + * + * lazy val mySubproject = configureAsSubproject(project) + * + * We pass `project` as an argument which is in fact a macro call. This macro determines + * project.id based on the name of the lazy val on the left-hand side. + */ +def configureAsSubproject(project: Project): Project = { + val base = file(".") / "src" / project.id + (project in base).settings(commonSettings: _*) +} + lazy val copyrightString = SettingKey[String]("Copyright string.") lazy val generateVersionPropertiesFile = taskKey[File]("Generating version properties file.") -- cgit v1.2.3 From a5e64642bc706dd781708d218deaafbf34ed5da8 Mon Sep 17 00:00:00 2001 From: Grzegorz Kossakowski Date: Thu, 19 Feb 2015 15:05:37 +0100 Subject: Introduce `buildDirectory` setting. This settings determines location of where class files, jars, and other build products will go. By default, it's `./build-sbt`. It makes it easier to compare what Ant and sbt builds do. Also, changed `SettingKey` to `settingKey`. The latter is a macro that automatically determines `label` of a setting and takes a description as argument. Before that, `SettingKey("my desc")` would create a setting key with `label` set to "my desc". Not what we wanted. --- build.sbt | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/build.sbt b/build.sbt index c0e8a4fb0d..842bf41d23 100644 --- a/build.sbt +++ b/build.sbt @@ -70,7 +70,7 @@ lazy val commonSettings = Seq[Setting[_]]( scalaSource in Compile := (sourceDirectory in Compile).value, javaSource in Compile := (sourceDirectory in Compile).value, target := (baseDirectory in ThisBuild).value / "target" / name.value, - classDirectory in Compile := (baseDirectory in ThisBuild).value / "build/quick/classes" / name.value, + classDirectory in Compile := buildDirectory.value / "quick/classes" / name.value, artifactPath in packageBin in Compile := { // two lines below are copied over from sbt's sources: // https://github.com/sbt/sbt/blob/0.13/main/src/main/scala/sbt/Defaults.scala#L628 @@ -80,7 +80,7 @@ lazy val commonSettings = Seq[Setting[_]]( // uncomment the other definition of the `resolvedArtifactName` val resolvedArtifact = artifact.value val resolvedArtifactName = s"${resolvedArtifact.name}.${resolvedArtifact.extension}" - (baseDirectory in ThisBuild).value / "build/pack/lib" / resolvedArtifactName + buildDirectory.value / "pack/lib" / resolvedArtifactName }, // given that classDirectory is overriden to be _outside_ of target directory, we have // to make sure its being cleaned properly @@ -158,8 +158,8 @@ def configureAsSubproject(project: Project): Project = { (project in base).settings(commonSettings: _*) } -lazy val copyrightString = SettingKey[String]("Copyright string.") - +lazy val buildDirectory = settingKey[File]("The directory where all build products go. By default ./build") +lazy val copyrightString = settingKey[String]("Copyright string.") lazy val generateVersionPropertiesFile = taskKey[File]("Generating version properties file.") lazy val generateVersionPropertiesFileImpl: Def.Initialize[Task[File]] = Def.task { @@ -208,3 +208,5 @@ lazy val generateVersionPropertiesFileImpl: Def.Initialize[Task[File]] = Def.tas propFile } + +buildDirectory in ThisBuild := (baseDirectory in ThisBuild).value / "build-sbt" -- cgit v1.2.3 From 083cd8c4e3fe6c73755d9162b275d9683778f1a2 Mon Sep 17 00:00:00 2001 From: Grzegorz Kossakowski Date: Thu, 19 Feb 2015 16:16:33 +0100 Subject: Split `commonSettings` into smaller chunks. Move most of `commonSettings` to `subprojectSettings` and keep `commonSettings` truly minimal. --- build.sbt | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/build.sbt b/build.sbt index 842bf41d23..14f05bb550 100644 --- a/build.sbt +++ b/build.sbt @@ -64,7 +64,10 @@ lazy val commonSettings = Seq[Setting[_]]( // we don't want any unmanaged jars; as a reminder: unmanaged jar is a jar stored // directly on the file system and it's not resolved through Ivy // Ant's build stored unmanaged jars in `lib/` directory - unmanagedJars in Compile := Seq.empty, + unmanagedJars in Compile := Seq.empty +) + +lazy val subprojectSettings = commonSettings ++ Seq[Setting[_]]( sourceDirectory in Compile := baseDirectory.value, sourceDirectories in Compile := Seq(sourceDirectory.value), scalaSource in Compile := (sourceDirectory in Compile).value, @@ -155,7 +158,7 @@ lazy val root = (project in file(".")). */ def configureAsSubproject(project: Project): Project = { val base = file(".") / "src" / project.id - (project in base).settings(commonSettings: _*) + (project in base).settings(subprojectSettings: _*) } lazy val buildDirectory = settingKey[File]("The directory where all build products go. By default ./build") -- cgit v1.2.3 From ae55324fd07e0042eb14098a2ad9fad0279d24ec Mon Sep 17 00:00:00 2001 From: Grzegorz Kossakowski Date: Fri, 20 Feb 2015 13:14:18 +0100 Subject: Asm and fork/join: no docs, publishing,... As they are forks of external Java projects, don't document/publish them. TODO: move them out of the repo As in ant, pass -g, -source and -target options to javac to get byte-identical outputs for Java files between Ant and sbt builds. --- build.sbt | 51 +++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 41 insertions(+), 10 deletions(-) diff --git a/build.sbt b/build.sbt index 14f05bb550..6c601219ec 100644 --- a/build.sbt +++ b/build.sbt @@ -61,19 +61,23 @@ lazy val commonSettings = Seq[Setting[_]]( // we always assume that Java classes are standalone and do not have any dependency // on Scala classes compileOrder := CompileOrder.JavaThenScala, + javacOptions in Compile ++= Seq("-g", "-source", "1.5", "-target", "1.6"), // we don't want any unmanaged jars; as a reminder: unmanaged jar is a jar stored // directly on the file system and it's not resolved through Ivy // Ant's build stored unmanaged jars in `lib/` directory - unmanagedJars in Compile := Seq.empty -) - -lazy val subprojectSettings = commonSettings ++ Seq[Setting[_]]( + unmanagedJars in Compile := Seq.empty, sourceDirectory in Compile := baseDirectory.value, sourceDirectories in Compile := Seq(sourceDirectory.value), scalaSource in Compile := (sourceDirectory in Compile).value, javaSource in Compile := (sourceDirectory in Compile).value, target := (baseDirectory in ThisBuild).value / "target" / name.value, classDirectory in Compile := buildDirectory.value / "quick/classes" / name.value, + // given that classDirectory is overriden to be _outside_ of target directory, we have + // to make sure its being cleaned properly + cleanFiles += (classDirectory in Compile).value +) + +lazy val scalaSubprojectSettings = commonSettings ++ Seq[Setting[_]]( artifactPath in packageBin in Compile := { // two lines below are copied over from sbt's sources: // https://github.com/sbt/sbt/blob/0.13/main/src/main/scala/sbt/Defaults.scala#L628 @@ -85,9 +89,6 @@ lazy val subprojectSettings = commonSettings ++ Seq[Setting[_]]( val resolvedArtifactName = s"${resolvedArtifact.name}.${resolvedArtifact.extension}" buildDirectory.value / "pack/lib" / resolvedArtifactName }, - // given that classDirectory is overriden to be _outside_ of target directory, we have - // to make sure its being cleaned properly - cleanFiles += (classDirectory in Compile).value, copyrightString := "Copyright 2002-2013, LAMP/EPFL", resourceGenerators in Compile += generateVersionPropertiesFile.map(file => Seq(file)).taskValue, generateVersionPropertiesFile := generateVersionPropertiesFileImpl.value @@ -132,9 +133,9 @@ lazy val scalap = configureAsSubproject(project). lazy val actors = configureAsSubproject(project). dependsOn(library) -lazy val forkjoin = configureAsSubproject(project) +lazy val forkjoin = configureAsForkOfJavaProject(project) -lazy val asm = configureAsSubproject(project) +lazy val asm = configureAsForkOfJavaProject(project) lazy val root = (project in file(".")). aggregate(library, forkjoin, reflect, compiler, asm, interactive, repl, @@ -158,7 +159,37 @@ lazy val root = (project in file(".")). */ def configureAsSubproject(project: Project): Project = { val base = file(".") / "src" / project.id - (project in base).settings(subprojectSettings: _*) + (project in base).settings(scalaSubprojectSettings: _*) +} + +/** + * Configuration for subprojects that are forks of some Java projects + * we depend on. At the moment there are just two: asm and forkjoin. + * + * We do not publish artifacts for those projects but we package their + * binaries in a jar of other project (compiler or library). + * + * For that reason we disable docs generation, packaging and publishing. + */ +def configureAsForkOfJavaProject(project: Project): Project = { + val base = file(".") / "src" / project.id + // disable various tasks that do not make sense for forks of Java projects + // we disable those task by overriding them and returning bogus files when + // needed. This is a bit sketchy but I haven't found any better way. + val disableTasks = Seq[Setting[_]]( + (doc := file("!!! NO DOCS !!!")), + (publishLocal := {}), + (publish := {}), + (packageBin in Compile := file("!!! NO PACKAGING !!!")) + ) + (project in base). + settings(commonSettings: _*). + settings(disableTasks: _*). + settings( + sourceDirectory in Compile := baseDirectory.value, + javaSource in Compile := (sourceDirectory in Compile).value, + sources in Compile in doc := Seq.empty + ) } lazy val buildDirectory = settingKey[File]("The directory where all build products go. By default ./build") -- cgit v1.2.3 From 79a760dbb19f65122b567da0c030a1d57a8b8a3f Mon Sep 17 00:00:00 2001 From: Grzegorz Kossakowski Date: Fri, 20 Feb 2015 19:04:07 +0100 Subject: Scaladoc Put library class files to scaladoc's classpath when documenting the library. TODO: why is this needed? Bug in scaladoc? Pass `src/library-aux` to scaladoc using `-doc-no-compile` option. To mimic ant, output docs into `buildDirectory/scaladoc/$name`. --- build.sbt | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/build.sbt b/build.sbt index 6c601219ec..19ada760f5 100644 --- a/build.sbt +++ b/build.sbt @@ -71,6 +71,7 @@ lazy val commonSettings = Seq[Setting[_]]( scalaSource in Compile := (sourceDirectory in Compile).value, javaSource in Compile := (sourceDirectory in Compile).value, target := (baseDirectory in ThisBuild).value / "target" / name.value, + target in Compile in doc := buildDirectory.value / "scaladoc" / name.value, classDirectory in Compile := buildDirectory.value / "quick/classes" / name.value, // given that classDirectory is overriden to be _outside_ of target directory, we have // to make sure its being cleaned properly @@ -96,7 +97,18 @@ lazy val scalaSubprojectSettings = commonSettings ++ Seq[Setting[_]]( lazy val library = configureAsSubproject(project). settings( - scalacOptions ++= Seq[String]("-sourcepath", (scalaSource in Compile).value.toString) + scalacOptions in Compile ++= Seq[String]("-sourcepath", (scalaSource in Compile).value.toString), + // Workaround for a bug in `scaladoc` that it seems to not respect the `-sourcepath` option + // as a result of this bug, the compiler cannot even initialize Definitions without + // binaries of the library on the classpath. Specifically, we get this error: + // (library/compile:doc) scala.reflect.internal.FatalError: package class scala does not have a member Int + // Ant build does the same thing always: it puts binaries for documented classes on the classpath + // sbt never does this by default (which seems like a good default) + dependencyClasspath in Compile in doc += (classDirectory in Compile).value, + scalacOptions in Compile in doc ++= { + val libraryAuxDir = (baseDirectory in ThisBuild).value / "src/library-aux" + Seq("-doc-no-compile", libraryAuxDir.toString) + } ) dependsOn (forkjoin) lazy val reflect = configureAsSubproject(project). -- cgit v1.2.3 From 6eae83a7cf62306cba1bffb0d14f2f43d377d59c Mon Sep 17 00:00:00 2001 From: Grzegorz Kossakowski Date: Tue, 10 Mar 2015 10:20:35 -0700 Subject: Compiler jar includes interactive, scaladoc & repl Mirror ant by disabling docs and publishing tasks in these subprojects. Class files are included by merging `mappings`: the `package` task uses it to determine which class files should end up in a jar file. --- build.sbt | 36 +++++++++++++++++++++++++----------- 1 file changed, 25 insertions(+), 11 deletions(-) diff --git a/build.sbt b/build.sbt index 19ada760f5..adf7eb9886 100644 --- a/build.sbt +++ b/build.sbt @@ -78,6 +78,17 @@ lazy val commonSettings = Seq[Setting[_]]( cleanFiles += (classDirectory in Compile).value ) +// disable various tasks that are not needed for projects that are used +// only for compiling code and not publishing it as a standalone artifact +// we disable those tasks by overriding them and returning bogus files when +// needed. This is a bit sketchy but I haven't found any better way. +val disableDocsAndPublishingTasks = Seq[Setting[_]]( + (doc := file("!!! NO DOCS !!!")), + (publishLocal := {}), + (publish := {}), + (packageBin in Compile := file("!!! NO PACKAGING !!!")) +) + lazy val scalaSubprojectSettings = commonSettings ++ Seq[Setting[_]]( artifactPath in packageBin in Compile := { // two lines below are copied over from sbt's sources: @@ -115,16 +126,27 @@ lazy val reflect = configureAsSubproject(project). dependsOn(library) lazy val compiler = configureAsSubproject(project). - settings(libraryDependencies += "org.apache.ant" % "ant" % "1.9.4"). + settings(libraryDependencies += "org.apache.ant" % "ant" % "1.9.4", + // this a way to make sure that classes from interactive and scaladoc projects + // end up in compiler jar (that's what Ant build does) + // we need to use LocalProject references (with strings) to deal with mutual recursion + mappings in Compile in packageBin := + (mappings in Compile in packageBin).value ++ + (mappings in Compile in packageBin in LocalProject("interactive")).value ++ + (mappings in Compile in packageBin in LocalProject("scaladoc")).value ++ + (mappings in Compile in packageBin in LocalProject("repl")).value + ). dependsOn(library, reflect, asm) lazy val interactive = configureAsSubproject(project). + settings(disableDocsAndPublishingTasks: _*). dependsOn(compiler) lazy val repl = configureAsSubproject(project). // TODO: in Ant build def, this version is defined in versions.properties // figure out whether we also want to externalize jline's version settings(libraryDependencies += "jline" % "jline" % "2.12"). + settings(disableDocsAndPublishingTasks: _*). dependsOn(compiler) lazy val scaladoc = configureAsSubproject(project). @@ -135,6 +157,7 @@ lazy val scaladoc = configureAsSubproject(project). "org.scala-lang.modules" %% "scala-partest" % "1.0.5" ) ). + settings(disableDocsAndPublishingTasks: _*). dependsOn(compiler) lazy val scalap = configureAsSubproject(project). @@ -185,18 +208,9 @@ def configureAsSubproject(project: Project): Project = { */ def configureAsForkOfJavaProject(project: Project): Project = { val base = file(".") / "src" / project.id - // disable various tasks that do not make sense for forks of Java projects - // we disable those task by overriding them and returning bogus files when - // needed. This is a bit sketchy but I haven't found any better way. - val disableTasks = Seq[Setting[_]]( - (doc := file("!!! NO DOCS !!!")), - (publishLocal := {}), - (publish := {}), - (packageBin in Compile := file("!!! NO PACKAGING !!!")) - ) (project in base). settings(commonSettings: _*). - settings(disableTasks: _*). + settings(disableDocsAndPublishingTasks: _*). settings( sourceDirectory in Compile := baseDirectory.value, javaSource in Compile := (sourceDirectory in Compile).value, -- cgit v1.2.3 From ed93fac8b23b7b999ad88327beed8b2b2514d457 Mon Sep 17 00:00:00 2001 From: Grzegorz Kossakowski Date: Mon, 9 Mar 2015 15:11:37 -0700 Subject: Avoid circular deps: override `scalaInstance` As suggested in sbt/sbt#1872, specify ScalaInstance manually and use sbt's launcher as a context for Ivy resolution. This way we avoid the circular dependency problem described in the issue. --- build.sbt | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/build.sbt b/build.sbt index adf7eb9886..29922e70d2 100644 --- a/build.sbt +++ b/build.sbt @@ -58,6 +58,12 @@ lazy val commonSettings = Seq[Setting[_]]( crossPaths := false, // do not add Scala library jar as a dependency automatically autoScalaLibrary := false, + // we also do not add scala instance automatically because it introduces + // a circular instance, see: https://github.com/sbt/sbt/issues/1872 + managedScalaInstance := false, + // this is a way to workaround issue described in https://github.com/sbt/sbt/issues/1872 + // check it out for more details + scalaInstance := ScalaInstance(scalaVersion.value, appConfiguration.value.provider.scalaProvider.launcher getScala scalaVersion.value), // we always assume that Java classes are standalone and do not have any dependency // on Scala classes compileOrder := CompileOrder.JavaThenScala, -- cgit v1.2.3 From 54b7daccec0ce72b6077ebad137810cb4d864ae0 Mon Sep 17 00:00:00 2001 From: Jason Zaugg Date: Tue, 10 Mar 2015 11:35:51 -0700 Subject: Exclude transitive scala-library dependency Scaladoc depends on scala-xml, etc, built with the previous stable version of Scala. This isn't ideal, but we're stuck with this for now. We were relying on Ivy to evict the transitive dependency on the old version of scala-library. However, SBT treats evictions of scala-library as suspicious and emits a custom warning. This commit explicitly excludes the transitive dependency so as to avoid these warnings. It also makes the output of `show update` a little easier to read. --- build.sbt | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/build.sbt b/build.sbt index 29922e70d2..d86e4ad779 100644 --- a/build.sbt +++ b/build.sbt @@ -11,13 +11,13 @@ * layout. For that reason we have to configure a lot more explicitly. I've tried explain in * comments the less obvious settings. * - * This nicely leads me to explaning goal and non-goals of this build definition. Goals are: + * This nicely leads me to explaining goal and non-goals of this build definition. Goals are: * * - to be easy to tweak it in case a bug or small inconsistency is found * - to mimic Ant's behavior as closely as possible * - to be super explicit about any departure from standard sbt settings * - to achieve functional parity with Ant build as quickly as possible - * - to be readable and not necessarily succint + * - to be readable and not necessarily succinct * - to provide the nicest development experience for people hacking on Scala * * Non-goals are: @@ -29,7 +29,7 @@ * * It boils down to simple rules: * - * - project laytout is set in stone for now + * - project layout is set in stone for now * - if you need to work on convincing sbt to follow non-standard layout then * explain everything you did in comments * - constantly check where Ant build produces class files, artifacts, what kind of other @@ -50,10 +50,12 @@ * https://groups.google.com/d/topic/scala-internals/gp5JsM1E0Fo/discussion */ +val bootstrapScalaVersion = "2.11.5" + lazy val commonSettings = Seq[Setting[_]]( organization := "org.scala-lang", version := "2.11.6-SNAPSHOT", - scalaVersion := "2.11.5", + scalaVersion := bootstrapScalaVersion, // we don't cross build Scala itself crossPaths := false, // do not add Scala library jar as a dependency automatically @@ -155,13 +157,13 @@ lazy val repl = configureAsSubproject(project). settings(disableDocsAndPublishingTasks: _*). dependsOn(compiler) +def moduleDependency(name: String) = + // exclusion of the scala-library transitive dependency avoids eviction warnings during `update`. + "org.scala-lang.modules" %% name % "1.0.3" exclude("org.scala-lang", "scala-library") + lazy val scaladoc = configureAsSubproject(project). settings( - libraryDependencies ++= Seq( - "org.scala-lang.modules" %% "scala-xml" % "1.0.3", - "org.scala-lang.modules" %% "scala-parser-combinators" % "1.0.3", - "org.scala-lang.modules" %% "scala-partest" % "1.0.5" - ) + libraryDependencies ++= Seq("scala-xml", "scala-parser-combinators", "scala-partest").map(moduleDependency) ). settings(disableDocsAndPublishingTasks: _*). dependsOn(compiler) @@ -180,12 +182,10 @@ lazy val asm = configureAsForkOfJavaProject(project) lazy val root = (project in file(".")). aggregate(library, forkjoin, reflect, compiler, asm, interactive, repl, - scaladoc, scalap, actors). - // make the root project an aggragate-only - // we disable sbt's built-in Ivy plugin in the root project - // so it doesn't produce any artifact including not building - // an empty jar - disablePlugins(plugins.IvyPlugin) + scaladoc, scalap, actors).settings( + scalaVersion := bootstrapScalaVersion, + ivyScala := ivyScala.value map { _.copy(overrideScalaVersion = true) } + ) /** * Configures passed project as a subproject (e.g. compiler or repl) -- cgit v1.2.3 From 4bb0406950dee57616edcf5c0e4b866be0d18906 Mon Sep 17 00:00:00 2001 From: Grzegorz Kossakowski Date: Tue, 10 Mar 2015 12:41:54 -0700 Subject: Root project shouldn't compile any sources. Let's override `sources` task to return empty collections of sources to compile. Otherwise, sbt by default would pick up sources in the root directory which is not what we want. --- build.sbt | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/build.sbt b/build.sbt index d86e4ad779..d4cad4d973 100644 --- a/build.sbt +++ b/build.sbt @@ -184,7 +184,8 @@ lazy val root = (project in file(".")). aggregate(library, forkjoin, reflect, compiler, asm, interactive, repl, scaladoc, scalap, actors).settings( scalaVersion := bootstrapScalaVersion, - ivyScala := ivyScala.value map { _.copy(overrideScalaVersion = true) } + ivyScala := ivyScala.value map { _.copy(overrideScalaVersion = true) }, + sources in Compile := Seq.empty ) /** -- cgit v1.2.3 From b4aa93c0db41b1ab429e35792f94b05d0db48a98 Mon Sep 17 00:00:00 2001 From: Grzegorz Kossakowski Date: Thu, 16 Apr 2015 15:13:05 +0200 Subject: Document sbt build as experimental. Add welcome message that is shown at sbt's startup. The message clearly indicates experimental status of sbt build. Also, add a short paragraph to README.md explaining status of sbt build. --- README.md | 9 +++++++++ build.sbt | 6 +++++- 2 files changed, 14 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 830dfa8d6c..1651333188 100644 --- a/README.md +++ b/README.md @@ -138,6 +138,15 @@ The Scala build system is based on Apache Ant. Most required pre-compiled libraries are part of the repository (in 'lib/'). The following however is assumed to be installed on the build machine: +## Building with Sbt (EXPERIMENTAL) + +The experimental sbt-based build definition has arrived! Run `sbt package` +to build the compiler. You can run `sbt test` to run unit (JUnit) tests. +Use `sbt test/it:test` to run integration (partest) tests. + +We would like to migrate to sbt build as quickly as possible. If you would +like to help please contact scala-internals@ mailing list to discuss your +ideas and coordinate your effort with others. ### Tips and tricks diff --git a/build.sbt b/build.sbt index d4cad4d973..51004a8f72 100644 --- a/build.sbt +++ b/build.sbt @@ -185,7 +185,11 @@ lazy val root = (project in file(".")). scaladoc, scalap, actors).settings( scalaVersion := bootstrapScalaVersion, ivyScala := ivyScala.value map { _.copy(overrideScalaVersion = true) }, - sources in Compile := Seq.empty + sources in Compile := Seq.empty, + onLoadMessage := """|*** Welcome to the sbt build definition for Scala! *** + |This build definition has an EXPERIMENTAL status. If you are not + |interested in testing or working on the build itself, please use + |the Ant build definition for now. Check README.md for more information.""".stripMargin ) /** -- cgit v1.2.3 From 42dace101f1239e3dfd6f522de4e97b807dfc0aa Mon Sep 17 00:00:00 2001 From: Grzegorz Kossakowski Date: Sun, 22 Feb 2015 18:06:10 +0100 Subject: Add "scala-" prefix to the name of a subproject. Scala artifacts (like library or compiler) are published are published with "scala-" prefix in their name. We keep project ids short so we can refer to projects simply by e.g. `library` and not `scala-library`. Given that `name` settings will have a different value, we switch to `Project.id` when constructing paths for target directories. We had to tweak properties generation because over there we use project name as a file name without "scala-" prefix. Since our projects are not named uniformly (e.g., "repl" and not "scala-repl"), add a setting for each subproject that requires special handling. --- build.sbt | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/build.sbt b/build.sbt index 51004a8f72..6d35345fce 100644 --- a/build.sbt +++ b/build.sbt @@ -78,9 +78,9 @@ lazy val commonSettings = Seq[Setting[_]]( sourceDirectories in Compile := Seq(sourceDirectory.value), scalaSource in Compile := (sourceDirectory in Compile).value, javaSource in Compile := (sourceDirectory in Compile).value, - target := (baseDirectory in ThisBuild).value / "target" / name.value, - target in Compile in doc := buildDirectory.value / "scaladoc" / name.value, - classDirectory in Compile := buildDirectory.value / "quick/classes" / name.value, + target := (baseDirectory in ThisBuild).value / "target" / thisProject.value.id, + target in Compile in doc := buildDirectory.value / "scaladoc" / thisProject.value.id, + classDirectory in Compile := buildDirectory.value / "quick/classes" / thisProject.value.id, // given that classDirectory is overriden to be _outside_ of target directory, we have // to make sure its being cleaned properly cleanFiles += (classDirectory in Compile).value @@ -116,6 +116,7 @@ lazy val scalaSubprojectSettings = commonSettings ++ Seq[Setting[_]]( lazy val library = configureAsSubproject(project). settings( + name := "scala-library", scalacOptions in Compile ++= Seq[String]("-sourcepath", (scalaSource in Compile).value.toString), // Workaround for a bug in `scaladoc` that it seems to not respect the `-sourcepath` option // as a result of this bug, the compiler cannot even initialize Definitions without @@ -131,10 +132,13 @@ lazy val library = configureAsSubproject(project). ) dependsOn (forkjoin) lazy val reflect = configureAsSubproject(project). + settings(name := "scala-reflect"). dependsOn(library) lazy val compiler = configureAsSubproject(project). - settings(libraryDependencies += "org.apache.ant" % "ant" % "1.9.4", + settings( + name := "scala-compiler", + libraryDependencies += "org.apache.ant" % "ant" % "1.9.4", // this a way to make sure that classes from interactive and scaladoc projects // end up in compiler jar (that's what Ant build does) // we need to use LocalProject references (with strings) to deal with mutual recursion @@ -174,6 +178,7 @@ lazy val scalap = configureAsSubproject(project). // deprecated Scala Actors project // TODO: it packages into actors.jar but it should be scala-actors.jar lazy val actors = configureAsSubproject(project). + settings(name := "scala-actors"). dependsOn(library) lazy val forkjoin = configureAsForkOfJavaProject(project) @@ -225,7 +230,8 @@ def configureAsForkOfJavaProject(project: Project): Project = { settings( sourceDirectory in Compile := baseDirectory.value, javaSource in Compile := (sourceDirectory in Compile).value, - sources in Compile in doc := Seq.empty + sources in Compile in doc := Seq.empty, + classDirectory in Compile := buildDirectory.value / "libs/classes" / thisProject.value.id ) } @@ -234,7 +240,7 @@ lazy val copyrightString = settingKey[String]("Copyright string.") lazy val generateVersionPropertiesFile = taskKey[File]("Generating version properties file.") lazy val generateVersionPropertiesFileImpl: Def.Initialize[Task[File]] = Def.task { - val propFile = (resourceManaged in Compile).value / s"${name.value}.properties" + val propFile = (resourceManaged in Compile).value / s"${thisProject.value.id}.properties" val props = new java.util.Properties /** -- cgit v1.2.3 From e29646b0e7d3c25a73748d7d64223ebd5188db4a Mon Sep 17 00:00:00 2001 From: Grzegorz Kossakowski Date: Tue, 10 Mar 2015 11:42:46 -0700 Subject: Included required resources in jars Static resources are stored along source files in our current layout. Therefore we set resourceDirectory to be sourceDirectory but make sure that includeFilter includes nothing by default. Subprojects like library or compiler set their own include filters. The include filters have been ported from the Ant build definition. Generate properties for: library, reflect, compiler & actors. --- build.sbt | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-) diff --git a/build.sbt b/build.sbt index 6d35345fce..d883aac057 100644 --- a/build.sbt +++ b/build.sbt @@ -78,6 +78,10 @@ lazy val commonSettings = Seq[Setting[_]]( sourceDirectories in Compile := Seq(sourceDirectory.value), scalaSource in Compile := (sourceDirectory in Compile).value, javaSource in Compile := (sourceDirectory in Compile).value, + // resources are stored along source files in our current layout + resourceDirectory in Compile := (sourceDirectory in Compile).value, + // each subproject has to ask specifically for files they want to include + includeFilter in unmanagedResources in Compile := NothingFilter, target := (baseDirectory in ThisBuild).value / "target" / thisProject.value.id, target in Compile in doc := buildDirectory.value / "scaladoc" / thisProject.value.id, classDirectory in Compile := buildDirectory.value / "quick/classes" / thisProject.value.id, @@ -108,13 +112,18 @@ lazy val scalaSubprojectSettings = commonSettings ++ Seq[Setting[_]]( val resolvedArtifact = artifact.value val resolvedArtifactName = s"${resolvedArtifact.name}.${resolvedArtifact.extension}" buildDirectory.value / "pack/lib" / resolvedArtifactName - }, + } +) + +lazy val generatePropertiesFileSettings = Seq[Setting[_]]( copyrightString := "Copyright 2002-2013, LAMP/EPFL", resourceGenerators in Compile += generateVersionPropertiesFile.map(file => Seq(file)).taskValue, generateVersionPropertiesFile := generateVersionPropertiesFileImpl.value ) +val libIncludes: FileFilter = "*.tmpl" | "*.xml" | "*.js" | "*.css" | "rootdoc.txt" lazy val library = configureAsSubproject(project). + settings(generatePropertiesFileSettings: _*). settings( name := "scala-library", scalacOptions in Compile ++= Seq[String]("-sourcepath", (scalaSource in Compile).value.toString), @@ -128,14 +137,20 @@ lazy val library = configureAsSubproject(project). scalacOptions in Compile in doc ++= { val libraryAuxDir = (baseDirectory in ThisBuild).value / "src/library-aux" Seq("-doc-no-compile", libraryAuxDir.toString) - } + }, + includeFilter in unmanagedResources in Compile := libIncludes ) dependsOn (forkjoin) lazy val reflect = configureAsSubproject(project). + settings(generatePropertiesFileSettings: _*). settings(name := "scala-reflect"). dependsOn(library) +val compilerIncludes: FileFilter = + "*.tmpl" | "*.xml" | "*.js" | "*.css" | "*.html" | "*.properties" | "*.swf" | + "*.png" | "*.gif" | "*.gif" | "*.txt" lazy val compiler = configureAsSubproject(project). + settings(generatePropertiesFileSettings: _*). settings( name := "scala-compiler", libraryDependencies += "org.apache.ant" % "ant" % "1.9.4", @@ -146,7 +161,8 @@ lazy val compiler = configureAsSubproject(project). (mappings in Compile in packageBin).value ++ (mappings in Compile in packageBin in LocalProject("interactive")).value ++ (mappings in Compile in packageBin in LocalProject("scaladoc")).value ++ - (mappings in Compile in packageBin in LocalProject("repl")).value + (mappings in Compile in packageBin in LocalProject("repl")).value, + includeFilter in unmanagedResources in Compile := compilerIncludes ). dependsOn(library, reflect, asm) @@ -178,6 +194,7 @@ lazy val scalap = configureAsSubproject(project). // deprecated Scala Actors project // TODO: it packages into actors.jar but it should be scala-actors.jar lazy val actors = configureAsSubproject(project). + settings(generatePropertiesFileSettings: _*). settings(name := "scala-actors"). dependsOn(library) -- cgit v1.2.3 From 9ce6a9d07c84c7436a50549603e7a8908841e22d Mon Sep 17 00:00:00 2001 From: Grzegorz Kossakowski Date: Tue, 10 Mar 2015 15:20:20 -0700 Subject: Generate shell scripts. Shell scripts are generated with `mkBin` sbt task that calls ScalaTool. ScalaTool is defined in project/ and is a port of Ant task that lives in scala.tools.ant.ScalaTool. While porting, we've simplified the code significantly. The `mkBin` task is defined in a new subproject: dist. This subproject will become responsible for building the distribution in the future. --- build.sbt | 40 ++++++++++++++++++++++++++++++++++++++++ project/ScalaTool.scala | 44 ++++++++++++++++++++++++++++++++++++++++++++ project/plugins.sbt | 1 + 3 files changed, 85 insertions(+) create mode 100644 project/ScalaTool.scala create mode 100644 project/plugins.sbt diff --git a/build.sbt b/build.sbt index d883aac057..e74d8c2f1d 100644 --- a/build.sbt +++ b/build.sbt @@ -214,6 +214,10 @@ lazy val root = (project in file(".")). |the Ant build definition for now. Check README.md for more information.""".stripMargin ) +lazy val dist = (project in file("dist")).settings( + mkBin := mkBinImpl.value +) + /** * Configures passed project as a subproject (e.g. compiler or repl) * with common settings attached to it. @@ -255,6 +259,7 @@ def configureAsForkOfJavaProject(project: Project): Project = { lazy val buildDirectory = settingKey[File]("The directory where all build products go. By default ./build") lazy val copyrightString = settingKey[String]("Copyright string.") lazy val generateVersionPropertiesFile = taskKey[File]("Generating version properties file.") +lazy val mkBin = taskKey[Seq[File]]("Generate shell script (bash or Windows batch).") lazy val generateVersionPropertiesFileImpl: Def.Initialize[Task[File]] = Def.task { val propFile = (resourceManaged in Compile).value / s"${thisProject.value.id}.properties" @@ -303,4 +308,39 @@ lazy val generateVersionPropertiesFileImpl: Def.Initialize[Task[File]] = Def.tas propFile } +lazy val mkBinImpl: Def.Initialize[Task[Seq[File]]] = Def.task { + def mkScalaTool(mainCls: String, classpath: Seq[Attributed[File]]): ScalaTool = + ScalaTool(mainClass = mainCls, + classpath = classpath.toList.map(_.data.getAbsolutePath), + properties = Map.empty, + javaOpts = "-Xmx256M -Xms32M", + toolFlags = "") + val rootDir = (classDirectory in Compile in compiler).value + def writeScripts(scalaTool: ScalaTool, file: String, outDir: File): Seq[File] = + Seq( + scalaTool.writeScript(file, "unix", rootDir, outDir), + scalaTool.writeScript(file, "windows", rootDir, outDir) + ) + def mkQuickBin(file: String, mainCls: String, classpath: Seq[Attributed[File]]): Seq[File] = { + val scalaTool = mkScalaTool(mainCls, classpath) + val outDir = buildDirectory.value / "quick/bin" + writeScripts(scalaTool, file, outDir) + } + + def mkPackBin(file: String, mainCls: String): Seq[File] = { + val scalaTool = mkScalaTool(mainCls, classpath = Nil) + val outDir = buildDirectory.value / "pack/bin" + writeScripts(scalaTool, file, outDir) + } + + def mkBin(file: String, mainCls: String, classpath: Seq[Attributed[File]]): Seq[File] = + mkQuickBin(file, mainCls, classpath) ++ mkPackBin(file, mainCls) + + mkBin("scala" , "scala.tools.nsc.MainGenericRunner", (fullClasspath in Compile in repl).value) ++ + mkBin("scalac" , "scala.tools.nsc.Main", (fullClasspath in Compile in compiler).value) ++ + mkBin("fsc" , "scala.tools.nsc.CompileClient", (fullClasspath in Compile in compiler).value) ++ + mkBin("scaladoc" , "scala.tools.nsc.ScalaDoc", (fullClasspath in Compile in scaladoc).value) ++ + mkBin("scalap" , "scala.tools.scalap.Main", (fullClasspath in Compile in scalap).value) +} + buildDirectory in ThisBuild := (baseDirectory in ThisBuild).value / "build-sbt" diff --git a/project/ScalaTool.scala b/project/ScalaTool.scala new file mode 100644 index 0000000000..559b215c18 --- /dev/null +++ b/project/ScalaTool.scala @@ -0,0 +1,44 @@ +import sbt._ +import org.apache.commons.lang3.StringUtils.replaceEach + +/** + * A class that generates a shell or batch script to execute a Scala program. + * + * This is a simplified copy of Ant task (see scala.tools.ant.ScalaTool). + */ +case class ScalaTool(mainClass: String, + classpath: List[String], + properties: Map[String, String], + javaOpts: String, + toolFlags: String) { + // For classpath, the platform specific + // demarcation of any script variables (e.g. `${SCALA_HOME}` or + // `%SCALA_HOME%`) can be specified in a platform independent way (e.g. + // `@SCALA_HOME@`) and automatically translated for you. + def patchedToolScript(template: String, platform: String) = { + val varRegex = """@(\w+)@""" // the group should be able to capture each of the keys of the map below + + val variables = Map( + ("@@" -> "@"), // for backwards compatibility + ("@class@" -> mainClass), + ("@properties@" -> (properties map { case (k, v) => s"""-D$k="$v""""} mkString " ")), + ("@javaflags@" -> javaOpts), + ("@toolflags@" -> toolFlags), + ("@classpath@" -> (platform match { + case "unix" => classpath.mkString(":").replace('\\', '/').replaceAll(varRegex, """\${$1}""") + case "windows" => classpath.mkString(";").replace('/', '\\').replaceAll(varRegex, "%$1%") + })) + ) + + val (from, to) = variables.unzip + replaceEach(template, from.toArray, to.toArray) + } + + def writeScript(file: String, platform: String, rootDir: File, outDir: File): File = { + val templatePath = s"scala/tools/ant/templates/tool-$platform.tmpl" + val suffix = platform match { case "windows" => ".bat" case _ => "" } + val scriptFile = outDir / s"$file$suffix" + IO.write(scriptFile, patchedToolScript(IO.read(rootDir / templatePath), platform)) + scriptFile + } +} diff --git a/project/plugins.sbt b/project/plugins.sbt new file mode 100644 index 0000000000..dc266a8db1 --- /dev/null +++ b/project/plugins.sbt @@ -0,0 +1 @@ +libraryDependencies += "org.apache.commons" % "commons-lang3" % "3.3.2" \ No newline at end of file -- cgit v1.2.3 From ac75b7471efa4099351f19b5a9b52be88fdd394b Mon Sep 17 00:00:00 2001 From: Grzegorz Kossakowski Date: Tue, 10 Mar 2015 16:27:31 -0700 Subject: Always fork in run. This way we don't need to worry about mixing classes for Scala compiler that sbt is invoking and the one we just built. For an example of possible problem, check out: https://github.com/gkossakowski/scala/pull/2#issuecomment-78169188 --- build.sbt | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/build.sbt b/build.sbt index e74d8c2f1d..dcc4e40d6b 100644 --- a/build.sbt +++ b/build.sbt @@ -87,7 +87,8 @@ lazy val commonSettings = Seq[Setting[_]]( classDirectory in Compile := buildDirectory.value / "quick/classes" / thisProject.value.id, // given that classDirectory is overriden to be _outside_ of target directory, we have // to make sure its being cleaned properly - cleanFiles += (classDirectory in Compile).value + cleanFiles += (classDirectory in Compile).value, + fork in run := true ) // disable various tasks that are not needed for projects that are used -- cgit v1.2.3 From f4fb8f17ada10d5a10b13312df1d7ae801bcea94 Mon Sep 17 00:00:00 2001 From: Grzegorz Kossakowski Date: Fri, 13 Mar 2015 17:59:43 -0700 Subject: Rework library dependencies Common library dependencies are extracted into vals. Those dependencies define "scala-library" as an excluded transitive dependency so we don't get eviction warnings during `update`. Lastly, version numbers for dependencies are obtained from `versions.properties` file. --- build.sbt | 33 ++++++++++++++++++++++++--------- 1 file changed, 24 insertions(+), 9 deletions(-) diff --git a/build.sbt b/build.sbt index dcc4e40d6b..322d8efbaf 100644 --- a/build.sbt +++ b/build.sbt @@ -52,6 +52,13 @@ val bootstrapScalaVersion = "2.11.5" +// exclusion of the scala-library transitive dependency avoids eviction warnings during `update`. +val scalaParserCombinatorsDep = "org.scala-lang.modules" %% "scala-parser-combinators" % versionNumber("scala-parser-combinators") exclude("org.scala-lang", "scala-library") +val scalaXmlDep = "org.scala-lang.modules" %% "scala-xml" % versionNumber("scala-xml") exclude("org.scala-lang", "scala-library") +val partestDep = "org.scala-lang.modules" %% "scala-partest" % versionNumber("partest") exclude("org.scala-lang", "scala-library") +val jlineDep = "jline" % "jline" % versionProps("jline.version") +val antDep = "org.apache.ant" % "ant" % "1.9.4" + lazy val commonSettings = Seq[Setting[_]]( organization := "org.scala-lang", version := "2.11.6-SNAPSHOT", @@ -154,7 +161,7 @@ lazy val compiler = configureAsSubproject(project). settings(generatePropertiesFileSettings: _*). settings( name := "scala-compiler", - libraryDependencies += "org.apache.ant" % "ant" % "1.9.4", + libraryDependencies += antDep, // this a way to make sure that classes from interactive and scaladoc projects // end up in compiler jar (that's what Ant build does) // we need to use LocalProject references (with strings) to deal with mutual recursion @@ -172,19 +179,13 @@ lazy val interactive = configureAsSubproject(project). dependsOn(compiler) lazy val repl = configureAsSubproject(project). - // TODO: in Ant build def, this version is defined in versions.properties - // figure out whether we also want to externalize jline's version - settings(libraryDependencies += "jline" % "jline" % "2.12"). + settings(libraryDependencies += jlineDep). settings(disableDocsAndPublishingTasks: _*). dependsOn(compiler) -def moduleDependency(name: String) = - // exclusion of the scala-library transitive dependency avoids eviction warnings during `update`. - "org.scala-lang.modules" %% name % "1.0.3" exclude("org.scala-lang", "scala-library") - lazy val scaladoc = configureAsSubproject(project). settings( - libraryDependencies ++= Seq("scala-xml", "scala-parser-combinators", "scala-partest").map(moduleDependency) + libraryDependencies ++= Seq(scalaXmlDep, scalaParserCombinatorsDep, partestDep) ). settings(disableDocsAndPublishingTasks: _*). dependsOn(compiler) @@ -345,3 +346,17 @@ lazy val mkBinImpl: Def.Initialize[Task[Seq[File]]] = Def.task { } buildDirectory in ThisBuild := (baseDirectory in ThisBuild).value / "build-sbt" + +lazy val versionProps: Map[String, String] = { + import java.io.FileInputStream + import java.util.Properties + val props = new Properties() + val in = new FileInputStream(file("versions.properties")) + try props.load(in) + finally in.close() + import scala.collection.JavaConverters._ + props.asScala.toMap +} + +def versionNumber(name: String): String = + versionProps(s"$name.version.number") -- cgit v1.2.3 From 9d67e867ea2a71863f890deea33652f06015dc72 Mon Sep 17 00:00:00 2001 From: Jason Zaugg Date: Tue, 10 Mar 2015 17:41:09 -0700 Subject: Projects for partest-extras and junit - Explicitly set [un]managed[re]sourceDirectories for all projects to improve the import into IntelliJ - Add partest extras project, that contains scala/scala specific extensions to partest for use in partest and junit tests. - Add junit project, which contains unit tests for the compiler and library. --- build.sbt | 39 ++++++++++++++++++++++++++++++++++++--- 1 file changed, 36 insertions(+), 3 deletions(-) diff --git a/build.sbt b/build.sbt index 322d8efbaf..b44812173b 100644 --- a/build.sbt +++ b/build.sbt @@ -56,10 +56,11 @@ val bootstrapScalaVersion = "2.11.5" val scalaParserCombinatorsDep = "org.scala-lang.modules" %% "scala-parser-combinators" % versionNumber("scala-parser-combinators") exclude("org.scala-lang", "scala-library") val scalaXmlDep = "org.scala-lang.modules" %% "scala-xml" % versionNumber("scala-xml") exclude("org.scala-lang", "scala-library") val partestDep = "org.scala-lang.modules" %% "scala-partest" % versionNumber("partest") exclude("org.scala-lang", "scala-library") +val junitDep = "junit" % "junit" % "4.11" val jlineDep = "jline" % "jline" % versionProps("jline.version") val antDep = "org.apache.ant" % "ant" % "1.9.4" -lazy val commonSettings = Seq[Setting[_]]( +lazy val commonSettings = clearSourceAndResourceDirectories ++ Seq[Setting[_]]( organization := "org.scala-lang", version := "2.11.6-SNAPSHOT", scalaVersion := bootstrapScalaVersion, @@ -82,7 +83,7 @@ lazy val commonSettings = Seq[Setting[_]]( // Ant's build stored unmanaged jars in `lib/` directory unmanagedJars in Compile := Seq.empty, sourceDirectory in Compile := baseDirectory.value, - sourceDirectories in Compile := Seq(sourceDirectory.value), + unmanagedSourceDirectories in Compile := List(baseDirectory.value), scalaSource in Compile := (sourceDirectory in Compile).value, javaSource in Compile := (sourceDirectory in Compile).value, // resources are stored along source files in our current layout @@ -204,9 +205,32 @@ lazy val forkjoin = configureAsForkOfJavaProject(project) lazy val asm = configureAsForkOfJavaProject(project) +lazy val partestExtras = ( + configureAsSubproject(Project("partest-extras", file(".") / "src" / "partest-extras")) + .dependsOn(repl) + .settings(clearSourceAndResourceDirectories: _*) + .settings( + scalaVersion := bootstrapScalaVersion, + ivyScala := ivyScala.value map { _.copy(overrideScalaVersion = true) }, + libraryDependencies += partestDep, + unmanagedSourceDirectories in Compile := List(baseDirectory.value) + ) + ) + +lazy val junit = project.in(file("test") / "junit") + .dependsOn(library, reflect, compiler, partestExtras) + .settings(clearSourceAndResourceDirectories: _*) + .settings(commonSettings: _*) + .settings( + scalaVersion := bootstrapScalaVersion, + ivyScala := ivyScala.value map { _.copy(overrideScalaVersion = true) }, + libraryDependencies += junitDep, + unmanagedSourceDirectories in Test := List(baseDirectory.value) + ) + lazy val root = (project in file(".")). aggregate(library, forkjoin, reflect, compiler, asm, interactive, repl, - scaladoc, scalap, actors).settings( + scaladoc, scalap, actors, partestExtras, junit).settings( scalaVersion := bootstrapScalaVersion, ivyScala := ivyScala.value map { _.copy(overrideScalaVersion = true) }, sources in Compile := Seq.empty, @@ -310,6 +334,15 @@ lazy val generateVersionPropertiesFileImpl: Def.Initialize[Task[File]] = Def.tas propFile } +// Defining these settings is somewhat redundant as we also redefine settings that depend on them. +// However, IntelliJ's project import works better when these are set correctly. +def clearSourceAndResourceDirectories = Seq(Compile, Test).flatMap(config => inConfig(config)(Seq( + unmanagedSourceDirectories := Nil, + managedSourceDirectories := Nil, + unmanagedResourceDirectories := Nil, + managedResourceDirectories := Nil +))) + lazy val mkBinImpl: Def.Initialize[Task[Seq[File]]] = Def.task { def mkScalaTool(mainCls: String, classpath: Seq[Attributed[File]]): ScalaTool = ScalaTool(mainClass = mainCls, -- cgit v1.2.3 From fc2b35a52fc51d6faf8ff07591be991e1c32102a Mon Sep 17 00:00:00 2001 From: Jason Zaugg Date: Tue, 10 Mar 2015 18:33:59 -0700 Subject: Tweak formatting of build.sbt --- build.sbt | 89 +++++++++++++++++++++++++++++++-------------------------------- 1 file changed, 44 insertions(+), 45 deletions(-) diff --git a/build.sbt b/build.sbt index b44812173b..3cf8723dbd 100644 --- a/build.sbt +++ b/build.sbt @@ -104,10 +104,10 @@ lazy val commonSettings = clearSourceAndResourceDirectories ++ Seq[Setting[_]]( // we disable those tasks by overriding them and returning bogus files when // needed. This is a bit sketchy but I haven't found any better way. val disableDocsAndPublishingTasks = Seq[Setting[_]]( - (doc := file("!!! NO DOCS !!!")), - (publishLocal := {}), - (publish := {}), - (packageBin in Compile := file("!!! NO PACKAGING !!!")) + doc := file("!!! NO DOCS !!!"), + publishLocal := {}, + publish := {}, + packageBin in Compile := file("!!! NO PACKAGING !!!") ) lazy val scalaSubprojectSettings = commonSettings ++ Seq[Setting[_]]( @@ -131,9 +131,10 @@ lazy val generatePropertiesFileSettings = Seq[Setting[_]]( ) val libIncludes: FileFilter = "*.tmpl" | "*.xml" | "*.js" | "*.css" | "rootdoc.txt" -lazy val library = configureAsSubproject(project). - settings(generatePropertiesFileSettings: _*). - settings( + +lazy val library = configureAsSubproject(project) + .settings(generatePropertiesFileSettings: _*) + .settings( name := "scala-library", scalacOptions in Compile ++= Seq[String]("-sourcepath", (scalaSource in Compile).value.toString), // Workaround for a bug in `scaladoc` that it seems to not respect the `-sourcepath` option @@ -147,20 +148,21 @@ lazy val library = configureAsSubproject(project). val libraryAuxDir = (baseDirectory in ThisBuild).value / "src/library-aux" Seq("-doc-no-compile", libraryAuxDir.toString) }, - includeFilter in unmanagedResources in Compile := libIncludes - ) dependsOn (forkjoin) + includeFilter in unmanagedResources in Compile := libIncludes) + .dependsOn (forkjoin) -lazy val reflect = configureAsSubproject(project). - settings(generatePropertiesFileSettings: _*). - settings(name := "scala-reflect"). - dependsOn(library) +lazy val reflect = configureAsSubproject(project) + .settings(generatePropertiesFileSettings: _*) + .settings(name := "scala-reflect") + .dependsOn(library) val compilerIncludes: FileFilter = "*.tmpl" | "*.xml" | "*.js" | "*.css" | "*.html" | "*.properties" | "*.swf" | "*.png" | "*.gif" | "*.gif" | "*.txt" -lazy val compiler = configureAsSubproject(project). - settings(generatePropertiesFileSettings: _*). - settings( + +lazy val compiler = configureAsSubproject(project) + .settings(generatePropertiesFileSettings: _*) + .settings( name := "scala-compiler", libraryDependencies += antDep, // this a way to make sure that classes from interactive and scaladoc projects @@ -171,50 +173,47 @@ lazy val compiler = configureAsSubproject(project). (mappings in Compile in packageBin in LocalProject("interactive")).value ++ (mappings in Compile in packageBin in LocalProject("scaladoc")).value ++ (mappings in Compile in packageBin in LocalProject("repl")).value, - includeFilter in unmanagedResources in Compile := compilerIncludes - ). - dependsOn(library, reflect, asm) + includeFilter in unmanagedResources in Compile := compilerIncludes) + .dependsOn(library, reflect, asm) -lazy val interactive = configureAsSubproject(project). - settings(disableDocsAndPublishingTasks: _*). - dependsOn(compiler) +lazy val interactive = configureAsSubproject(project) + .settings(disableDocsAndPublishingTasks: _*) + .dependsOn(compiler) -lazy val repl = configureAsSubproject(project). - settings(libraryDependencies += jlineDep). - settings(disableDocsAndPublishingTasks: _*). - dependsOn(compiler) +lazy val repl = configureAsSubproject(project) + .settings(libraryDependencies += jlineDep) + .settings(disableDocsAndPublishingTasks: _*) + .dependsOn(compiler) -lazy val scaladoc = configureAsSubproject(project). - settings( +lazy val scaladoc = configureAsSubproject(project) + .settings( libraryDependencies ++= Seq(scalaXmlDep, scalaParserCombinatorsDep, partestDep) - ). - settings(disableDocsAndPublishingTasks: _*). - dependsOn(compiler) + ) + .settings(disableDocsAndPublishingTasks: _*) + .dependsOn(compiler) lazy val scalap = configureAsSubproject(project). dependsOn(compiler) // deprecated Scala Actors project // TODO: it packages into actors.jar but it should be scala-actors.jar -lazy val actors = configureAsSubproject(project). - settings(generatePropertiesFileSettings: _*). - settings(name := "scala-actors"). - dependsOn(library) +lazy val actors = configureAsSubproject(project) + .settings(generatePropertiesFileSettings: _*) + .settings(name := "scala-actors") + .dependsOn(library) lazy val forkjoin = configureAsForkOfJavaProject(project) lazy val asm = configureAsForkOfJavaProject(project) -lazy val partestExtras = ( - configureAsSubproject(Project("partest-extras", file(".") / "src" / "partest-extras")) - .dependsOn(repl) - .settings(clearSourceAndResourceDirectories: _*) - .settings( - scalaVersion := bootstrapScalaVersion, - ivyScala := ivyScala.value map { _.copy(overrideScalaVersion = true) }, - libraryDependencies += partestDep, - unmanagedSourceDirectories in Compile := List(baseDirectory.value) - ) +lazy val partestExtras = configureAsSubproject(Project("partest-extras", file(".") / "src" / "partest-extras")) + .dependsOn(repl) + .settings(clearSourceAndResourceDirectories: _*) + .settings( + scalaVersion := bootstrapScalaVersion, + ivyScala := ivyScala.value map { _.copy(overrideScalaVersion = true) }, + libraryDependencies += partestDep, + unmanagedSourceDirectories in Compile := List(baseDirectory.value) ) lazy val junit = project.in(file("test") / "junit") -- cgit v1.2.3 From dbb5274ae1a9f8f2a2aa89c18ca22733478743e7 Mon Sep 17 00:00:00 2001 From: Jason Zaugg Date: Sat, 14 Mar 2015 19:39:30 -0700 Subject: `test` task runs Junit tests Add dependency on scaladoc in junit subproject, as required by 6e6632e (it introduced a junit test for scaladoc). Enable assertion stack traces and info logging of each test. --- build.sbt | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/build.sbt b/build.sbt index 3cf8723dbd..2f2b30147a 100644 --- a/build.sbt +++ b/build.sbt @@ -57,6 +57,7 @@ val scalaParserCombinatorsDep = "org.scala-lang.modules" %% "scala-parser-combin val scalaXmlDep = "org.scala-lang.modules" %% "scala-xml" % versionNumber("scala-xml") exclude("org.scala-lang", "scala-library") val partestDep = "org.scala-lang.modules" %% "scala-partest" % versionNumber("partest") exclude("org.scala-lang", "scala-library") val junitDep = "junit" % "junit" % "4.11" +val junitIntefaceDep = "com.novocode" % "junit-interface" % "0.11" % "test" val jlineDep = "jline" % "jline" % versionProps("jline.version") val antDep = "org.apache.ant" % "ant" % "1.9.4" @@ -217,13 +218,15 @@ lazy val partestExtras = configureAsSubproject(Project("partest-extras", file(". ) lazy val junit = project.in(file("test") / "junit") - .dependsOn(library, reflect, compiler, partestExtras) + .dependsOn(library, reflect, compiler, partestExtras, scaladoc) .settings(clearSourceAndResourceDirectories: _*) .settings(commonSettings: _*) .settings( scalaVersion := bootstrapScalaVersion, ivyScala := ivyScala.value map { _.copy(overrideScalaVersion = true) }, - libraryDependencies += junitDep, + fork in Test := true, + libraryDependencies ++= Seq(junitDep, junitIntefaceDep), + testOptions += Tests.Argument(TestFrameworks.JUnit, "-a", "-v"), unmanagedSourceDirectories in Test := List(baseDirectory.value) ) -- cgit v1.2.3 From 0e70049f9bcda462fcc30971227bf0c53b90cb9f Mon Sep 17 00:00:00 2001 From: Grzegorz Kossakowski Date: Mon, 23 Mar 2015 17:13:11 -0700 Subject: `test/it:test` runs partest regression tests Partest tests are defined in Integration configuration. This means you need to run them with `test/it:test` command. We put them in separate configuration because they are slow to run so user needs to ask explicitly for partest to run. Introduce a common way of setting jar location so it can be reused between `scalaSubprojectSettings` and `partestJavaAgent`. --- build.sbt | 62 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 60 insertions(+), 2 deletions(-) diff --git a/build.sbt b/build.sbt index 2f2b30147a..d89bdd8067 100644 --- a/build.sbt +++ b/build.sbt @@ -56,10 +56,12 @@ val bootstrapScalaVersion = "2.11.5" val scalaParserCombinatorsDep = "org.scala-lang.modules" %% "scala-parser-combinators" % versionNumber("scala-parser-combinators") exclude("org.scala-lang", "scala-library") val scalaXmlDep = "org.scala-lang.modules" %% "scala-xml" % versionNumber("scala-xml") exclude("org.scala-lang", "scala-library") val partestDep = "org.scala-lang.modules" %% "scala-partest" % versionNumber("partest") exclude("org.scala-lang", "scala-library") +val partestInterfaceDep = "org.scala-lang.modules" %% "scala-partest-interface" % "0.5.0" exclude("org.scala-lang", "scala-library") val junitDep = "junit" % "junit" % "4.11" val junitIntefaceDep = "com.novocode" % "junit-interface" % "0.11" % "test" val jlineDep = "jline" % "jline" % versionProps("jline.version") val antDep = "org.apache.ant" % "ant" % "1.9.4" +val scalacheckDep = "org.scalacheck" %% "scalacheck" % "1.11.4" exclude("org.scala-lang", "scala-library") lazy val commonSettings = clearSourceAndResourceDirectories ++ Seq[Setting[_]]( organization := "org.scala-lang", @@ -111,7 +113,7 @@ val disableDocsAndPublishingTasks = Seq[Setting[_]]( packageBin in Compile := file("!!! NO PACKAGING !!!") ) -lazy val scalaSubprojectSettings = commonSettings ++ Seq[Setting[_]]( +lazy val setJarLocation: Setting[_] = artifactPath in packageBin in Compile := { // two lines below are copied over from sbt's sources: // https://github.com/sbt/sbt/blob/0.13/main/src/main/scala/sbt/Defaults.scala#L628 @@ -123,7 +125,7 @@ lazy val scalaSubprojectSettings = commonSettings ++ Seq[Setting[_]]( val resolvedArtifactName = s"${resolvedArtifact.name}.${resolvedArtifact.extension}" buildDirectory.value / "pack/lib" / resolvedArtifactName } -) +lazy val scalaSubprojectSettings: Seq[Setting[_]] = commonSettings :+ setJarLocation lazy val generatePropertiesFileSettings = Seq[Setting[_]]( copyrightString := "Copyright 2002-2013, LAMP/EPFL", @@ -179,6 +181,10 @@ lazy val compiler = configureAsSubproject(project) lazy val interactive = configureAsSubproject(project) .settings(disableDocsAndPublishingTasks: _*) + .settings( + scalaVersion := bootstrapScalaVersion, + ivyScala := ivyScala.value map { _.copy(overrideScalaVersion = true) } + ) .dependsOn(compiler) lazy val repl = configureAsSubproject(project) @@ -230,6 +236,58 @@ lazy val junit = project.in(file("test") / "junit") unmanagedSourceDirectories in Test := List(baseDirectory.value) ) +lazy val partestJavaAgent = (project in file(".") / "src" / "partest-javaagent"). + dependsOn(asm). + settings(commonSettings: _*). + settings( + doc := file("!!! NO DOCS !!!"), + publishLocal := {}, + publish := {}, + scalaVersion := bootstrapScalaVersion, + ivyScala := ivyScala.value map { _.copy(overrideScalaVersion = true) }, + // Setting name to "scala-partest-javaagent" so that the jar file gets that name, which the Runner relies on + name := "scala-partest-javaagent", + // writing jar file to $buildDirectory/pack/lib because that's where it's expected to be found + setJarLocation, + // add required manifest entry - previously included from file + packageOptions in (Compile, packageBin) += + Package.ManifestAttributes( "Premain-Class" -> "scala.tools.partest.javaagent.ProfilingAgent" ), + // we need to build this to a JAR + exportJars := true + ) + +lazy val test = project. + dependsOn(compiler, interactive, actors, repl, scalap, partestExtras, partestJavaAgent, asm, scaladoc). + configs(IntegrationTest). + settings(disableDocsAndPublishingTasks: _*). + settings(commonSettings: _*). + settings(Defaults.itSettings: _*). + settings( + scalaVersion := bootstrapScalaVersion, + ivyScala := ivyScala.value map { _.copy(overrideScalaVersion = true) }, + libraryDependencies ++= Seq(partestDep, scalaXmlDep, partestInterfaceDep, scalacheckDep), + unmanagedBase in Test := baseDirectory.value / "files" / "lib", + unmanagedJars in Test <+= (unmanagedBase) (j => Attributed.blank(j)) map(identity), + // no main sources + sources in Compile := Seq.empty, + // test sources are compiled in partest run, not here + sources in IntegrationTest := Seq.empty, + fork in IntegrationTest := true, + javaOptions in IntegrationTest += "-Xmx1G", + testFrameworks += new TestFramework("scala.tools.partest.Framework"), + testOptions in IntegrationTest += Tests.Setup( () => root.base.getAbsolutePath + "/pull-binary-libs.sh" ! ), + definedTests in IntegrationTest += ( + new sbt.TestDefinition( + "partest", + // marker fingerprint since there are no test classes + // to be discovered by sbt: + new sbt.testing.AnnotatedFingerprint { + def isModule = true + def annotationName = "partest" + }, true, Array()) + ) + ) + lazy val root = (project in file(".")). aggregate(library, forkjoin, reflect, compiler, asm, interactive, repl, scaladoc, scalap, actors, partestExtras, junit).settings( -- cgit v1.2.3 From ccd586f68af14ea693c873d38ad8d5210caa48e0 Mon Sep 17 00:00:00 2001 From: Grzegorz Kossakowski Date: Sat, 21 Feb 2015 21:13:07 +0100 Subject: Script that diffs `build/` and `build-sbt/` To ensure the sbt build matches the ant build, diff their output. The scripts just delegates to `diff` command that compares directories recursively. We've added some handy ignore patterns (e.g. for .complete files). Ignore locker and deps dirs when comparing build and build-sbt, these directories are needed by Ant build only. Ignore irrelevant jars in build/pack/lib generated by the Ant build. --- compare-build-dirs-ignore-patterns | 8 ++++++++ compare-build-dirs.sh | 5 +++++ 2 files changed, 13 insertions(+) create mode 100644 compare-build-dirs-ignore-patterns create mode 100755 compare-build-dirs.sh diff --git a/compare-build-dirs-ignore-patterns b/compare-build-dirs-ignore-patterns new file mode 100644 index 0000000000..8c8160ba15 --- /dev/null +++ b/compare-build-dirs-ignore-patterns @@ -0,0 +1,8 @@ +.DS_Store +*.complete +locker +deps +scala-continuations-*.jar +scala-parser-combinators*.jar +scala-swing*.jar +scala-xml*.jar diff --git a/compare-build-dirs.sh b/compare-build-dirs.sh new file mode 100755 index 0000000000..f6806dd422 --- /dev/null +++ b/compare-build-dirs.sh @@ -0,0 +1,5 @@ +# Compares build directories generated by Ant and sbt build definitions +# This let's us to see how far are we from achieving perfect parity +# between the builds + +diff -X compare-build-dirs-ignore-patterns -qr build/ build-sbt/ -- cgit v1.2.3 From 0d5ff65499a16ed1004bbdf7f204995082ab5dd0 Mon Sep 17 00:00:00 2001 From: Grzegorz Kossakowski Date: Sat, 21 Feb 2015 21:44:11 +0100 Subject: Align how ant builds asm & forkjoin There were two inconsistencies in building asm & forkjoin: 1. Asm classes lived in `build/asm`, while forkjoin's were in `build/libs/classes/forkjoin`. 2. Though no jars are needed for these projects, forkjoin was packaged, but asm was not. No reason for these inconsistencies could be found in the history. --- build.xml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/build.xml b/build.xml index 4b79b68a02..0a67f8a563 100755 --- a/build.xml +++ b/build.xml @@ -165,7 +165,7 @@ TODO: - + @@ -588,8 +588,8 @@ TODO: - - + + @@ -1061,7 +1061,7 @@ TODO: ============================================================================ --> - +