diff options
author | Michael Armbrust <michael@databricks.com> | 2015-02-07 00:14:38 -0800 |
---|---|---|
committer | Reynold Xin <rxin@databricks.com> | 2015-02-07 00:14:38 -0800 |
commit | e9a4fe12d3dc4afeb715c7649df2c5527a2d5d35 (patch) | |
tree | 49dea7304f9020994c9dd3b0aedcebb1760ba5dc /project/SparkBuild.scala | |
parent | 1390e56fa86d07b588f0c3a1a2cc25ab5bb0db67 (diff) | |
download | spark-e9a4fe12d3dc4afeb715c7649df2c5527a2d5d35.tar.gz spark-e9a4fe12d3dc4afeb715c7649df2c5527a2d5d35.tar.bz2 spark-e9a4fe12d3dc4afeb715c7649df2c5527a2d5d35.zip |
[BUILD] Add the ability to launch spark-shell from SBT.
Now you can quickly launch the spark-shell without building an assembly. For quick development iteration run `build/sbt ~sparkShell` and calling exit will relaunch with any changes.
Author: Michael Armbrust <michael@databricks.com>
Closes #4438 from marmbrus/sparkShellSbt and squashes the following commits:
b4e44fe [Michael Armbrust] [BUILD] Add the ability to launch spark-shell from SBT.
Diffstat (limited to 'project/SparkBuild.scala')
-rw-r--r-- | project/SparkBuild.scala | 23 |
1 files changed, 23 insertions, 0 deletions
diff --git a/project/SparkBuild.scala b/project/SparkBuild.scala index 5e3051c091..95f8dfa3d2 100644 --- a/project/SparkBuild.scala +++ b/project/SparkBuild.scala @@ -177,6 +177,29 @@ object SparkBuild extends PomBuild { enable(Flume.settings)(streamingFlumeSink) + + /** + * Adds the ability to run the spark shell directly from SBT without building an assembly + * jar. + * + * Usage: `build/sbt sparkShell` + */ + val sparkShell = taskKey[Unit]("start a spark-shell.") + + enable(Seq( + connectInput in run := true, + fork := true, + outputStrategy in run := Some (StdoutOutput), + + javaOptions ++= Seq("-Xmx2G", "-XX:MaxPermSize=1g"), + + sparkShell := { + (runMain in Compile).toTask(" org.apache.spark.repl.Main -usejavacp").value + } + ))(assembly) + + enable(Seq(sparkShell := sparkShell in "assembly"))(spark) + // TODO: move this to its upstream project. override def projectDefinitions(baseDirectory: File): Seq[Project] = { super.projectDefinitions(baseDirectory).map { x => |