aboutsummaryrefslogtreecommitdiff
path: root/spark-class
Commit message (Collapse)AuthorAgeFilesLines
* Removed sbt folder and changed docs accordinglyPrashant Sharma2014-01-021-1/+1
|
* Response to Shivaram's reviewPatrick Wendell2013-12-301-14/+17
|
* SPARK-1008: Logging improvmentsPatrick Wendell2013-12-291-10/+12
| | | | | | 1. Adds a default log4j file that gets loaded if users haven't specified a log4j file. 2. Isolates use of the tools assembly jar. I found this produced SLF4J warnings after building with SBT (and I've seen similar warnings on the mailing list).
* Merge pull request #275 from ueshin/wip/changeclasspathorderMatei Zaharia2013-12-241-3/+3
|\ | | | | | | | | | | | | | | | | | | | | Change the order of CLASSPATH. SPARK_TOOLS_JAR should be placed after CLASSPATH or at least after SPARK_CLASSPATH. If SPARK_TOOLS_JAR is placed before CLASSPATH, all assembled classes and resources in spark-tools-assembly.jar beat those in CLASSPATH or SPARK_CLASSPATH, which might be replaced by customized versions.
| * Change the order of CLASSPATH.Takuya UESHIN2013-12-191-1/+1
| |
* | Fix Cygwin support in several scripts.Josh Rosen2013-12-151-0/+10
|/ | | | | | | | | This allows the spark-shell, spark-class, run-example, make-distribution.sh, and ./bin/start-* scripts to work under Cygwin. Note that this doesn't support PySpark under Cygwin, since that requires many additional `cygpath` calls from within Python and will be non-trivial to implement. This PR was inspired by, and subsumes, #253 (so close #253 after this is merged).
* Use CoarseGrainedExecutorBackend in spark-classShivaram Venkataraman2013-12-151-1/+1
|
* Remote death watch has a funny bug.Prashant Sharma2013-11-251-0/+1
| | | | https://gist.github.com/ScrapCodes/4805fd84906e40b7b03d
* Merge branch 'master' into scala-2.10Raymond Liu2013-11-141-0/+13
|\
| * Add spark-tools assembly to spark-class classpath.Josh Rosen2013-11-091-0/+13
| | | | | | | | | | This allows the JavaAPICompletenessChecker to be run with Spark 0.8+.
* | Merge branch 'master' into scala-2.10Raymond Liu2013-11-131-4/+11
|\|
| * Adding improved error message when multiple assembly jars are present.Patrick Wendell2013-10-251-4/+11
| | | | | | | | This can happen easily if building different hadoop versions.
* | Sync with master and some build fixesPrashant Sharma2013-09-261-6/+6
|\|
| * Add "org.apache." prefix to packages in spark-classAaron Davidson2013-09-201-6/+6
| | | | | | | | Lacking this, the if/case statements never trigger on Spark 0.8.0+.
* | Fixed repl suitePrashant Sharma2013-09-151-1/+1
| |
* | Merged with masterPrashant Sharma2013-09-061-0/+1
|/
* Run script fixes for Windows after package & assembly changeMatei Zaharia2013-09-011-1/+1
|
* Merge pull request #880 from mateiz/ui-tweaksMatei Zaharia2013-08-311-7/+0
|\ | | | | Various UI tweaks
| * Delete some code that was added back in a merge and print less info inMatei Zaharia2013-08-311-7/+0
| | | | | | | | spark-daemon
* | More doc improvements + better warnings when you haven't built SparkMatei Zaharia2013-08-301-2/+2
|/
* Change build and run instructions to use assembliesMatei Zaharia2013-08-291-0/+124
This commit makes Spark invocation saner by using an assembly JAR to find all of Spark's dependencies instead of adding all the JARs in lib_managed. It also packages the examples into an assembly and uses that as SPARK_EXAMPLES_JAR. Finally, it replaces the old "run" script with two better-named scripts: "run-examples" for examples, and "spark-class" for Spark internal classes (e.g. REPL, master, etc). This is also designed to minimize the confusion people have in trying to use "run" to run their own classes; it's not meant to do that, but now at least if they look at it, they can modify run-examples to do a decent job for them. As part of this, Bagel's examples are also now properly moved to the examples package instead of bagel.