| Commit message (Collapse) | Author | Age | Files | Lines |
|\
| |
| |
| |
| |
| |
| |
| | |
Conflicts:
run
Previous changes that I made to run and set-dev-classpath.sh instead
have been folded into compute-classpath.sh
|
| | |
|
| |
| |
| |
| |
| |
| |
| |
| |
| | |
The previous version assumed that a CLASSPATH environment variable was
set by the "run" script when launching the process that starts the
ExecutorRunner, but unfortunately this is not true in tests. Instead, we
factor the classpath calculation into an extenral script and call that.
NOTE: This includes a Windows version but hasn't yet been tested there.
|
| |\
| | |
| | |
| | |
| | | |
Conflicts:
run
|
| | |
| | |
| | |
| | |
| | | |
- we do not need getEnvOrEmpty
- Instead of saving SPARK_NONDAEMON_JAVA_OPTS, it would be better to modify the scripts to use a different variable name for the JAVA_OPTS they do eventually use
|
| | |
| | |
| | |
| | | |
scripts.
|
| | | |
|
| | | |
|
|/ / |
|
| |
| |
| |
| |
| | |
check for core classes in run. This fixed the problem that core tests
depend on whether repl module is compiled or not.
|
| |
| |
| |
| |
| |
| | |
which is used to specify the client side configuration directory : which needs to be part of the CLASSPATH.
2) Move from var+=".." to var="$var.." : the former does not work on older bash shells unfortunately.
|
| | |
|
| |
| |
| |
| |
| |
| |
| |
| | |
SPARK_LAUNCH_WITH_SCALA is defined): instead of checking in the PATH first, and only then (if not found) for SCALA_HOME, now we check for SCALA_HOME first, and only then (if not defined) do we look in the PATH. The advantage is that now if the user has a more recent (non-compatible) version of scala in her PATH, she can use SCALA_HOME to point to the older (compatible) version for use with spark.
Suggested by Josh Rosen in this thread:
https://groups.google.com/forum/?fromgroups=#!topic/spark-users/NC9JKvP8808
|
|\ \
| | |
| | | |
SPARK-724 - Have Akka logging enabled by default for standalone daemons
|
| | | |
|
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
See the JIRA for more details.
I was only able to test the bash version (don't have Windows)
so maybe check over that the syntax is correct there.
|
|/ /
| |
| |
| |
| | |
Conflicts:
project/SparkBuild.scala
|
| | |
|
| | |
|
| | |
|
| | |
|
|\ \ |
|
| |/ |
|
|\|
| |
| |
| |
| |
| | |
Conflicts:
core/src/main/scala/spark/rdd/CheckpointRDD.scala
streaming/src/main/scala/spark/streaming/dstream/ReducedWindowedDStream.scala
|
| |
| |
| |
| |
| | |
Conflicts:
docs/_config.yml
|
|\| |
|
| | |
|
|\ \
| |/
|/| |
Removing offset management code that is non-existent in kafka 0.7.0+
|
| | |
|
|\ \
| | |
| | |
| | |
| | | |
Conflicts:
core/src/main/scala/spark/api/python/PythonRDD.scala
|
| | | |
|
|\| |
| |/
|/|
| |
| |
| |
| |
| |
| | |
Conflicts:
core/src/main/scala/spark/rdd/CoGroupedRDD.scala
core/src/main/scala/spark/rdd/FilteredRDD.scala
docs/_layouts/global.html
docs/index.md
run
|
| |\
| | |
| | | |
Python API (PySpark)
|
| | | |
|
| | |\
| | | |
| | | |
| | | |
| | | | |
Conflicts:
docs/quick-start.md
|
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
- Bundle Py4J binaries, since it's hard to install
- Uses Spark's `run` script to launch the Py4J
gateway, inheriting the settings in spark-env.sh
With these changes, (hopefully) nothing more than
running `sbt/sbt package` will be necessary to run
PySpark.
|
| | |/
| |/| |
|
| | | |
|
|\| |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Conflicts:
core/src/main/scala/spark/MapOutputTracker.scala
core/src/main/scala/spark/PairRDDFunctions.scala
core/src/main/scala/spark/ParallelCollection.scala
core/src/main/scala/spark/RDD.scala
core/src/main/scala/spark/rdd/BlockRDD.scala
core/src/main/scala/spark/rdd/CartesianRDD.scala
core/src/main/scala/spark/rdd/CoGroupedRDD.scala
core/src/main/scala/spark/rdd/CoalescedRDD.scala
core/src/main/scala/spark/rdd/FilteredRDD.scala
core/src/main/scala/spark/rdd/FlatMappedRDD.scala
core/src/main/scala/spark/rdd/GlommedRDD.scala
core/src/main/scala/spark/rdd/HadoopRDD.scala
core/src/main/scala/spark/rdd/MapPartitionsRDD.scala
core/src/main/scala/spark/rdd/MapPartitionsWithSplitRDD.scala
core/src/main/scala/spark/rdd/MappedRDD.scala
core/src/main/scala/spark/rdd/PipedRDD.scala
core/src/main/scala/spark/rdd/SampledRDD.scala
core/src/main/scala/spark/rdd/ShuffledRDD.scala
core/src/main/scala/spark/rdd/UnionRDD.scala
core/src/main/scala/spark/storage/BlockManager.scala
core/src/main/scala/spark/storage/BlockManagerId.scala
core/src/main/scala/spark/storage/BlockManagerMaster.scala
core/src/main/scala/spark/storage/StorageLevel.scala
core/src/main/scala/spark/util/MetadataCleaner.scala
core/src/main/scala/spark/util/TimeStampedHashMap.scala
core/src/test/scala/spark/storage/BlockManagerSuite.scala
run
|
| | | |
|
| | |
| | |
| | |
| | | |
ReducedWindowedDStream persist RDDs to MEMOERY_SER_ONLY by default. Removed unncessary examples. Added streaming-env.sh.template to add recommended setting for streaming.
|
|\| |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Conflicts:
core/src/main/scala/spark/BlockStoreShuffleFetcher.scala
core/src/main/scala/spark/KryoSerializer.scala
core/src/main/scala/spark/MapOutputTracker.scala
core/src/main/scala/spark/RDD.scala
core/src/main/scala/spark/SparkContext.scala
core/src/main/scala/spark/executor/Executor.scala
core/src/main/scala/spark/network/Connection.scala
core/src/main/scala/spark/network/ConnectionManagerTest.scala
core/src/main/scala/spark/rdd/BlockRDD.scala
core/src/main/scala/spark/rdd/NewHadoopRDD.scala
core/src/main/scala/spark/scheduler/ShuffleMapTask.scala
core/src/main/scala/spark/scheduler/cluster/StandaloneSchedulerBackend.scala
core/src/main/scala/spark/storage/BlockManager.scala
core/src/main/scala/spark/storage/BlockMessage.scala
core/src/main/scala/spark/storage/BlockStore.scala
core/src/main/scala/spark/storage/StorageLevel.scala
core/src/main/scala/spark/util/AkkaUtils.scala
project/SparkBuild.scala
run
|
| |/ |
|
| | |
|
| |
| |
| |
| | |
set; fixes #216
|
| | |
|
| | |
|
| |
| |
| |
| | |
involved in that
|
|/ |
|
| |
|