aboutsummaryrefslogtreecommitdiff
path: root/core/src
Commit message (Collapse)AuthorAgeFilesLines
* Merge branch 'master' into master-mergePrashant Sharma2013-07-03173-2365/+9792
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/pom.xml core/src/main/scala/spark/MapOutputTracker.scala core/src/main/scala/spark/RDD.scala core/src/main/scala/spark/RDDCheckpointData.scala core/src/main/scala/spark/SparkContext.scala core/src/main/scala/spark/Utils.scala core/src/main/scala/spark/api/python/PythonRDD.scala core/src/main/scala/spark/deploy/client/Client.scala core/src/main/scala/spark/deploy/master/MasterWebUI.scala core/src/main/scala/spark/deploy/worker/Worker.scala core/src/main/scala/spark/deploy/worker/WorkerWebUI.scala core/src/main/scala/spark/rdd/BlockRDD.scala core/src/main/scala/spark/rdd/ZippedRDD.scala core/src/main/scala/spark/scheduler/cluster/StandaloneSchedulerBackend.scala core/src/main/scala/spark/storage/BlockManager.scala core/src/main/scala/spark/storage/BlockManagerMaster.scala core/src/main/scala/spark/storage/BlockManagerMasterActor.scala core/src/main/scala/spark/storage/BlockManagerUI.scala core/src/main/scala/spark/util/AkkaUtils.scala core/src/test/scala/spark/SizeEstimatorSuite.scala pom.xml project/SparkBuild.scala repl/src/main/scala/spark/repl/SparkILoop.scala repl/src/test/scala/spark/repl/ReplSuite.scala streaming/src/main/scala/spark/streaming/StreamingContext.scala streaming/src/main/scala/spark/streaming/api/java/JavaStreamingContext.scala streaming/src/main/scala/spark/streaming/dstream/KafkaInputDStream.scala streaming/src/main/scala/spark/streaming/util/MasterFailureTest.scala
| * Fixed PySpark perf regression by not using socket.makefile(), and improvedroot2013-07-012-4/+26
| | | | | | | | | | | | | | debuggability by letting "print" statements show up in the executor's stderr Conflicts: core/src/main/scala/spark/api/python/PythonRDD.scala
| * Fix performance bug with new Python code not using buffered streamsroot2013-07-012-17/+19
| |
| * Made use of spark.executor.memory setting consistent and documented itMatei Zaharia2013-06-302-17/+18
| | | | | | | | | | | | Conflicts: core/src/main/scala/spark/SparkContext.scala
| * Merge pull request #664 from pwendell/test-fixMatei Zaharia2013-06-271-1/+0
| |\ | | | | | | Removing incorrect test statement
| | * Removing incorrect test statementPatrick Wendell2013-06-271-1/+0
| | |
| * | Wrap lines.Stephen Haberman2013-06-261-1/+3
| | |
| * | Be cute with Option and getenv.Stephen Haberman2013-06-261-17/+4
| | |
| * | Refactored tests to share SparkContexts in some of themMatei Zaharia2013-06-2510-425/+373
| | | | | | | | | | | | | | | Creating these seems to take a while and clutters the output with Akka stuff, so it would be nice to share them.
| * | Fix computation of classpath when we launch java directlyMatei Zaharia2013-06-252-16/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | The previous version assumed that a CLASSPATH environment variable was set by the "run" script when launching the process that starts the ExecutorRunner, but unfortunately this is not true in tests. Instead, we factor the classpath calculation into an extenral script and call that. NOTE: This includes a Windows version but hasn't yet been tested there.
| * | Some fixes to the launch-java-directly change:Matei Zaharia2013-06-253-31/+138
| | | | | | | | | | | | | | | | | | | | | | | | - Split SPARK_JAVA_OPTS into multiple command-line arguments if it contains spaces; this splitting follows quoting rules in bash - Add the Scala JARs to the classpath if they're not in the CLASSPATH variable because the ExecutorRunner is launched with "scala" (this can happen when using local-cluster URLs in spark-shell)
| * | Fixed deprecated use of expect in SizeEstimatorSuiteMatei Zaharia2013-06-251-36/+36
| | |
| * | Merge remote-tracking branch 'cgrothaus/SPARK-698'Matei Zaharia2013-06-251-4/+29
| |\ \ | | | | | | | | | | | | | | | | Conflicts: run
| | * | Incorporate feedback from mateiz:Christoph Grothaus2013-02-241-17/+7
| | | | | | | | | | | | | | | | | | | | - we do not need getEnvOrEmpty - Instead of saving SPARK_NONDAEMON_JAVA_OPTS, it would be better to modify the scripts to use a different variable name for the JAVA_OPTS they do eventually use
| | * | Fix SPARK-698. From ExecutorRunner, launch java directly instead via the run ↵Christoph Grothaus2013-02-201-4/+39
| | | | | | | | | | | | | | | | scripts.
| * | | Clone the zero value for each key in foldByKeyMatei Zaharia2013-06-232-3/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The old version reused the object within each task, leading to overwriting of the object when a mutable type is used, which is expected to be common in fold. Conflicts: core/src/test/scala/spark/ShuffleSuite.scala
| * | | Fix search path for REPL class loader to really find added JARsMatei Zaharia2013-06-221-16/+22
| | | |
| * | | Merge pull request #648 from shivaram/netty-dbgMatei Zaharia2013-06-227-86/+151
| |\ \ \ | | |_|/ | |/| | Shuffle fixes and cleanup
| | * | Fix some comments and styleShivaram Venkataraman2013-06-135-14/+8
| | | |
| | * | Merge branch 'netty-dbg' of github.com:shivaram/spark into netty-dbgShivaram Venkataraman2013-06-124-39/+94
| | |\ \
| | | * | Clean up variables and counters in BlockFetcherIteratorShivaram Venkataraman2013-06-061-23/+31
| | | | |
| | | * | Pass in bufferSize to BufferedOutputStreamShivaram Venkataraman2013-06-051-1/+1
| | | | |
| | | * | Don't write zero block files with java serializerShivaram Venkataraman2013-06-054-18/+61
| | | | |
| | | * | var to valShivaram Venkataraman2013-06-031-1/+1
| | | | |
| | | * | Reuse the file object as it is valid after deleteShivaram Venkataraman2013-06-031-2/+0
| | | | |
| | | * | Delete a file for a block if it already exists.Shivaram Venkataraman2013-06-031-2/+8
| | | | |
| | * | | Merge branch 'master' of git://github.com/mesos/spark into netty-dbgShivaram Venkataraman2013-06-126-23/+34
| | |\ \ \ | | | |/ / | | |/| |
| | * | | Make connect timeout configurableShivaram Venkataraman2013-05-312-3/+6
| | | | |
| | * | | Another round of Netty fixes.Shivaram Venkataraman2013-05-314-45/+58
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1. Avoid race condition between stop and copier completion 2. Handle socket exceptions by reporting them and filling in a failed FetchResult
| * | | | Merge pull request #644 from shimingfei/jobloggerMatei Zaharia2013-06-2211-19/+497
| |\ \ \ \ | | | | | | | | | | | | add Joblogger to Spark (on new Spark code)
| | * | | | small modify in DAGSchedulerMingfei2013-06-211-1/+1
| | | | | |
| | * | | | small format modificationMingfei2013-06-211-2/+2
| | | | | |
| | * | | | some format modificationMingfei2013-06-212-8/+7
| | | | | |
| | * | | | edit according to commentsMingfei2013-06-217-69/+62
| | | | | |
| | * | | | modify sparklister function interface according to commentsMingfei2013-06-134-73/+79
| | | | | |
| | * | | | not check return value of eventQueue.takeMingfei2013-06-081-15/+13
| | | | | |
| | * | | | delete test code for joblogger in SparkContextMingfei2013-06-081-2/+1
| | | | | |
| | * | | | Merge branch 'master' of https://github.com/mesos/sparkMingfei2013-06-083-20/+31
| | |\ \ \ \ | | | | |/ / | | | |/| |
| | * | | | modify to pass job annotation by localProperties and use daeamon thread to ↵Mingfei2013-06-0811-12/+495
| | | | | | | | | | | | | | | | | | | | | | | | do joblogger's work
| * | | | | use parens when calling method with side-effectsJey Kottalam2013-06-213-10/+10
| | | | | |
| * | | | | Rename PythonWorker to PythonWorkerFactoryJey Kottalam2013-06-213-15/+17
| | | | | |
| * | | | | Add tests and fixes for Python daemon shutdownJey Kottalam2013-06-212-0/+5
| | | | | |
| * | | | | Prefork Python worker processesJey Kottalam2013-06-213-41/+125
| | | | | |
| * | | | | Add Python timing instrumentationJey Kottalam2013-06-211-0/+12
| | | | | |
| * | | | | Merge pull request #654 from lyogavin/enhance_pipeMatei Zaharia2013-06-192-8/+8
| |\ \ \ \ \ | | | | | | | | | | | | | | fix typo and coding style in #638
| | * | | | | fix typo and coding styleGavin Li2013-06-182-8/+8
| | | | | | |
| * | | | | | Update ASM to version 4.0Matei Zaharia2013-06-191-6/+5
| | | | | | |
| * | | | | | reduce ZippedPartitionsRDD's getPreferredLocations complexityjerryshao2013-06-181-8/+4
| | | | | | |
| * | | | | | Merge pull request #643 from adatao/masterMatei Zaharia2013-06-172-12/+35
| |\ \ \ \ \ \ | | | | | | | | | | | | | | | | Bug fix: Zero-length partitions result in NaN for overall mean & variance
| | * | | | | | Import just scala.math.abs rather than scala.math._Christopher Nguyen2013-06-161-1/+1
| | | | | | | |
generated by cgit v1.2.3 (git 2.39.1) at 2024-08-06 08:10:30 +0000