aboutsummaryrefslogtreecommitdiff
path: root/core
Commit message (Collapse)AuthorAgeFilesLines
* Support for Hadoop 2 distributions such as cdh4Thomas Dudziak2012-10-187-20/+45
|
* Made ShuffleDependency automatically find a shuffle ID for itselfMatei Zaharia2012-10-143-5/+6
|
* Take executor environment vars as an arguemnt to SparkContextMatei Zaharia2012-10-137-79/+107
|
* Protect from null env variables in mesos.Denny2012-10-132-8/+16
|
* Protect from setting null environment variables.Denny2012-10-131-1/+5
|
* Don't use system envs for Mesos.Denny2012-10-132-2/+2
|
* Let the user specify environment variables to be passed to the Executors.Denny2012-10-135-48/+21
| | | | Also removed unused variables in the ExecutorRunner.
* More doc updates, and moved Serializer to a subpackage.Matei Zaharia2012-10-1212-25/+51
|
* Some doc and usability improvements:Matei Zaharia2012-10-1211-22/+83
| | | | | | | - Added a StorageLevels class for easy access to StorageLevel constants in Java - Added doc comments on Function classes in Java - Updated Accumulator and HadoopWriter docs slightly
* Added a test for when an RDD only partially fits in memoryMatei Zaharia2012-10-121-2/+18
|
* Document cartesian() operationMatei Zaharia2012-10-122-0/+8
|
* Merge pull request #271 from shivaram/block-manager-npe-fixMatei Zaharia2012-10-128-39/+58
|\ | | | | Change block manager to accept a ArrayBuffer
| * Add test to verify if RDD is computed even if block manager has insufficientShivaram Venkataraman2012-10-121-0/+10
| | | | | | | | memory
| * Change block manager to accept a ArrayBuffer instead of an iterator to ensureShivaram Venkataraman2012-10-117-39/+48
| | | | | | | | | | that the computation can proceed even if we run out of memory to cache the block. Update CacheTracker to use this new interface
* | Adding Java documentationPatrick Wendell2012-10-116-10/+454
|/
* Fixed bug when fetching Jar dependencies.Denny2012-10-102-6/+6
| | | | Instead of checking currentFiles check currentJars.
* Added documentation to all the *RDDFunction classes, and moved them intoMatei Zaharia2012-10-099-54/+273
| | | | | the spark package to make them more visible. Also documented various other miscellaneous things in the API.
* Updates to documentation:Matei Zaharia2012-10-091-1/+1
| | | | | | | | - Edited quick start and tuning guide to simplify them a little - Simplified top menu bar - Made private a SparkContext constructor parameter that was left as public - Various small fixes
* Fixes a typo, adds scaladoc comments to SparkContext constructors.Andy Konwinski2012-10-082-5/+11
|
* More docs in RDD classPatrick Wendell2012-10-081-1/+45
|
* A start on scaladoc for the public APIs.Andy Konwinski2012-10-081-6/+29
|
* Merge branch 'dev' into bc-fix-devMosharaf Chowdhury2012-10-0871-477/+1108
|\
| * Made compression configurable separately for shuffle, broadcast and RDDsMatei Zaharia2012-10-074-38/+118
| |
| * Merge pull request #251 from JoshRosen/docs/internalsMatei Zaharia2012-10-075-11/+40
| |\ | | | | | | Document Dependency classes and make minor interface improvements
| | * Make ShuffleDependency.aggregator explicitly optional.Josh Rosen2012-10-074-7/+11
| | | | | | | | | | | | | | | | | | | | | | | | It was confusing to be using new Aggregator[K, V, V](null, null, null, false) to represent the absence of an aggregator.
| | * Document the Dependency classes.Josh Rosen2012-10-072-1/+26
| | |
| | * Remove unused isShuffle field from Dependency.Josh Rosen2012-10-071-3/+3
| | |
| * | Changed the println to logInfo in Utils.fetchFile.Reynold Xin2012-10-071-1/+1
| | |
| * | Merge pull request #250 from rxin/devMatei Zaharia2012-10-072-18/+40
| |\ \ | | | | | | | | Fixed a bug in addFile that if the file is specified as "file:///", the symlink is created incorrectly for local mode.
| | * | Fixed a bug in addFile that if the file is specified as "file:///", theReynold Xin2012-10-072-18/+40
| | | | | | | | | | | | | | | | symlink is created wrong for local mode.
| * | | Improve error messageMatei Zaharia2012-10-071-1/+1
| | | |
| * | | Don't crash on ask timeout exceptions in deploy.Client.stop() (fixes a crash ↵Matei Zaharia2012-10-071-3/+8
| |/ / | | | | | | | | | in tests)
| * / Removed the need to sleep in tests due to waiting for Akka to shut downMatei Zaharia2012-10-0715-23/+40
| |/
| * Log messageMatei Zaharia2012-10-071-1/+1
| |
| * More loggingMatei Zaharia2012-10-071-4/+7
| |
| * Log more info in MapOutputTrackerroot2012-10-071-4/+7
| |
| * Made Akka thread pool and message batch sizes configurableroot2012-10-071-3/+5
| |
| * Made run script add test-classes onto the classpath only if SPARK_TESTING is ↵root2012-10-073-3/+6
| | | | | | | | set; fixes #216
| * Avoid acquiring locks in BlockManager when fetching shuffle outputsroot2012-10-071-0/+24
| |
| * Log initial number of fetches in reducerroot2012-10-071-1/+2
| |
| * Modified shuffle to limit the maximum outstanding data size in bytes,Matei Zaharia2012-10-064-61/+109
| | | | | | | | | | | | instead of the maximum number of outstanding fetches. This should make it faster when there are many small map output files, as well as more robust to overallocating memory on large map outputs.
| * Pass sizes of map outputs back to MapOutputTrackerMatei Zaharia2012-10-069-83/+152
| |
| * Made block store return the size of each block put inMatei Zaharia2012-10-066-36/+70
| |
| * Warn about user programs that try to set spark.cache.classMatei Zaharia2012-10-061-1/+7
| |
| * Only group elements ten at a time into SequenceFile records inMatei Zaharia2012-10-061-2/+2
| | | | | | | | saveAsObjectFile
| * Minor formatting fixesMatei Zaharia2012-10-053-3/+2
| |
| * Merge branch 'dev' of github.com:mesos/spark into devMatei Zaharia2012-10-0525-99/+228
| |\
| | * Factor subclasses of RDD out of RDD.scala into their own classesAndy Konwinski2012-10-058-66/+106
| | | | | | | | | | | | in the rdd package.
| | * Moves all files in core/src/main/scala/ that have RDD in their nameAndy Konwinski2012-10-0513-0/+0
| | | | | | | | | | | | from that directory to a new core/src/main/scala/rdd directory.
| | * Moves all files in core/src/main/scala/ that have RDD in them fromAndy Konwinski2012-10-0518-33/+122
| | | | | | | | | | | | package spark to package spark.rdd and updates all references to them.
generated by cgit v1.2.3 (git 2.39.1) at 2024-09-09 14:26:28 +0000