Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | Support for Hadoop 2 distributions such as cdh4 | Thomas Dudziak | 2012-10-18 | 7 | -20/+45 |
| | |||||
* | Made ShuffleDependency automatically find a shuffle ID for itself | Matei Zaharia | 2012-10-14 | 3 | -5/+6 |
| | |||||
* | Take executor environment vars as an arguemnt to SparkContext | Matei Zaharia | 2012-10-13 | 7 | -79/+107 |
| | |||||
* | Protect from null env variables in mesos. | Denny | 2012-10-13 | 2 | -8/+16 |
| | |||||
* | Protect from setting null environment variables. | Denny | 2012-10-13 | 1 | -1/+5 |
| | |||||
* | Don't use system envs for Mesos. | Denny | 2012-10-13 | 2 | -2/+2 |
| | |||||
* | Let the user specify environment variables to be passed to the Executors. | Denny | 2012-10-13 | 5 | -48/+21 |
| | | | | Also removed unused variables in the ExecutorRunner. | ||||
* | More doc updates, and moved Serializer to a subpackage. | Matei Zaharia | 2012-10-12 | 12 | -25/+51 |
| | |||||
* | Some doc and usability improvements: | Matei Zaharia | 2012-10-12 | 11 | -22/+83 |
| | | | | | | | - Added a StorageLevels class for easy access to StorageLevel constants in Java - Added doc comments on Function classes in Java - Updated Accumulator and HadoopWriter docs slightly | ||||
* | Added a test for when an RDD only partially fits in memory | Matei Zaharia | 2012-10-12 | 1 | -2/+18 |
| | |||||
* | Document cartesian() operation | Matei Zaharia | 2012-10-12 | 2 | -0/+8 |
| | |||||
* | Merge pull request #271 from shivaram/block-manager-npe-fix | Matei Zaharia | 2012-10-12 | 8 | -39/+58 |
|\ | | | | | Change block manager to accept a ArrayBuffer | ||||
| * | Add test to verify if RDD is computed even if block manager has insufficient | Shivaram Venkataraman | 2012-10-12 | 1 | -0/+10 |
| | | | | | | | | memory | ||||
| * | Change block manager to accept a ArrayBuffer instead of an iterator to ensure | Shivaram Venkataraman | 2012-10-11 | 7 | -39/+48 |
| | | | | | | | | | | that the computation can proceed even if we run out of memory to cache the block. Update CacheTracker to use this new interface | ||||
* | | Adding Java documentation | Patrick Wendell | 2012-10-11 | 6 | -10/+454 |
|/ | |||||
* | Fixed bug when fetching Jar dependencies. | Denny | 2012-10-10 | 2 | -6/+6 |
| | | | | Instead of checking currentFiles check currentJars. | ||||
* | Added documentation to all the *RDDFunction classes, and moved them into | Matei Zaharia | 2012-10-09 | 9 | -54/+273 |
| | | | | | the spark package to make them more visible. Also documented various other miscellaneous things in the API. | ||||
* | Updates to documentation: | Matei Zaharia | 2012-10-09 | 1 | -1/+1 |
| | | | | | | | | - Edited quick start and tuning guide to simplify them a little - Simplified top menu bar - Made private a SparkContext constructor parameter that was left as public - Various small fixes | ||||
* | Fixes a typo, adds scaladoc comments to SparkContext constructors. | Andy Konwinski | 2012-10-08 | 2 | -5/+11 |
| | |||||
* | More docs in RDD class | Patrick Wendell | 2012-10-08 | 1 | -1/+45 |
| | |||||
* | A start on scaladoc for the public APIs. | Andy Konwinski | 2012-10-08 | 1 | -6/+29 |
| | |||||
* | Merge branch 'dev' into bc-fix-dev | Mosharaf Chowdhury | 2012-10-08 | 71 | -477/+1108 |
|\ | |||||
| * | Made compression configurable separately for shuffle, broadcast and RDDs | Matei Zaharia | 2012-10-07 | 4 | -38/+118 |
| | | |||||
| * | Merge pull request #251 from JoshRosen/docs/internals | Matei Zaharia | 2012-10-07 | 5 | -11/+40 |
| |\ | | | | | | | Document Dependency classes and make minor interface improvements | ||||
| | * | Make ShuffleDependency.aggregator explicitly optional. | Josh Rosen | 2012-10-07 | 4 | -7/+11 |
| | | | | | | | | | | | | | | | | | | | | | | | | It was confusing to be using new Aggregator[K, V, V](null, null, null, false) to represent the absence of an aggregator. | ||||
| | * | Document the Dependency classes. | Josh Rosen | 2012-10-07 | 2 | -1/+26 |
| | | | |||||
| | * | Remove unused isShuffle field from Dependency. | Josh Rosen | 2012-10-07 | 1 | -3/+3 |
| | | | |||||
| * | | Changed the println to logInfo in Utils.fetchFile. | Reynold Xin | 2012-10-07 | 1 | -1/+1 |
| | | | |||||
| * | | Merge pull request #250 from rxin/dev | Matei Zaharia | 2012-10-07 | 2 | -18/+40 |
| |\ \ | | | | | | | | | Fixed a bug in addFile that if the file is specified as "file:///", the symlink is created incorrectly for local mode. | ||||
| | * | | Fixed a bug in addFile that if the file is specified as "file:///", the | Reynold Xin | 2012-10-07 | 2 | -18/+40 |
| | | | | | | | | | | | | | | | | symlink is created wrong for local mode. | ||||
| * | | | Improve error message | Matei Zaharia | 2012-10-07 | 1 | -1/+1 |
| | | | | |||||
| * | | | Don't crash on ask timeout exceptions in deploy.Client.stop() (fixes a crash ↵ | Matei Zaharia | 2012-10-07 | 1 | -3/+8 |
| |/ / | | | | | | | | | | in tests) | ||||
| * / | Removed the need to sleep in tests due to waiting for Akka to shut down | Matei Zaharia | 2012-10-07 | 15 | -23/+40 |
| |/ | |||||
| * | Log message | Matei Zaharia | 2012-10-07 | 1 | -1/+1 |
| | | |||||
| * | More logging | Matei Zaharia | 2012-10-07 | 1 | -4/+7 |
| | | |||||
| * | Log more info in MapOutputTracker | root | 2012-10-07 | 1 | -4/+7 |
| | | |||||
| * | Made Akka thread pool and message batch sizes configurable | root | 2012-10-07 | 1 | -3/+5 |
| | | |||||
| * | Made run script add test-classes onto the classpath only if SPARK_TESTING is ↵ | root | 2012-10-07 | 3 | -3/+6 |
| | | | | | | | | set; fixes #216 | ||||
| * | Avoid acquiring locks in BlockManager when fetching shuffle outputs | root | 2012-10-07 | 1 | -0/+24 |
| | | |||||
| * | Log initial number of fetches in reducer | root | 2012-10-07 | 1 | -1/+2 |
| | | |||||
| * | Modified shuffle to limit the maximum outstanding data size in bytes, | Matei Zaharia | 2012-10-06 | 4 | -61/+109 |
| | | | | | | | | | | | | instead of the maximum number of outstanding fetches. This should make it faster when there are many small map output files, as well as more robust to overallocating memory on large map outputs. | ||||
| * | Pass sizes of map outputs back to MapOutputTracker | Matei Zaharia | 2012-10-06 | 9 | -83/+152 |
| | | |||||
| * | Made block store return the size of each block put in | Matei Zaharia | 2012-10-06 | 6 | -36/+70 |
| | | |||||
| * | Warn about user programs that try to set spark.cache.class | Matei Zaharia | 2012-10-06 | 1 | -1/+7 |
| | | |||||
| * | Only group elements ten at a time into SequenceFile records in | Matei Zaharia | 2012-10-06 | 1 | -2/+2 |
| | | | | | | | | saveAsObjectFile | ||||
| * | Minor formatting fixes | Matei Zaharia | 2012-10-05 | 3 | -3/+2 |
| | | |||||
| * | Merge branch 'dev' of github.com:mesos/spark into dev | Matei Zaharia | 2012-10-05 | 25 | -99/+228 |
| |\ | |||||
| | * | Factor subclasses of RDD out of RDD.scala into their own classes | Andy Konwinski | 2012-10-05 | 8 | -66/+106 |
| | | | | | | | | | | | | in the rdd package. | ||||
| | * | Moves all files in core/src/main/scala/ that have RDD in their name | Andy Konwinski | 2012-10-05 | 13 | -0/+0 |
| | | | | | | | | | | | | from that directory to a new core/src/main/scala/rdd directory. | ||||
| | * | Moves all files in core/src/main/scala/ that have RDD in them from | Andy Konwinski | 2012-10-05 | 18 | -33/+122 |
| | | | | | | | | | | | | package spark to package spark.rdd and updates all references to them. |