Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | Rename environment variable for hadoop profiles to hadoopVersion | Shivaram Venkataraman | 2013-01-12 | 6 | -12/+13 |
| | |||||
* | Activate hadoop2 profile in pom.xml with -Dhadoop=2 | Shivaram Venkataraman | 2013-01-10 | 6 | -0/+36 |
| | |||||
* | Activate hadoop1 if property hadoop is missing. hadoop2 can be activated now | Shivaram Venkataraman | 2013-01-08 | 6 | -6/+18 |
| | | | | by using -Dhadoop -Phadoop2. | ||||
* | Activate hadoop1 profile by default for maven builds | Shivaram Venkataraman | 2013-01-07 | 6 | -1/+19 |
| | |||||
* | Update expected size of strings to match our dummy string class | Shivaram Venkataraman | 2013-01-07 | 1 | -18/+13 |
| | |||||
* | Make default hadoop version 1.0.3 in pom.xml | Shivaram Venkataraman | 2013-01-07 | 1 | -1/+1 |
| | |||||
* | Use a dummy string class in Size Estimator tests to make it resistant to jdk | Shivaram Venkataraman | 2013-01-07 | 1 | -12/+21 |
| | | | | versions | ||||
* | Remove unused BoundedMemoryCache file and associated test case. | Shivaram Venkataraman | 2013-01-07 | 2 | -176/+0 |
| | |||||
* | Update Hadoop dependency to 1.0.3 as 0.20 has Sun specific dependencies. Also | Shivaram Venkataraman | 2013-01-07 | 2 | -2/+8 |
| | | | | | fix SequenceFileRDDFunctions to pick the right type conversion across Hadoop versions | ||||
* | Remove dependencies on sun jvm classes. Instead use reflection to infer | Shivaram Venkataraman | 2013-01-07 | 2 | -6/+29 |
| | | | | HotSpot options and total physical memory size | ||||
* | Merge branch 'master' of github.com:mesos/spark | Matei Zaharia | 2013-01-07 | 2 | -0/+17 |
|\ | |||||
| * | Merge pull request #351 from stephenh/values | Matei Zaharia | 2013-01-05 | 2 | -0/+17 |
| |\ | | | | | | | Add PairRDDFunctions.keys and values. | ||||
| | * | Fix typo. | Stephen Haberman | 2013-01-05 | 1 | -1/+1 |
| | | | |||||
| | * | Add PairRDDFunctions.keys and values. | Stephen Haberman | 2013-01-05 | 2 | -0/+17 |
| | | | |||||
* | | | Add Accumulable.setValue for easier use in Java | Matei Zaharia | 2013-01-07 | 2 | -5/+19 |
| | | | |||||
* | | | Add PairRDD.keys() and values() to Java API | Matei Zaharia | 2013-01-05 | 1 | -0/+10 |
|/ / | |||||
* | | Add methods called just 'accumulator' for int/double in Java API | Matei Zaharia | 2013-01-05 | 2 | -2/+15 |
| | | |||||
* | | Fix Accumulators in Java, and add a test for them | Matei Zaharia | 2013-01-05 | 4 | -13/+79 |
| | | |||||
* | | Merge pull request #352 from stephenh/collect | Matei Zaharia | 2013-01-05 | 2 | -0/+8 |
|\ \ | |/ |/| | Add RDD.collect(PartialFunction). | ||||
| * | Add RDD.collect(PartialFunction). | Stephen Haberman | 2013-01-05 | 2 | -0/+8 |
|/ | |||||
* | Merge pull request #349 from woggling/cache-finally | Matei Zaharia | 2013-01-01 | 2 | -11/+28 |
|\ | | | | | Avoid stalls when computation of cached RDD throws exception | ||||
| * | Remove some dead comments | Charles Reiss | 2013-01-01 | 1 | -6/+0 |
| | | |||||
| * | Test with exception while computing cached RDD. | Charles Reiss | 2013-01-01 | 1 | -0/+23 |
| | | |||||
| * | Mark key as not loading in CacheTracker even when compute() fails | Charles Reiss | 2013-01-01 | 1 | -10/+10 |
| | | |||||
* | | Merge pull request #348 from JoshRosen/spark-597 | Matei Zaharia | 2013-01-01 | 4 | -0/+58 |
|\ \ | |/ |/| | Raise exception when hashing Java arrays (SPARK-597) | ||||
| * | Raise exception when hashing Java arrays (SPARK-597) | Josh Rosen | 2012-12-31 | 4 | -0/+58 |
|/ | |||||
* | Merge pull request #345 from JoshRosen/fix/add-file | Matei Zaharia | 2012-12-29 | 4 | -36/+101 |
|\ | | | | | Fix deletion of files in current working directory by clearFiles() | ||||
| * | Change Utils.fetchFile() warning to SparkException. | Josh Rosen | 2012-12-28 | 2 | -7/+10 |
| | | |||||
| * | Add addFile() and addJar() to JavaSparkContext. | Josh Rosen | 2012-12-28 | 1 | -0/+34 |
| | | |||||
| * | Add synchronization to LocalScheduler.updateDependencies(). | Josh Rosen | 2012-12-28 | 1 | -16/+18 |
| | | |||||
| * | Skip deletion of files in clearFiles(). | Josh Rosen | 2012-12-28 | 2 | -20/+46 |
|/ | | | | | | | | | | | | | | | | This fixes an issue where Spark could delete original files in the current working directory that were added to the job using addFile(). There was also the potential for addFile() to overwrite local files, which is addressed by changing Utils.fetchFile() to log a warning instead of overwriting a file with new contents. This is a short-term fix; a better long-term solution would be to remove the dependence on storing files in the current working directory, since we can't change the cwd from Java. | ||||
* | Merge pull request #343 from markhamstra/spark-601 | Matei Zaharia | 2012-12-24 | 2 | -1/+12 |
|\ | | | | | lookup() needn't fail when there is no partitioner | ||||
| * | fall back to filter-map-collect when calling lookup() on an RDD without a ↵ | Mark Hamstra | 2012-12-24 | 2 | -1/+12 |
|/ | | | | partitioner | ||||
* | Merge pull request #342 from markhamstra/spark-645 | Matei Zaharia | 2012-12-24 | 2 | -5/+11 |
|\ | | | | | Allow distinct() to be called without parentheses | ||||
| * | Allow distinct() to be called without parentheses when using the default ↵ | Mark Hamstra | 2012-12-24 | 2 | -5/+11 |
|/ | | | | number of splits. | ||||
* | Updated Kryo version for Maven pom file. | Reynold Xin | 2012-12-21 | 1 | -1/+1 |
| | |||||
* | Updated Kryo documentation for Kryo version update. | Reynold Xin | 2012-12-21 | 1 | -14/+16 |
| | |||||
* | Remove the call to close input stream in Kryo serializer. | Reynold Xin | 2012-12-21 | 1 | -1/+1 |
| | |||||
* | Merge pull request #341 from rxin/4a3fb06ac2d11125feb08acbbd4df76d1e91b677 | Matei Zaharia | 2012-12-21 | 2 | -138/+69 |
|\ | | | | | Kryo2 update against Spark master | ||||
| * | Updated Kryo to 2.20. | Reynold Xin | 2012-10-16 | 2 | -138/+69 |
| | | |||||
* | | Merge pull request #327 from rxin/spark-633 | Matei Zaharia | 2012-12-20 | 17 | -770/+1097 |
|\ \ | | | | | | | Added the ability in block manager to remove blocks. | ||||
| * | | Let the slave notify the master block removal. | Reynold Xin | 2012-12-20 | 5 | -93/+84 |
| | | | |||||
| * | | Moved BlockManager's IdGenerator into BlockManager object. Removed some | Reynold Xin | 2012-12-19 | 5 | -36/+22 |
| | | | | | | | | | | | | excessive debug messages. | ||||
| * | | Merge branch 'spark-633' of github.com:rxin/spark into spark-633 | Reynold Xin | 2012-12-14 | 4 | -7/+23 |
| |\ \ | |||||
| | * | | Return success or failure in BlockStore.remove(). | Josh Rosen | 2012-12-13 | 4 | -7/+23 |
| | | | | |||||
| * | | | Fixed conflicts from merging Charles' and TD's block manager changes. | Reynold Xin | 2012-12-14 | 3 | -173/+158 |
| | | | | |||||
| * | | | Merge branch 'master' of github.com:mesos/spark into spark-633 | Reynold Xin | 2012-12-13 | 1 | -7/+9 |
| |\ \ \ | |||||
| * | | | | Merged TD's block manager refactoring. | Reynold Xin | 2012-12-13 | 9 | -647/+805 |
| | | | | | |||||
| * | | | | Merge branch 'master' of github.com:mesos/spark into spark-633 | Reynold Xin | 2012-12-13 | 12 | -21/+138 |
| |\ \ \ \ | |||||
| * \ \ \ \ | Merge branch 'master' of github.com:mesos/spark into spark-633 | Reynold Xin | 2012-12-13 | 29 | -231/+268 |
| |\ \ \ \ \ | | |_|_|/ / | |/| | | | |