Commit message (Collapse) | Author | Age | Files | Lines | ||
---|---|---|---|---|---|---|
... | ||||||
* | | | | | | Automatically configure Netty port. This makes unit tests using | Reynold Xin | 2013-05-24 | 10 | -112/+129 | |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | local-cluster pass. Previously they were failing because Netty was trying to bind to the same port for all processes. Pair programmed with @shivaram. | |||||
* | | | | | | Fixed the bug that shuffle serializer is ignored by the new shuffle | Reynold Xin | 2013-05-24 | 2 | -4/+19 | |
| |_|/ / / |/| | | | | | | | | | | | | | | block iterators for local blocks. Also added a unit test for that. | |||||
* | | | | | Merge pull request #619 from woggling/adjust-sampling | Matei Zaharia | 2013-05-21 | 1 | -1/+1 | |
|\ \ \ \ \ | | | | | | | | | | | | | Use ARRAY_SAMPLE_SIZE constant instead of hard-coded 100.0 in SizeEstimator | |||||
| * | | | | | Use ARRAY_SAMPLE_SIZE constant instead of 100.0 | Charles Reiss | 2013-05-21 | 1 | -1/+1 | |
| | | | | | | ||||||
* | | | | | | DistributedSuite: remove dead test code | Charles Reiss | 2013-05-21 | 1 | -1/+0 | |
|/ / / / / | ||||||
* | | | | | Merge pull request #610 from JoshRosen/spark-747 | Reynold Xin | 2013-05-17 | 4 | -0/+26 | |
|\ \ \ \ \ | | | | | | | | | | | | | Throw exception if TaskResult exceeds Akka frame size | |||||
| * | | | | | Abort job if result exceeds Akka frame size; add test. | Josh Rosen | 2013-05-16 | 4 | -1/+23 | |
| | | | | | | ||||||
| * | | | | | Throw exception if task result exceeds Akka frame size. | Josh Rosen | 2013-05-11 | 1 | -0/+4 | |
| | |_|_|/ | |/| | | | | | | | | | | | | | This partially addresses SPARK-747. | |||||
* | | | | | Changed the logging level from info to warning when addJar(null) is | Reynold Xin | 2013-05-17 | 1 | -9/+10 | |
| | | | | | | | | | | | | | | | | | | | | called. | |||||
* | | | | | Merge branch 'master' of github.com:mesos/spark | Reynold Xin | 2013-05-17 | 1 | -0/+4 | |
|\ \ \ \ \ | | |_|/ / | |/| | | | ||||||
| * | | | | Added dependency on netty-all in Maven. | Reynold Xin | 2013-05-16 | 1 | -0/+4 | |
| | | | | | ||||||
* | | | | | Hope this does not turn into a bike shed change | Mridul Muralidharan | 2013-05-17 | 1 | -2/+4 | |
| | | | | | ||||||
* | | | | | Filter out nulls - prevent NPE | Mridul Muralidharan | 2013-05-16 | 1 | -0/+2 | |
|/ / / / | ||||||
* | | | | Merge pull request #602 from rxin/shufflemerge | Matei Zaharia | 2013-05-15 | 23 | -440/+1258 | |
|\ \ \ \ | | | | | | | | | | | Manual merge & cleanup of Shane's Shuffle Performance Optimization | |||||
| * \ \ \ | Merge branch 'master' of github.com:mesos/spark into shufflemerge | Reynold Xin | 2013-05-15 | 28 | -178/+585 | |
| |\ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/spark/storage/BlockManager.scala core/src/test/scala/spark/DistributedSuite.scala project/SparkBuild.scala | |||||
| * | | | | | Added test for Netty suite. | Reynold Xin | 2013-05-07 | 2 | -1/+19 | |
| | | | | | | ||||||
| * | | | | | Cleaned up connection manager (moved many classes to their own files). | Reynold Xin | 2013-05-07 | 7 | -252/+315 | |
| | | | | | | ||||||
| * | | | | | Minor style fix in DiskStore... | Reynold Xin | 2013-05-07 | 1 | -2/+2 | |
| | | | | | | ||||||
| * | | | | | Cleaned up Scala files in network/netty from Shane's PR. | Reynold Xin | 2013-05-07 | 2 | -33/+35 | |
| | | | | | | ||||||
| * | | | | | Cleaned up the Java files from Shane's PR. | Reynold Xin | 2013-05-07 | 8 | -118/+85 | |
| | | | | | | ||||||
| * | | | | | Cleaned up BlockManager and BlockFetcherIterator from Shane's PR. | Reynold Xin | 2013-05-07 | 3 | -99/+102 | |
| | | | | | | ||||||
| * | | | | | Moved BlockFetcherIterator to its own file. | Reynold Xin | 2013-05-07 | 2 | -339/+362 | |
| | | | | | | ||||||
| * | | | | | Merge branch 'shuffle-performance-fix-0.7' of github.com:shane-huang/spark ↵ | Reynold Xin | 2013-05-07 | 12 | -68/+808 | |
| |\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | into shufflemerge Conflicts: core/src/main/scala/spark/storage/BlockManager.scala core/src/main/scala/spark/storage/DiskStore.scala project/SparkBuild.scala | |||||
| | * | | | | | fix a bug in netty Block Fetcher | shane-huang | 2013-04-16 | 3 | -37/+37 | |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: shane-huang <shengsheng.huang@intel.com> | |||||
| | * | | | | | Shuffle Performance fix: Use netty embeded OIO file server instead of ↵ | shane-huang | 2013-04-07 | 12 | -54/+792 | |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ConnectionManager Shuffle Performance Optimization: do not send 0-byte block requests to reduce network messages change reference from io.Source to scala.io.Source to avoid looking into io.netty package Signed-off-by: shane-huang <shengsheng.huang@intel.com> | |||||
* | | | | | | | Merge pull request #593 from squito/driver_ui_link | Reynold Xin | 2013-05-15 | 7 | -13/+21 | |
|\ \ \ \ \ \ \ | |_|_|/ / / / |/| | | | | | | Master UI has link to Application UI | |||||
| * | | | | | | fix linebreak | Imran Rashid | 2013-05-09 | 1 | -2/+2 | |
| | | | | | | | ||||||
| * | | | | | | switch from separating appUI host & port to combining into just appUiUrl | Imran Rashid | 2013-05-05 | 7 | -11/+8 | |
| | | | | | | | ||||||
| * | | | | | | applications (aka drivers) send their webUI address to master when ↵ | Imran Rashid | 2013-05-03 | 7 | -13/+24 | |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | registering so it can be displayed in the master web ui | |||||
* | | | | | | | Added a comment in JdbcRDD for example usage. | Reynold Xin | 2013-05-14 | 1 | -1/+3 | |
| | | | | | | | ||||||
* | | | | | | | Added derby dependency to Maven pom files for the JDBC Java test. | Reynold Xin | 2013-05-14 | 1 | -0/+5 | |
| | | | | | | | ||||||
* | | | | | | | Merge branch 'jdbc' of github.com:koeninger/spark | Reynold Xin | 2013-05-14 | 2 | -0/+157 | |
|\ \ \ \ \ \ \ | |_|_|_|_|_|/ |/| | | | | | | | | | | | | | | | | | | | | Conflicts: project/SparkBuild.scala | |||||
| * | | | | | | add test for JdbcRDD using embedded derby, per rxin suggestion | Cody Koeninger | 2013-05-14 | 1 | -0/+56 | |
| | | | | | | | ||||||
| * | | | | | | code cleanup per rxin comments | Cody Koeninger | 2013-05-11 | 1 | -30/+37 | |
| | | | | | | | ||||||
| * | | | | | | prevent mysql driver from pulling entire resultset into memory. explicitly ↵ | koeninger | 2013-04-22 | 1 | -5/+20 | |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | close resultset and statement. | |||||
| * | | | | | | first attempt at an RDD to pull data from JDBC sources | koeninger | 2013-04-21 | 1 | -0/+79 | |
| | | | | | | | ||||||
* | | | | | | | Merge pull request #601 from rxin/emptyrdd-master | Matei Zaharia | 2013-05-13 | 2 | -1/+37 | |
|\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | EmptyRDD (master branch 0.8) | |||||
| * | | | | | | | Added tests for joins, cogroups, and unions for EmptyRDD. | Reynold Xin | 2013-05-06 | 1 | -0/+8 | |
| | | | | | | | | ||||||
| * | | | | | | | Added EmptyRDD. | Reynold Xin | 2013-05-06 | 2 | -1/+29 | |
| | | | | | | | | ||||||
* | | | | | | | | Merge pull request #589 from mridulm/master | Matei Zaharia | 2013-05-13 | 19 | -166/+339 | |
|\ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | Add support for instance local scheduling | |||||
| * | | | | | | | | Remove explicit hardcoding of yarn-standalone as args(0) if it is missing. | Mridul Muralidharan | 2013-05-09 | 3 | -18/+3 | |
| | | | | | | | | | ||||||
| * | | | | | | | | Merge branch 'master' of github.com:mridulm/spark | Mridul Muralidharan | 2013-05-04 | 2 | -10/+38 | |
| |\ \ \ \ \ \ \ \ | ||||||
| | * | | | | | | | | Add support for instance local in getPreferredLocations of ↵ | Mridul Muralidharan | 2013-05-04 | 2 | -10/+38 | |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ZippedPartitionsBaseRDD. Add comments to both ZippedPartitionsBaseRDD and ZippedRDD to better describe the potential problem with the approach | |||||
| * | | | | | | | | | Merge from master | Mridul Muralidharan | 2013-05-04 | 20 | -102/+452 | |
| |\ \ \ \ \ \ \ \ \ | | |/ / / / / / / / | |/| | | | | | | | | ||||||
| * | | | | | | | | | pull from master | Mridul Muralidharan | 2013-05-04 | 17 | -110/+369 | |
| |\ \ \ \ \ \ \ \ \ | | | |_|_|/ / / / / | | |/| | | | | | | | ||||||
| * | | | | | | | | | Fix ZippedRDD as part Matei's suggestion | Mridul Muralidharan | 2013-05-03 | 1 | -4/+15 | |
| | | | | | | | | | | ||||||
| * | | | | | | | | | comment out debug versions of checkHost, etc from Utils - which were used to ↵ | Mridul Muralidharan | 2013-05-02 | 1 | -2/+2 | |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | test | |||||
| * | | | | | | | | | Integrate review comments 2 | Mridul Muralidharan | 2013-05-02 | 3 | -77/+77 | |
| | | | | | | | | | | ||||||
| * | | | | | | | | | Integrate review comments on pull request | Mridul Muralidharan | 2013-05-02 | 6 | -50/+47 | |
| | | | | | | | | | | ||||||
| * | | | | | | | | | Fix some npe introduced accidentally | Mridul Muralidharan | 2013-05-01 | 2 | -9/+23 | |
| | | | | | | | | | |