aboutsummaryrefslogtreecommitdiff
path: root/core
Commit message (Collapse)AuthorAgeFilesLines
...
* | | | | | Automatically configure Netty port. This makes unit tests usingReynold Xin2013-05-2410-112/+129
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | local-cluster pass. Previously they were failing because Netty was trying to bind to the same port for all processes. Pair programmed with @shivaram.
* | | | | | Fixed the bug that shuffle serializer is ignored by the new shuffleReynold Xin2013-05-242-4/+19
| |_|/ / / |/| | | | | | | | | | | | | | block iterators for local blocks. Also added a unit test for that.
* | | | | Merge pull request #619 from woggling/adjust-samplingMatei Zaharia2013-05-211-1/+1
|\ \ \ \ \ | | | | | | | | | | | | Use ARRAY_SAMPLE_SIZE constant instead of hard-coded 100.0 in SizeEstimator
| * | | | | Use ARRAY_SAMPLE_SIZE constant instead of 100.0Charles Reiss2013-05-211-1/+1
| | | | | |
* | | | | | DistributedSuite: remove dead test codeCharles Reiss2013-05-211-1/+0
|/ / / / /
* | | | | Merge pull request #610 from JoshRosen/spark-747Reynold Xin2013-05-174-0/+26
|\ \ \ \ \ | | | | | | | | | | | | Throw exception if TaskResult exceeds Akka frame size
| * | | | | Abort job if result exceeds Akka frame size; add test.Josh Rosen2013-05-164-1/+23
| | | | | |
| * | | | | Throw exception if task result exceeds Akka frame size.Josh Rosen2013-05-111-0/+4
| | |_|_|/ | |/| | | | | | | | | | | | | This partially addresses SPARK-747.
* | | | | Changed the logging level from info to warning when addJar(null) isReynold Xin2013-05-171-9/+10
| | | | | | | | | | | | | | | | | | | | called.
* | | | | Merge branch 'master' of github.com:mesos/sparkReynold Xin2013-05-171-0/+4
|\ \ \ \ \ | | |_|/ / | |/| | |
| * | | | Added dependency on netty-all in Maven.Reynold Xin2013-05-161-0/+4
| | | | |
* | | | | Hope this does not turn into a bike shed changeMridul Muralidharan2013-05-171-2/+4
| | | | |
* | | | | Filter out nulls - prevent NPEMridul Muralidharan2013-05-161-0/+2
|/ / / /
* | | | Merge pull request #602 from rxin/shufflemergeMatei Zaharia2013-05-1523-440/+1258
|\ \ \ \ | | | | | | | | | | Manual merge & cleanup of Shane's Shuffle Performance Optimization
| * \ \ \ Merge branch 'master' of github.com:mesos/spark into shufflemergeReynold Xin2013-05-1528-178/+585
| |\ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/spark/storage/BlockManager.scala core/src/test/scala/spark/DistributedSuite.scala project/SparkBuild.scala
| * | | | | Added test for Netty suite.Reynold Xin2013-05-072-1/+19
| | | | | |
| * | | | | Cleaned up connection manager (moved many classes to their own files).Reynold Xin2013-05-077-252/+315
| | | | | |
| * | | | | Minor style fix in DiskStore...Reynold Xin2013-05-071-2/+2
| | | | | |
| * | | | | Cleaned up Scala files in network/netty from Shane's PR.Reynold Xin2013-05-072-33/+35
| | | | | |
| * | | | | Cleaned up the Java files from Shane's PR.Reynold Xin2013-05-078-118/+85
| | | | | |
| * | | | | Cleaned up BlockManager and BlockFetcherIterator from Shane's PR.Reynold Xin2013-05-073-99/+102
| | | | | |
| * | | | | Moved BlockFetcherIterator to its own file.Reynold Xin2013-05-072-339/+362
| | | | | |
| * | | | | Merge branch 'shuffle-performance-fix-0.7' of github.com:shane-huang/spark ↵Reynold Xin2013-05-0712-68/+808
| |\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | into shufflemerge Conflicts: core/src/main/scala/spark/storage/BlockManager.scala core/src/main/scala/spark/storage/DiskStore.scala project/SparkBuild.scala
| | * | | | | fix a bug in netty Block Fetchershane-huang2013-04-163-37/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: shane-huang <shengsheng.huang@intel.com>
| | * | | | | Shuffle Performance fix: Use netty embeded OIO file server instead of ↵shane-huang2013-04-0712-54/+792
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ConnectionManager Shuffle Performance Optimization: do not send 0-byte block requests to reduce network messages change reference from io.Source to scala.io.Source to avoid looking into io.netty package Signed-off-by: shane-huang <shengsheng.huang@intel.com>
* | | | | | | Merge pull request #593 from squito/driver_ui_linkReynold Xin2013-05-157-13/+21
|\ \ \ \ \ \ \ | |_|_|/ / / / |/| | | | | | Master UI has link to Application UI
| * | | | | | fix linebreakImran Rashid2013-05-091-2/+2
| | | | | | |
| * | | | | | switch from separating appUI host & port to combining into just appUiUrlImran Rashid2013-05-057-11/+8
| | | | | | |
| * | | | | | applications (aka drivers) send their webUI address to master when ↵Imran Rashid2013-05-037-13/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | registering so it can be displayed in the master web ui
* | | | | | | Added a comment in JdbcRDD for example usage.Reynold Xin2013-05-141-1/+3
| | | | | | |
* | | | | | | Added derby dependency to Maven pom files for the JDBC Java test.Reynold Xin2013-05-141-0/+5
| | | | | | |
* | | | | | | Merge branch 'jdbc' of github.com:koeninger/sparkReynold Xin2013-05-142-0/+157
|\ \ \ \ \ \ \ | |_|_|_|_|_|/ |/| | | | | | | | | | | | | | | | | | | | Conflicts: project/SparkBuild.scala
| * | | | | | add test for JdbcRDD using embedded derby, per rxin suggestionCody Koeninger2013-05-141-0/+56
| | | | | | |
| * | | | | | code cleanup per rxin commentsCody Koeninger2013-05-111-30/+37
| | | | | | |
| * | | | | | prevent mysql driver from pulling entire resultset into memory. explicitly ↵koeninger2013-04-221-5/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | close resultset and statement.
| * | | | | | first attempt at an RDD to pull data from JDBC sourceskoeninger2013-04-211-0/+79
| | | | | | |
* | | | | | | Merge pull request #601 from rxin/emptyrdd-masterMatei Zaharia2013-05-132-1/+37
|\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | EmptyRDD (master branch 0.8)
| * | | | | | | Added tests for joins, cogroups, and unions for EmptyRDD.Reynold Xin2013-05-061-0/+8
| | | | | | | |
| * | | | | | | Added EmptyRDD.Reynold Xin2013-05-062-1/+29
| | | | | | | |
* | | | | | | | Merge pull request #589 from mridulm/masterMatei Zaharia2013-05-1319-166/+339
|\ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | Add support for instance local scheduling
| * | | | | | | | Remove explicit hardcoding of yarn-standalone as args(0) if it is missing.Mridul Muralidharan2013-05-093-18/+3
| | | | | | | | |
| * | | | | | | | Merge branch 'master' of github.com:mridulm/sparkMridul Muralidharan2013-05-042-10/+38
| |\ \ \ \ \ \ \ \
| | * | | | | | | | Add support for instance local in getPreferredLocations of ↵Mridul Muralidharan2013-05-042-10/+38
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ZippedPartitionsBaseRDD. Add comments to both ZippedPartitionsBaseRDD and ZippedRDD to better describe the potential problem with the approach
| * | | | | | | | | Merge from masterMridul Muralidharan2013-05-0420-102/+452
| |\ \ \ \ \ \ \ \ \ | | |/ / / / / / / / | |/| | | | | | | |
| * | | | | | | | | pull from masterMridul Muralidharan2013-05-0417-110/+369
| |\ \ \ \ \ \ \ \ \ | | | |_|_|/ / / / / | | |/| | | | | | |
| * | | | | | | | | Fix ZippedRDD as part Matei's suggestionMridul Muralidharan2013-05-031-4/+15
| | | | | | | | | |
| * | | | | | | | | comment out debug versions of checkHost, etc from Utils - which were used to ↵Mridul Muralidharan2013-05-021-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | test
| * | | | | | | | | Integrate review comments 2Mridul Muralidharan2013-05-023-77/+77
| | | | | | | | | |
| * | | | | | | | | Integrate review comments on pull requestMridul Muralidharan2013-05-026-50/+47
| | | | | | | | | |
| * | | | | | | | | Fix some npe introduced accidentallyMridul Muralidharan2013-05-012-9/+23
| | | | | | | | | |