aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Added the ability in block manager to remove blocks.Reynold Xin2012-12-139-170/+361
|
* Suppress shuffle block updates when a slave node comes back.Reynold Xin2012-12-102-12/+17
|
* Merge pull request #319 from mbautin/cdh4.1.2Matei Zaharia2012-12-101-2/+3
|\ | | | | Bump CDH version for the Hadoop 2 profile to 4.1.2
| * Bump CDH version for the Hadoop 2 profile to 4.1.2Mikhail Bautin2012-12-101-2/+3
| |
* | Merge pull request #320 from tomdz/masterMatei Zaharia2012-12-102-1/+3
|\ \ | | | | | | Shaded repl jar for hadoop1 profile needs to include hadoop classes
| * | Shaded repl jar for hadoop1 profile needs to include hadoop classesThomas Dudziak2012-12-102-1/+3
| |/
* | Make "run" script work with Maven buildsMatei Zaharia2012-12-101-5/+10
| |
* | Make sure the SSH key we copy to EC2 has permissions 600.Matei Zaharia2012-12-101-0/+1
|/ | | | SPARK-539 #resolve
* Merge pull request #317 from woggling/block-manager-heartbeatMatei Zaharia2012-12-106-62/+318
|\ | | | | Implement block manager heartbeat
| * Decrease BlockManagerMaster logging verbosityCharles Reiss2012-12-101-1/+2
| |
| * Use Akka scheduler for BlockManager heart beats.Charles Reiss2012-12-104-49/+41
| | | | | | | | Adds required ActorSystem argument to BlockManager constructors.
| * Changed default block manager heartbeat interval to 5 sCharles Reiss2012-12-091-1/+1
| |
| * Don't divide by milliseconds by 1000 more.Charles Reiss2012-12-061-1/+1
| |
| * map -> foreachCharles Reiss2012-12-061-1/+1
| |
| * Make LocalSparkCluster use distinct IPsCharles Reiss2012-12-061-2/+6
| |
| * Tests for block manager heartbeats.Charles Reiss2012-12-051-0/+68
| |
| * Add block manager heart beats.Charles Reiss2012-12-052-25/+193
| | | | | | | | | | | | | | | | | | | | | | | | | | Renames old message called 'HeartBeat' to 'BlockUpdate'. The BlockManager periodically sends a heart beat message to the master. If the manager is currently not registered. The master responds to the heart beat by indicating whether the BlockManager is currently registered with the master. Additionally, the master now also responds to block updates by indicating whether the BlockManager in question is registered. When the BlockManager detects (by heart beat or failed block update) that it stopped being registered, it reregisters and sends block updates for all its blocks.
| * Track block managers by hostname; handle manager removal.Charles Reiss2012-12-051-3/+27
| |
| * Actually put millis in _lastSeenMsCharles Reiss2012-12-051-1/+1
| |
| * Don't use bogus port number in notifyADeadHost().Charles Reiss2012-12-051-2/+1
| |
* | Merge pull request #318 from tomdz/masterMatei Zaharia2012-12-101-1/+3
|\ \ | | | | | | Minor tweaks to the debian build
| * | Minor tweaks to the debian buildThomas Dudziak2012-12-101-1/+3
| | |
* | | Use the same output directories that SBT had in subprojectsMatei Zaharia2012-12-104-3/+9
|/ / | | | | | | This will make it easier to make the "run" script work with a Maven build
* | Merge pull request #310 from tomdz/master-mavenizedMatei Zaharia2012-12-099-0/+1340
|\ \ | | | | | | Maven build setup
| * | Code review feedback fixThomas Dudziak2012-11-281-1/+12
| | |
| * | Updated versions in the pom.xml files to match current masterThomas Dudziak2012-11-275-6/+6
| | |
| * | Addressed code review commentsThomas Dudziak2012-11-275-6/+17
| | |
| * | Include the configuration templates in the debian packageThomas Dudziak2012-11-201-0/+11
| | |
| * | Added maven and debian build filesThomas Dudziak2012-11-209-0/+1307
| | |
* | | Search for a non-loopback address in Utils.getLocalIpAddressMatei Zaharia2012-12-085-10/+33
| |/ |/|
* | Merge pull request #316 from JoshRosen/fix/ec2-web-ui-linksMatei Zaharia2012-12-048-15/+58
|\ \ | | | | | | Use external addresses in standalone web UI when running on EC2
| * | Use external addresses in standalone WebUI on EC2.Josh Rosen2012-12-018-15/+58
|/ /
* | Merge pull request #314 from pwendell/quickstart-bugfixMatei Zaharia2012-11-281-2/+2
|\ \ | | | | | | Adding multi-jar constructor in quickstart
| * | Adding multi-jar constructor in quickstartPatrick Wendell2012-11-271-2/+2
|/ /
* | Merge pull request #313 from rxin/pde_size_compressMatei Zaharia2012-11-272-33/+37
|\ \ | | | | | | Added a partition preserving flag to MapPartitionsWithSplitRDD.
| * | Added a partition preserving flag to MapPartitionsWithSplitRDD.Reynold Xin2012-11-272-33/+37
| | |
* | | Added zip to Java APIMatei Zaharia2012-11-272-0/+25
| | |
* | | Added a zip() operation for RDDs with the same shape (number ofMatei Zaharia2012-11-273-0/+75
| | | | | | | | | | | | partitions and number of elements in each partition)
* | | Use hostname instead of IP in deploy scripts to let Akka connect properlyMatei Zaharia2012-11-271-13/+2
| | |
* | | Merge branch 'master' of github.com:mesos/sparkMatei Zaharia2012-11-276-51/+185
|\ \ \
| * \ \ Merge pull request #311 from woggling/map-output-npeMatei Zaharia2012-11-272-1/+52
| |\ \ \ | | | | | | | | | | Fix NullPointerException when map output unregistered from MapOutputTracker twice
| | * | | Fix NullPointerException when unregistering a map output twice.Charles Reiss2012-11-271-1/+1
| | | | |
| | * | | Tests for MapOutputTracker.Charles Reiss2012-11-271-0/+51
| | | |/ | | |/|
| * | | Merge pull request #312 from rxin/pde_size_compressMatei Zaharia2012-11-272-15/+18
| |\ \ \ | | |/ / | |/| / | | |/ For size compression, compress non zero values into non zero values.
| | * For size compression, compress non zero values into non zero values.Reynold Xin2012-11-272-15/+18
| |/
| * Merge pull request #309 from admobius/use-boto-config-fileMatei Zaharia2012-11-191-11/+20
| |\ | | | | | | Improved use of Boto
| | * Allow Boto to use the other config options it supports, and gracefullyPeter Sankauskas2012-11-191-11/+20
| | | | | | | | | handling Boto connection exceptions (like AuthFailure)
| * | Merge pull request #308 from admobius/multi-zoneMatei Zaharia2012-11-191-22/+59
| |\ \ | | | | | | | | Let EC2 script launch slaves in multiple availability zones
| | * | Adding comment about additional bandwidth chargesPeter Sankauskas2012-11-171-1/+2
| | | |
| | * | Giving the Spark EC2 script the ability to launch instances spreadPeter Sankauskas2012-11-161-22/+58
| | | | | | | | | | | | | | | | | | | | across multiple availability zones in order to make the cluster more resilient to failure