aboutsummaryrefslogtreecommitdiff
path: root/external
diff options
context:
space:
mode:
authorAndrew Or <andrew@databricks.com>2014-12-10 14:27:53 -0800
committerAndrew Or <andrew@databricks.com>2014-12-10 14:27:53 -0800
commit4f93d0cabe5d1fc7c0fd0a33d992fd85df1fecb4 (patch)
treeb5886d48a7bcacdd8b7b6423547367277829320a /external
parent447ae2de5d4c2af865fdb63f8b876b865de60f74 (diff)
downloadspark-4f93d0cabe5d1fc7c0fd0a33d992fd85df1fecb4.tar.gz
spark-4f93d0cabe5d1fc7c0fd0a33d992fd85df1fecb4.tar.bz2
spark-4f93d0cabe5d1fc7c0fd0a33d992fd85df1fecb4.zip
[SPARK-4759] Fix driver hanging from coalescing partitions
The driver hangs sometimes when we coalesce RDD partitions. See JIRA for more details and reproduction. This is because our use of empty string as default preferred location in `CoalescedRDDPartition` causes the `TaskSetManager` to schedule the corresponding task on host `""` (empty string). The intended semantics here, however, is that the partition does not have a preferred location, and the TSM should schedule the corresponding task accordingly. Author: Andrew Or <andrew@databricks.com> Closes #3633 from andrewor14/coalesce-preferred-loc and squashes the following commits: e520d6b [Andrew Or] Oops 3ebf8bd [Andrew Or] A few comments f370a4e [Andrew Or] Fix tests 2f7dfb6 [Andrew Or] Avoid using empty string as default preferred location
Diffstat (limited to 'external')
0 files changed, 0 insertions, 0 deletions